crawl_web
Crawl a specific webpage and extract its content in various formats including Markdown, raw HTML, and AI-enhanced HTML.
Search or fetch a page through Crawleo and get the body back as markdown, raw HTML or enhanced HTML, with country, language and device controls on the search side.
LABSCO SUMMARY
Both tools return the body in the same three shapes — markdown, raw HTML and enhanced HTML — so a page arrives ready for a model to read rather than needing conversion afterwards. The search tool is the one carrying locale: cc, geolocation, setLang and device change what comes back, and max_pages decides how far it goes. With 2 tools there is nothing else in the surface: no crawl of a whole site, no queue, no extraction schema.
An MCP server in front of Crawleo's search and crawl service.
CRAWLEO_API_KEY, since both tools run against Crawleo's own service.
One command plus a key — npx crawleo-mcp, then supply credentials
USE CASES
Fetch one URL and take it back as markdown instead of raw markup, or as HTML when the structure is what matters.
Run the same query with the country, locale and device of the audience you are checking for, across more than one page of results.
Counted once with only startup setup. Other versions, modes or optional credentials may expose a different list.
crawl_webCrawl a specific webpage and extract its content in various formats including Markdown, raw HTML, and AI-enhanced HTML.
search_webSearch the web using Crawleo's AI-powered search engine. Returns results with optional AI-enhanced HTML, markdown content, and structured data.
BEFORE YOU INSTALL
2 tools when we connected. The npm build registers search_web and crawl_web, while the documentation uses web.search and web.crawl; check tools/list on the route you use. Both are read-only.
CONTEXT COST
Rated by the size of the 2 tool definitions that ride along on every turn, not by how many there are
INSTALLATION
npx crawleo-mcphttps://api.crawleo.dev/mcpAdd this hosted endpoint to your MCP client using the credentials listed under Requirements.
{
"mcpServers": {
"crawleo": {
"url": "https://api.crawleo.dev/mcp",
"headers": {
"Authorization": "Bearer <YOUR_CRAWLEO_API_KEY>"
}
}
}
}Global: ~/.cursor/mcp.json · Per project: .cursor/mcp.json in the project folder
{
"mcpServers": {
"crawleo": {
"serverUrl": "https://api.crawleo.dev/mcp",
"headers": {
"Authorization": "Bearer <YOUR_CRAWLEO_API_KEY>"
}
}
}
}~/.codeium/windsurf/mcp_config.json
{
"servers": {
"Crawleo": {
"type": "http",
"url": "https://api.crawleo.dev/mcp",
"headers": {
"Authorization": "Bearer <YOUR_CRAWLEO_API_KEY>"
}
}
}
}{
"type": "mcp",
"server_label": "Crawleo",
"server_url": "https://api.crawleo.dev/mcp",
"server_description": "Crawleo MCP Server - Real-Time Web Knowledge for AI",
"authorization": "<YOUR_CRAWLEO_API_KEY>",
"allowed_tools": [
"web.search",
"web.crawl"
],
"require_approval": "always"
}{
"mcpServers": {
"crawleo": {
"command": "npx",
"args": [
"crawleo-mcp"
],
"env": {
"CRAWLEO_API_KEY": "<YOUR_CRAWLEO_API_KEY>"
}
}
}
}<YOUR_CRAWLEO_API_KEY> is your Crawleo API key (it starts with sk_; copy it from your dashboard at crawleo.dev).
Clone the repository and build the image
git clone https://github.com/Crawleo/Crawleo-MCP.git
cd Crawleo-MCP
docker build -t crawleo-mcp .Add this to your client's MCP configuration
<YOUR_CRAWLEO_API_KEY> is your sk_ key from the crawleo.dev dashboard.
{
"mcpServers": {
"crawleo": {
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"-e",
"CRAWLEO_API_KEY=<YOUR_CRAWLEO_API_KEY>",
"crawleo-mcp"
]
}
}
}Clone and build the server
git clone https://github.com/Crawleo/Crawleo-MCP.git
cd Crawleo-MCP
npm install
npm run buildAdd this to your client's MCP configuration
<YOUR_PATH_TO_CRAWLEO_MCP> is the folder you cloned into, and <YOUR_CRAWLEO_API_KEY> is your sk_ key.
{
"mcpServers": {
"crawleo": {
"command": "node",
"args": [
"<YOUR_PATH_TO_CRAWLEO_MCP>/dist/index.js"
],
"env": {
"CRAWLEO_API_KEY": "<YOUR_CRAWLEO_API_KEY>"
}
}
}
}