Local-model extraction by default is the unusual choice here.
extract_with_llm points at Ollama on localhost unless told otherwise, so schema extraction costs nothing per token and the page never leaves the machine. Around that sits a full ladder — a plain fetch at one end, a stealth browser and an autonomous agent at the other — so you escalate only when a site forces it.
What it is
A web toolkit for agents: fetch and clean a single page, crawl a site, search the web, check real Google organic rank, run multi-source research, and drive a real browser through logins and forms when a page demands it.
What you get
- Single pages — fetch_url, scrape for several formats in one call, extract_content for a clean article, extract_text, extract_metadata, extract_links, process_document for PDFs
- At scale — batch_scrape for 2 to 50 URLs, map_site to list a domain's URLs from its sitemap, crawl_deep to walk a site under depth, page and pattern limits, get_batch_results to page through a finished job
- Structured output — extract_structured against a JSON schema, scrape_structured with CSS selectors, scrape_template for known sites, extract_with_llm against a local Ollama model, list_ollama_models to see what is installed
- Search and research — search_web, serp_rank for a real Google organic position, deep_research across sources, and agent which plans and answers without being handed URLs
- Hard pages — scrape_with_actions for logins, clicks, forms and dynamic content; stealth_mode with randomised fingerprints for sites behind bot detection; localization to emulate a country, locale and timezone
- Analysis and monitoring — analyze_content for language, sentiment, topics, entities and readability, summarize_content, track_changes with baselines and diffs, generate_llms_txt
Requirements
The npm package crawlforge-mcp-server over stdio. The same package installs a crawlforge-setup command that walks the key setup and can configure Claude Code and Cursor for you — run it after the global install. CRAWLFORGE_API_KEY is required — a free account starts with 1,000 credits that never expire and roll over. DATAFORSEO_LOGIN and DATAFORSEO_PASSWORD are optional and enable serp_rank only, billed to your own DataForSEO account. extract_with_llm defaults to a local Ollama model at http://localhost:11434, so no LLM key is needed and the page content stays on your machine; cloud providers are opt-in.
Setup effort
One command plus a key — npm install -g crawlforge-mcp-server, then supply credentials