spider_crawl, spider_scrape and spider_unblocker take the same very wide parameter set, which is what lets a single call express an actual crawl policy: robots respect, proxy and country, session and cookies, selector-based inclusion and exclusion, output format, and a webhook with background execution instead of a blocking wait. The prompt-driven variants trade all of that for a sentence, and each says in its own description that it needs an active AI subscription — so a key that works fine for spider_scrape can return nothing for spider_ai_scrape. The core tools run on pay-per-use credits, and which call you pick is what moves the bill: spider_transform makes no network request at all, while spider_unblocker escalates through anti-bot bypass to fetch a single page.
A Spider Cloud client with 13 tools covering site crawls, single-page scrapes, web search, link extraction, screenshots, anti-bot unblocking, local HTML transformation, credit balance and prompt-driven variants of the crawl, scrape, search, browse and link paths.
- Crawling and scraping with the whole request surface exposed: spider_crawl walks a site under depth, limit and budget while spider_scrape fetches a single URL, and each takes the same long parameter set — return_format, proxy and country_code, cookies and session, request_timeout and request_max_retries, viewport and user_agent, readability and clean_html, blocking for ads, analytics, images and stylesheets, css_extraction_map and root_selector, respect_robots, and webhooks with run_in_background.
- Cheaper reads when the body is not the point: spider_links extracts a page's links without fetching content, and spider_transform converts HTML you already hold into markdown or text with no network request at all.
- Search and capture: spider_search returns results and can fetch page content for them, with country, language, location and auto_pagination controls; spider_screenshot returns base64 or binary images with full_page, viewport and omit_background.
- Getting past a block: spider_unblocker reaches bot-protected content with fingerprinting and proxy rotation over the same scrape parameters, with retry and stealth escalation behind it.
- Prompt-driven variants on a separate plan: spider_ai_crawl guides a crawl from a natural-language prompt, spider_ai_scrape returns structured JSON without CSS selectors, spider_ai_search ranks by intent, spider_ai_browser performs described browser actions, and spider_ai_links finds and categorises links — each stating that an active AI subscription plan is required.
- Balance before the run: spider_get_credits returns the available Spider API credit balance.
A Spider API key, supplied as SPIDER_API_KEY. The spider_ai_* tools additionally require an active Spider AI subscription plan.
One command plus a key — npx -y spider-cloud-mcp, then supply credentials