Most fetch tools hand back an error the moment a page throws a bot challenge. Here the challenge is a stage in the pipeline — render it in Chromium, retry through a proxy, or open a real window so you solve it once and the agent carries on alone.
A single self-hostable Rust binary that gives an agent search, page fetching, structured extraction and stateful browser automation over MCP stdio, with an optional HTTP server on port 5000. When a page resists, it escalates in stages: native retrieval, then Chromium CDP rendering, then a visible browser you drive yourself.
- `web_search` for URL discovery, or `web_search(include_content=true)` to search and scrape the top results in one round-trip
- `web_fetch` in `single`, `batch` or `crawl` mode, and `extract_fields` for structured extraction
- `scout_browser_automate` — one tool that takes an array of steps: navigation, hover, click, type, waits, locator assertions, tabs, screenshots and PDF, file upload, form fill, dialog policy, route mocking, console and network capture, and cookie or storage CRUD
- `scout_agent_profile_auth` opens the agent's own profile in a visible window so you clear a CAPTCHA or a Google login once; the cookies are saved and the agent goes back to running silently
- `hitl_web_fetch` with `auth_mode="challenge"` for CAPTCHA walls or `auth_mode="auth"` for login walls, and `visual_scout` to judge whether a page needs that first
- `memory_search` over a LanceDB-backed history of past research, so a repeat question can skip the live fetch
- `deep_research` for multi-hop search, scrape and synthesis against any OpenAI-compatible endpoint, including a local one
A prebuilt binary from GitHub Releases, or a source build with `protoc` on your PATH. The timeout guards belong in the MCP config — they are what stops a stalled page or browser launch from holding the session open. Use `RUST_LOG=warn`; at `info` the server floods stderr and confuses clients. `memory_search` needs `LANCEDB_URI` pointed at a directory, and `deep_research` needs an OpenAI-compatible endpoint. MIT.