academic_searchLABSCO SUMMARY
Built for the pages that do not come back on the first try.
Having stealth, geolocation and retry as separate tools means the failure has an answer instead of a shrug: blocked becomes stealth, region-locked becomes geolocation, flaky becomes retry with a count. Pairing that with academic search across three databases makes it a research server rather than a scraper — and the content-limit controls matter, because a full crawl will otherwise fill your context.
What it is
A research server combining web crawling with academic literature search. The crawling side is built on Crawl4AI and offers several modes for sites that resist a plain fetch; the search side queries scholarly databases directly. The README is in Chinese.
What you get
academic_searchqueries Google Scholar, arXiv or PubMed by passing the source as its second argumentcrawlfor an ordinary page;crawl_stealthfor one that blocks automationcrawl_with_geolocationpresents a chosen location for region-restricted contentcrawl_with_retrytakes amax_retriescount for flaky sitescrawl_with_intelligencepicks its approach from the page rather than a fixed modeconfigure_crawl_settings,quick_config_content_limitandquick_config_word_thresholdtune how much content comes backsystem_statusreports whether the server is healthy — the first thing to call after setupexperimental_claude_analysisfor content analysis, off unless you configure it
Requirements
Python 3.12+ with uv or pip. After installing dependencies, run crawl4ai-setup and install the Chromium browser — the crawler drives a real browser, so this is a heavier install than a fetch-based scraper. Configuration is shown for Amazon Q Developer, pointing at server_v9.py with absolute paths. No account or key for crawling and academic search; the Claude analysis tool needs its own API configuration.
Setup effort
Build from source — git clone https://github.com/ddipass/context-scraper-mcp-server.git
USE CASES
01
Reading a page that blocks ordinary scrapers
Switch from the basic crawl to stealth mode instead of giving up on the URL.
02
Gathering literature on a topic
Search arXiv, PubMed or Google Scholar by name and work from the results.
03
Keeping a long page from swamping the conversation
Set the content display limit and word threshold before crawling.
11 tools
Listed in project docs. Your client may show a different list.
crawlcrawl_stealthcrawl_with_geolocationcrawl_with_retrycrawl_with_intelligenceconfigure_crawl_settingsquick_config_content_limitquick_config_word_thresholdsystem_statusexperimental_claude_analysisBEFORE YOU INSTALL
What to know first
Tools you get
11 tools named in the project's documentation. The docs folder describes older server versions with tools such as crawl_with_selector and crawl_multiple; those aren't in server_v9.py, the server you configure.
Requires
Python 3.12+
Where it runs
The documented client runs the server on your machine.
INSTALLATION
Connect your client
You clone this one yourself.
Server setup
TERMINAL
# 1. Clone the project
git clone https://github.com/ddipass/context-scraper-mcp-server.git
cd context-scraper-mcp-server
# 2. Sync dependencies with uv
uv sync
# 3. Activate the virtual environment
source .venv/bin/activate
# 4. Run the Crawl4AI setup
crawl4ai-setupMCP Clients
Amazon Q Developer
AMAZON Q DEVELOPER · JSON
{
"mcpServers": {
"ContextScraper": {
"command": "<YOUR_PATH_TO_CONTEXT_SCRAPER_MCP_SERVER>/.venv/bin/mcp",
"args": [
"run",
"<YOUR_PATH_TO_CONTEXT_SCRAPER_MCP_SERVER>/server_v9.py"
],
"cwd": "<YOUR_PATH_TO_CONTEXT_SCRAPER_MCP_SERVER>"
}
}
}
