scrapeLABSCO SUMMARY
Skipping the headless browser is a real engineering choice, not a shortcut.
Most extractors drive Chrome, which works and costs hundreds of megabytes of memory and a second or two per page. Matching a browser's TLS fingerprint gets past the fetch-level blocking that motivates the browser in the first place, without the browser — much faster and much lighter, at the cost of pages that genuinely need JavaScript to render. Know which kind of page you are fetching. Also read the licence before embedding it in a product: AGPL is a deliberate choice by the author.
What it is
A content extractor for retrieval pipelines. Its distinguishing choice is fetching with browser-grade TLS fingerprinting rather than driving a real browser.
What you get
- A page returned as clean markdown, or as structured JSON
- Output shaped for retrieval pipelines rather than for reading
- The same engine available as a command-line tool, a REST API and SDKs, not only as a server
Requirements
Runs locally as a binary. AGPL-3.0 licensed, which matters if you plan to build a service on top of it.
Setup effort
One command — npx create-webclaw
USE CASES
01
Feeding a retrieval pipeline
Clean markdown rather than page chrome.
02
Pulling fields off a page
Structured JSON instead of prose.
03
Fetching at speed
No browser to start, so the per-page cost is small.
10 tools
Listed in project docs. Your client may show a different list.
crawlmapbatchextractsummarizediffbrandsearchresearchBEFORE YOU INSTALL
What to know first
Tools you get
10 tools named in the project's documentation.
Where it runs
All 4 documented clients run the server on your machine.
INSTALLATION
Connect your client
Server setup
TERMINAL
npx create-webclawMCP Clients
Claude Code, Claude Desktop, Cursor, Windsurf
JSON
{
"mcpServers": {
"webclaw": {
"command": "npx",
"args": [
"-y",
"@webclaw/mcp"
]
}
}
}
