fetch-pageLABSCO SUMMARY
Site-level questions, not just page-level ones.
fetch-page covers the single-page case, but the reason to install this one is crawl-site, check-links and generate-site-map, which answer questions about a whole site. Crawl depth is capped at 5, which keeps an exploratory crawl from turning into an open-ended scrape.
What it is
A web content scanning server: page fetching, link analysis, recursive crawling and sitemap generation over stdio.
What you get
- A page fetched and converted to markdown, optionally narrowed by a CSS selector (fetch-page)
- Every link on a page with its text, filterable by base URL and capped by a limit (extract-links)
- A recursive crawl from a starting URL, to a depth between 0 and 5 (crawl-site)
- Broken links on a page identified (check-links)
- URLs matching a JavaScript pattern you supply (find-patterns)
- An XML sitemap generated for a site (generate-site-map)
Requirements
Node.js with the project built from source using npm install and npm run build, or installed for Claude Desktop through Smithery. Runs on stdio.
Setup effort
Build from source — git clone https://github.com/bsmi021/mcp-server-webscan.git
USE CASES
01
Reading a page as clean markdown
Fetch it, and use a CSS selector when you only want one part of the document.
02
Finding the broken links on a site you maintain
Crawl the site, then check the links on the pages that come back.
03
Mapping what a site actually contains
Crawl to the depth you want and generate an XML sitemap from it.
04
Collecting every link of a certain shape
Extract the links, then filter them by the pattern you are looking for.
05
Pulling one section out of a long page
Fetch with a selector so only that section becomes context.
6 tools
Listed in project docs. Your client may show a different list.
extract-linkscrawl-sitecheck-linksfind-patternsgenerate-site-mapBEFORE YOU INSTALL
What to know first
Tools you get
6 tools named in the project's documentation.
Requires
- node on PATH
- Node.js >= 18
Where it runs
The documented client runs the server on your machine.
INSTALLATION
Connect your client
You clone and build this one yourself.
Server setup
Clone the repository
TERMINAL
git clone https://github.com/bsmi021/mcp-server-webscan.git
cd mcp-server-webscanInstall dependencies
TERMINAL
npm installBuild the project
TERMINAL
npm run buildMCP Clients
Claude Desktop
CLAUDE DESKTOP · JSON
{
"mcpServers": {
"webscan": {
"command": "node",
"args": [
"<YOUR_PATH_TO_MCP_SERVER_WEBSCAN>/build/index.js"
],
"env": {
"NODE_ENV": "development",
"LOG_LEVEL": "info"
}
}
}
}
