scrape_article_contentLABSCO SUMMARY
A browser-automation surface with the cookie-banner problem already solved.
The 29 tools match Microsoft's Playwright MCP surface, so anything written against that works here. What you get on top is the consent layer — the reason article scraping usually returns a wall of banner markup instead of the article.
What it is
A Playwright-based server with 29 tools: three for article extraction with cookie-consent handling, and the rest a full browser-automation surface — navigation, forms, tabs, network monitoring and element discovery.
What you get
- scrape_article_content — title, author, date, summary and body, in text, HTML or markdown
- get_page_screenshot and handle_cookie_consent — a screenshot after the banner is dealt with, or a consent check on its own
- browser_navigate, browser_click, browser_type, browser_hover, browser_select_option, browser_press_key, browser_handle_dialog, browser_file_upload, browser_close
- browser_pdf_save, browser_console_messages, browser_resize, browser_snapshot, browser_install, browser_generate_playwright_test
- manage_tabs, monitor_network, drag_drop and navigate_history for session-based work across pages
- browser_find_text, browser_find_element, browser_describe_element, browser_annotate_page, browser_get_element_text, browser_wait_for_page_state, browser_execute_javascript
- Cookie-consent handling across 30+ languages and 25+ consent frameworks
Requirements
Node and a Playwright install, for the source build your client starts. The README also offers a Docker image that serves HTTP on port 3001, with a health check at /health, but its Claude Desktop setup runs the source build. No credentials.
Setup effort
Build from source — git clone https://github.com/descoped/mcp-web-scraper.git
USE CASES
01
Pulling an article that hides behind a consent wall
One call handles the banner and returns title, author, date and body in whichever formats you ask for.
02
Filling in a form on a JavaScript-heavy page
Navigate, type, select and press keys against a live session, with dialogs and file uploads covered.
03
Finding out what a page is actually requesting
Start network monitoring on a session, do the thing, then read back the requests with timing, status codes and domains.
04
Working on an element you can only describe
Find it by description or by the text near it, get a description of what it is, then act on it.
29 tools
Listed in project docs. Your client may show a different list.
get_page_screenshothandle_cookie_consentbrowser_navigatebrowser_clickbrowser_typebrowser_hoverbrowser_select_optionbrowser_press_keybrowser_handle_dialogbrowser_file_uploadbrowser_closebrowser_pdf_savebrowser_console_messagesbrowser_resizebrowser_snapshotbrowser_installbrowser_generate_playwright_testmanage_tabsmonitor_networkdrag_dropnavigate_historybrowser_find_textbrowser_find_elementbrowser_describe_elementbrowser_annotate_pagebrowser_get_element_textbrowser_wait_for_page_statebrowser_execute_javascriptBEFORE YOU INSTALL
What to know first
Tools you get
29 tools named in the project's documentation.
Requires
Node.js and npm
Where it runs
The documented client runs the server on your machine.
INSTALLATION
Connect your client
Server setup
TERMINAL
git clone https://github.com/descoped/mcp-web-scraper.git
cd mcp-web-scraper
npm install
npm run buildMCP Clients
Claude Desktop
CLAUDE DESKTOP · JSON
{
"mcpServers": {
"mcp-web-scraper": {
"command": "node",
"args": [
"<YOUR_PATH_TO_MCP_WEB_SCRAPER>/dist/server.js"
],
"env": {
"BROWSER_POOL_SIZE": "3"
}
}
}
}<YOUR_PATH_TO_MCP_WEB_SCRAPER> is the full path of your clone — npm run build creates dist/server.js there.
