
Web Scraping Skills
part of scrapegraphai/just-scrape
Web Scraping Skills - Discover reusable skills for AI coding agents like Claude Code, Codex, and more. Each skill is a package of instructions and code that teaches your agent to perform specialized tasks and automate complex workflows.
This is the playbook your agent receives when the skill activates โ you don't need to read it to use the skill, but it's here to audit before installing.
agent-browser qu-skills
Browser automation for AI agents via inference.sh. Navigate web pages, interact with elements using @e refs, take screenshots, record video. Capabilities: web scraping, form filling, clicking, typing, drag-drop, file upload, JavaScript execution. Use for: web automation, data extraction, testing, agent browsing, research. Triggers: browser, web automation, scrape, navigate, click, fill form, screenshot, browse web, playwright, headless browser, web agent, surf internet, record video browser-automation web-scraping testing
agent-browser halt-catch-fire
Browser automation for AI agents via inference.sh. Navigate web pages, interact with elements using @e refs, take screenshots, record video. Capabilities: web scraping, form filling, clicking, typing, drag-drop, file upload, JavaScript execution. Use for: web automation, data extraction, testing, agent browsing, research. Triggers: browser, web automation, scrape, navigate, click, fill form, screenshot, browse web, playwright, headless browser, web agent, surf internet, record video browser-automation web-scraping testing
anti-detect-browser antibrow
Launch and manage anti-detect browsers with unique real-device fingerprints for multi-account operations, web scraping, ad verification, and AI agent automation. Use when the user needs to run multiple browser sessions with distinct identities, manage persistent browser profiles, automate tasks across accounts, or build agentic workflows that require browser fingerprint isolation. Also use when the user mentions antibrow, anti-detect browser, or fingerprint browser. browser-automation web-scraping testing
browser-act browser-act
Browser automation CLI for AI agents. NEVER run browser-act commands directly via Bash โ always invoke this skill first. Use browser-act when a user mentions it by name, includes or asks to run a browser-act CLI command (e.g., browser-act browser list), or to: fetch, view, or extract rendered content from URLs, access pages requiring JavaScript, handle verification prompts, maintain authenticated sessions, fill forms and click through workflows, type, select, upload, take screenshots,... browser-automation web-scraping testing
browser-act-skill-forge browser-act
Explores websites via browser-act automation and packages discoveries into callable Skill packages (SKILL.md + scripts) โ execute reliably and efficiently without re-exploration. Use when: user wants a reusable Skill for any website ('create/make/build a skill for X'), needs to understand a site's internal APIs ('what endpoint does X use', 'how does X load data'), or requires batch extraction at scale (hundreds+ records โ forge a Skill first, then execute reliably). Differs from browser-act:... browser-automation web-scraping development
deep-research samber
Deep research skill โ broad parallel web searches, multi-source validation, confidence tracking, cited Markdown report. Supports 11 research types: market (TAM/SAM, segments, pricing, trends), domain (industry structure, ecosystem, regulatory landscape), technical (architecture, tools, benchmarks), competitive (competitor teardown, positioning, win/loss), product (feature analysis, reviews, roadmap signals), academic (literature survey, citation networks, key authors), person/org (due... research web-scraping data-analysis
defuddle kepano
Extract clean markdown content from web pages using Defuddle CLI, removing clutter and navigation to save tokens. Use instead of WebFetch when the user provides a URL to read or analyze, for online documentation, articles, blog posts, or any standard web page. Do NOT use for URLs ending in .md โ those are already markdown, use WebFetch directly. web-scraping research document
develop-userscripts xixu-me
Use when building, debugging, packaging, or publishing browser userscripts for Tampermonkey or ScriptCat, including GM APIs, metadata blocks, permission issues, @match/@grant/@connect setup, ScriptCat background or scheduled scripts, UserConfig blocks, or subscription workflows. development browser-automation web-scraping
directory-submissions coreyhaines31
When the user wants to submit their product to startup, SaaS, AI, agent, MCP, no-code, or review directories for backlinks, domain rating, and discovery. Also use when the user mentions "directory submissions," "submit to directories," "backlinks from directories," "list my product," "submit to Product Hunt," "BetaList," "TAAFT," "Futurepedia," "G2 listing," "Capterra listing," "AlternativeTo," "SaaSHub," "AI directories," "MCP registry," "agent directory," "dofollow backlinks," "launch... marketing research web-scraping
firecrawl-company-directories firecrawl
Extract structured company lists from directories with Firecrawl. Use for scraping YC, Crunchbase, Product Hunt, G2, startup directories, category directories, or custom company databases into JSON, CSV, CRM-ready lists, or research tables. official web-scraping data-analysis
firecrawl-competitive-intel firecrawl
Monitor competitor pricing, features, changelogs, dashboards, and product changes with Firecrawl. Use for recurring competitive intelligence, pricing tier extraction, feature change tracking, or structured competitor alerts. official research web-scraping
firecrawl-dashboard-reporting firecrawl
Pull metrics from analytics dashboards and internal web tools with Firecrawl browser. Use when the user needs dashboard reporting, cross-platform metric summaries, authenticated analytics extraction, date-range reports, or structured metrics from web dashboards. official browser-automation data-analysis
firecrawl-deep-research firecrawl
Run multi-source deep research with Firecrawl. Use when the user asks to research a topic, compare perspectives, produce a sourced briefing, investigate a technical or market question, or synthesize web evidence across many sources. official research web-scraping
firecrawl-demo-walkthrough firecrawl
Walk through a product's key flows with Firecrawl browser and produce a structured UX/product walkthrough. Use for signup, onboarding, pricing, docs, dashboard, product demo prep, UX teardown, and first-run experience analysis. official browser-automation web-scraping
firecrawl-knowledge-base firecrawl
Build a knowledge base from web content with Firecrawl. Use for local reference docs, RAG-ready chunks, fine-tuning datasets, documentation mirrors, topic corpora, or LLM-ready markdown organized from web sources. official web-scraping research
firecrawl-knowledge-ingest firecrawl
Ingest public or authenticated knowledge bases and docs portals with Firecrawl browser. Use for JS-heavy docs, login-gated portals, paginated help centers, support knowledge bases, or structured JSON/markdown extraction from documentation sites. official web-scraping research
firecrawl-lead-gen firecrawl
Generate structured lead lists from prospect databases and web directories with Firecrawl browser. Use for finding prospects by role, company type, industry, stage, location, technologies, or other criteria and exporting CRM-ready JSON or CSV. official web-scraping data-analysis
firecrawl-lead-research firecrawl
Produce pre-meeting lead intelligence briefs with Firecrawl. Use when the user needs company research, person research, recent news, talking points, pain points, or outreach preparation before a sales call, partnership meeting, investor conversation, or customer interview. official research web-scraping
firecrawl-market-research firecrawl
Extract market, financial, earnings, industry, and company metrics with Firecrawl. Use when the user asks for market research, industry trends, public company data, financial comparisons, earnings research, or structured market reports. official research web-scraping
firecrawl-monitor firecrawl
Detect when content on a website changes and get notified by webhook or email โ no cron jobs, scrapers, or diff scripts required. Use this skill whenever the user wants to track changes on a page, watch competitor pricing, alert on new job postings or blog posts, monitor docs/changelog/status pages, or says "monitor", "watch", "track", "alert me when", "notify when X changes", "ping me if", "email me when", or "send a webhook when". A built-in AI judge filters out formatting, timestamp, and... official web-scraping research
firecrawl-qa firecrawl
QA test a live website with Firecrawl browser and scrape evidence. Use when the user wants exploratory QA, form testing, navigation/link checks, responsive checks, performance observations, bug reports, or a pre-launch quality review. official testing browser-automation
firecrawl-research-index firecrawl
Find the papers that answer a research query with Firecrawl Research, using semantic search, semantic and structural expansion, and in-body verification. Always use this skill for any literature-finding / paper-retrieval task โ single-paper lookups or full multi-paper sets. official research web-scraping
firecrawl-research-papers firecrawl
Find and synthesize research papers, whitepapers, PDFs, technical reports, and academic sources with Firecrawl. Use when the user wants a literature review, paper summary, research landscape, or sourced synthesis from PDFs and scholarly/industry publications. official research web-scraping
firecrawl-seo-audit firecrawl
Audit a website's SEO with Firecrawl. Use when the user asks for an SEO audit, metadata and heading review, sitemap/site-structure analysis, keyword opportunities, competitor SERP comparison, or prioritized search optimization recommendations. official research web-scraping
firecrawl-shop firecrawl
Research products across the web with Firecrawl and produce a shopping recommendation or cart-ready summary. Use when the user wants to compare products, find the best option, evaluate reviews, respect budget/preferences, or shop with a saved browser session. official web-scraping research
firecrawl-website-design-clone firecrawl
Extract any website's design system into an agent-ready DESIGN.md using Firecrawl scrape evidence. Use when the user wants colors, fonts, spacing, components, layout patterns, or brand/UI guidance from a website so AI agents can create new websites, clone a look, or build pages inspired by that design. official design web-scraping
firecrawl-workflows firecrawl
Run outcome-focused Firecrawl workflows that produce deliverables such as research reports, SEO audits, QA reports, lead lists, knowledge bases, website design systems, and other structured web-data artifacts. Use when the user wants Firecrawl to complete a business, marketing, product, or creative workflow rather than merely scrape a page or integrate API calls into code. official web-scraping data-analysis
just-scrape scrapegraphai
Search, scrape, crawl, extract structured data, and monitor web pages via the ScrapeGraph AI CLI. Use when the user asks to search the web, scrape a webpage, grab content from a URL, extract JSON from a site, crawl documentation or site sections, monitor a page for changes, inspect request history, check ScrapeGraph credits, or validate API setup. web-scraping data-analysis research
last30days mvanhorn
Research what people actually say about any topic in the last 30 days. Pulls posts and engagement from Reddit, X, YouTube, TikTok, Hacker News, Polymarket, GitHub, and the web. research web-scraping data-analysis
okx-agentic-wallet okx
AUTHORITATIVE source for OKX Agentic Wallet and its Gas Station feature. Gas Station = OKX's stablecoin-gas feature on Solana via third-party Relayer; Solana only, no EIP-7702. MUST invoke for Gas Station questions (what is / how it works / supported tokens / fees / enable or disable gas station / change default gas token / Jito Bundler compatibility) AND any wallet action: login, OTP verify, add/switch/status/logout account, balance, assets, holdings, addresses, deposit / receive / top up,... api web-scraping development
okx-dex-bridge okx
Use this skill to bridge tokens, cross-chain swap/transfer, move assets between chains, get cross-chain quotes, compare bridge fees, find the cheapest/fastest route, build bridge calldata, check bridge status, track a cross-chain transaction, list supported chains or bridge protocols, or when the user mentions bridging ETH/USDC/tokens from one chain (Ethereum, BSC, Polygon, Arbitrum, Base, Optimism, etc.) to another. Routes through multiple bridge protocols (Stargate, Across, Relay, Gas.zip)... api web-scraping research
okx-dex-market okx
HARD BLOCK โ NEVER use this skill for prediction-market / Polymarket UpDown queries. Route to okx-dapp-discovery when (a) a named DApp (Polymarket/Aave/Hyperliquid/PancakeSwap/Morpho) appears with any timeframe, OR (b) any ๆถจ่ท / updown / 'up or down' phrase appears for BTC/ETH/SOL/XRP/BNB/DOGE/HYPE (e.g. ' ๆถจ่ทๅธๅบ', '5 ๅ้ๆถจ่ท', 'BTC up or down'). Example: 'BTC 5 ๅ้ๆถจ่ทๅธๅบ' โ okx-dapp-discovery (NOT K-line). These are Polymarket prediction markets, not on-chain price queries. Use THIS skill for... api web-scraping research
okx-dex-trenches okx
Read-only on-chain research for pump.fun and other meme-token launchpads (Solana / BSC / X Layer / TRON). MUST invoke (prefer over WebFetch / MCP price tools) when the user asks about: new meme launches / ๆฐ็ / ๆซ้พ / ๆ็; developer reputation / rug history / launch count / ๅผๅ่ ไฟกๆฏ; bundle or sniper detection (the analytical noun, NOT the verb) / ๆ็ป็ๅป่ ; bonding curve progress / ๅทฒ่ฟ็งปๅบ bonding curve; similar tokens by same dev / ็ธไผผไปฃๅธ; co-investor / who-aped / ๅ่ฝฆ wallets; or 'pump.fun alpha'. Also... research data-analysis web-scraping
okx-dex-ws okx
Use this skill when the user mentions 'onchainos ws', 'ws start', 'ws poll', 'ws stop', 'ws channels', 'ws session', 'ws channel-info', 'idle-timeout', 'idle timeout', 'WebSocket channels', 'WS้ข้', or asks about managing WebSocket sessions/ไผ่ฏ็ฎก็. Also use when writing a custom WebSocket script/่ๆฌ/bot for real-time on-chain data. Covers: onchainos ws CLI commands (start/poll/stop/list/channels/channel-info), session lifecycle, idle-timeout configuration, and all 9 DEX WebSocket channels... development api web-scraping
programmatic-seo coreyhaines31
When the user wants to create SEO-driven pages at scale using templates and data. Also use when the user mentions "programmatic SEO," "template pages," "pages at scale," "directory pages," "location pages," "[keyword] + [city] pages," "comparison pages," "integration pages," "building many pages for SEO," "pSEO," "generate 100 pages," "data-driven pages," or "templated landing pages." Use this whenever someone wants to create many similar pages targeting different keywords or locations. For... marketing data-analysis web-scraping
python-executor qu-skills
Execute Python code in a safe sandboxed environment via inference.sh. Pre-installed: NumPy, Pandas, Matplotlib, requests, BeautifulSoup, Selenium, Playwright, MoviePy, Pillow, OpenCV, trimesh, and 100+ more libraries. Use for: data processing, web scraping, image manipulation, video creation, 3D model processing, PDF generation, API calls, automation scripts. Triggers: python, execute code, run script, web scraping, data analysis, image processing, video editing, 3D... development data-analysis web-scraping
python-executor halt-catch-fire
Execute Python code in a safe sandboxed environment via inference.sh. Pre-installed: NumPy, Pandas, Matplotlib, requests, BeautifulSoup, Selenium, Playwright, MoviePy, Pillow, OpenCV, trimesh, and 100+ more libraries. Use for: data processing, web scraping, image manipulation, video creation, 3D model processing, PDF generation, API calls, automation scripts. Triggers: python, execute code, run script, web scraping, data analysis, image processing, video editing, 3D... development data-analysis web-scraping
schema coreyhaines31
When the user wants to add, fix, or optimize schema markup and structured data on their site. Also use when the user mentions "schema markup," "structured data," "JSON-LD," "rich snippets," "schema.org," "FAQ schema," "product schema," "review schema," "breadcrumb schema," "Google rich results," "knowledge panel," "star ratings in search," or "add structured data." Use this whenever someone wants their pages to show enhanced results in Google. For broader SEO issues, see seo-audit. For AI... development data-analysis web-scraping
use-my-browser xixu-me
Use when work depends on the user's live browser session or visible rendered state rather than static fetches, especially for browser debugging contexts or DevTools-selected elements or requests, logged-in dashboards or CMS flows, localhost apps, forms, uploads, downloads, media inspection, DOM or iframe inspection, Shadow DOM, or browser failures that look like soft 404s, auth walls, anti-bot checks, or rate limits. browser-automation web-scraping testing
web-search qu-skills
Web search and content extraction with Tavily and Exa via inference.sh CLI. Apps: Tavily Search, Tavily Extract, Exa Search, Exa Answer, Exa Extract. Capabilities: AI-powered search, content extraction, direct answers, research. Use for: research, RAG pipelines, fact-checking, content aggregation, agents. Triggers: web search, tavily, exa, search api, content extraction, research, internet search, ai search, search assistant, web scraping, rag, perplexity alternative research web-scraping api
web-search halt-catch-fire
Web search and content extraction with Tavily and Exa via inference.sh CLI. Apps: Tavily Search, Tavily Extract, Exa Search, Exa Answer, Exa Extract. Capabilities: AI-powered search, content extraction, direct answers, research. Use for: research, RAG pipelines, fact-checking, content aggregation, agents. Triggers: web search, tavily, exa, search api, content extraction, research, internet search, ai search, search assistant, web scraping, rag, perplexity alternative research web-scraping api
npx skills add https://github.com/scrapegraphai/just-scrape --skill Web Scraping SkillsRun this in your project โ your agent picks the skill up automatically.
No common issues documented yet. If you hit a problem, the repository's GitHub Issues page is the best place to look.