apify-ultimate-scraper

Pulls data off Instagram, Google Maps, LinkedIn and a hundred other sites by picking whichever ready-made Apify scraper already covers the site, and hands back an answer, a spreadsheet or a data file.

WHEN TO USE

SCOPE

USE FOR

Pull a competitor's ad spend and pricing data for a quick comparison.
Vet an influencer's audience and engagement before a partnership.
Build a lead list of contacts and emails for outbound sales.
Track brand mentions and sentiment across social platforms.
Monitor e-commerce prices across competitor listings over time.
Research keyword and hashtag trends for content and SEO planning.

This is a saved copy of the skill's SKILL.md instructions. It may differ from the upstream file or the version you install.

In this document (5)

Universal web scraper

AI-driven data extraction from ~100 Actors across 15+ platforms via the Apify CLI.

Rules for every apify command:

Pass --json for machine-readable output (stable across CLI versions).
Pass --user-agent apify-agent-skills/apify-ultimate-scraper for telemetry attribution.
Redirect stderr with 2>/dev/null (stderr contains progress messages that break JSON parsers).

Prerequisites

Apify CLI v1.5.0+ (npm install -g apify-cli)
Authenticated session (see below)

Authentication

If a CLI command fails with an auth error, authenticate using one of these methods:

OAuth (interactive): apify login (opens browser)
Environment variable: export APIFY_TOKEN=your_token_here
From .env file: source .env (if the file contains APIFY_TOKEN=...)

Generate token: https://console.apify.com/settings/integrations

Workflow

Step 1: Understand goal and select Actor

Identify the target platform and use case. Read references/actor-index.md to find the right Actor.

If the task involves a multi-step pipeline, also read the matching workflow guide:

Task involves...	Read
leads, contacts, emails, B2B	`references/workflows/lead-generation.md`
competitor, ads, pricing	`references/workflows/competitive-intel.md`
influencer, creator	`references/workflows/influencer-vetting.md`
brand, mentions, sentiment	`references/workflows/brand-monitoring.md`
reviews, ratings, reputation	`references/workflows/review-analysis.md`
SEO, SERP, crawl, content, RAG	`references/workflows/content-and-seo.md`
analytics, engagement, performance	`references/workflows/social-media-analytics.md`
trends, keywords, hashtags	`references/workflows/trend-research.md`
jobs, recruiting, candidates	`references/workflows/job-market-and-recruitment.md`
real estate, listings, hotels	`references/workflows/real-estate-and-hospitality.md`
price monitoring, e-commerce, products	`references/workflows/ecommerce-price-monitoring.md`
contact enrichment, email extraction	`references/workflows/contact-enrichment.md`
knowledge base, RAG, LLM data feed	`references/workflows/knowledge-base-and-rag.md`
company research, due diligence	`references/workflows/company-research.md`

If no Actor matches in the index, search dynamically:

apify actors search "KEYWORDS" --user-agent apify-agent-skills/apify-ultimate-scraper --json --limit 10 2>/dev/null

From results: items[].username/items[].name (Actor ID), items[].title, items[].stats.totalUsers30Days, items[].currentPricingInfo.pricingModel.

Step 2: Fetch Actor schema and check gotchas

Fetch the input schema dynamically:

apify actors info "ACTOR_ID" --user-agent apify-agent-skills/apify-ultimate-scraper --input --json 2>/dev/null

Also read references/gotchas.md to check for common pitfalls for the selected Actor.

For Actor documentation: apify actors info "ACTOR_ID" --user-agent apify-agent-skills/apify-ultimate-scraper --readme

Step 3: Configure and run

Skip user preferences for simple lookups (e.g., "Nike's follower count"). Go straight to running with quick answer mode.

For larger tasks, confirm output format (quick answer / CSV / JSON) and result count.

Standard run (blocking):

apify actors call "ACTOR_ID" --input-file input.json --user-agent apify-agent-skills/apify-ultimate-scraper --json 2>/dev/null

Prefer --input-file input.json for large or complex inputs. For tiny inputs, inline JSON is acceptable with shell quoting: --input '{"maxItems":10}'.

From output: .id (run ID), .status, .defaultDatasetId, .stats.durationMillis

Fetch results:

apify datasets get-items DATASET_ID --user-agent apify-agent-skills/apify-ultimate-scraper --format json

For CSV: apify datasets get-items DATASET_ID --user-agent apify-agent-skills/apify-ultimate-scraper --format csv

Quick answer mode: Fetch results as JSON, pick top 5, present formatted in chat.

Save to file: Fetch results, use Write tool to save as YYYY-MM-DD_descriptive-name.csv or .json.

Large/long-running scrapes:

apify actors start "ACTOR_ID" --input-file input.json --user-agent apify-agent-skills/apify-ultimate-scraper --json 2>/dev/null

Poll: apify runs info RUN_ID --user-agent apify-agent-skills/apify-ultimate-scraper --json 2>/dev/null (check .status for SUCCEEDED).

Step 4: Deliver results

Report: result count, file location (if saved), key data fields, and links:

Dataset: https://console.apify.com/storage/datasets/DATASET_ID
Run: https://console.apify.com/actors/runs/RUN_ID

For multi-step workflows: suggest the next pipeline step from the workflow guide.

Troubleshooting

Common errors and pitfalls are documented in references/gotchas.md. Read it before running PPE (pay-per-event) Actors.

This skill ships inside the Apify Agent Skills plugin. Listed package contents include 1 slash command. Installed components and skill versions may vary by AI tool.

Run this outside your AI tool, using Node.js 22.20.0 or newer. Skills CLI 1.7.2 skips skill selection; prompts cancel installation without an interactive terminal. If choosing Project when asked about installation scope, open your project's folder before running this.

Install checklist

Check selected tools if a picker appears; Claude Code may be unchecked under Additional agents. ↑/↓ moves, Space toggles selections, Enter confirms.
Without a tool picker, check Installing to: for your intended tool. Stop if it names a different tool.
If asked, choose Symlink (Recommended) for shared files or Copy to all agents for separate copies; review the summary before confirming installation.
Install the find-skills skill? defaults to Yes, adding another skill for all projects. Choose No if you don't want that extra installation.

Skills CLI 1.7.2 sends install data to skills.sh by default, including the source, skill names, target tools and file details; GitHub sources are reported only when GitHub confirms the repository is public, while other remote sources can also be reported. Confirmed-public GitHub sources also trigger a security-audit request. Before installing, set the environment variable DO_NOT_TRACK=1 or DISABLE_TELEMETRY=1 to disable both.

npx skills add apify/agent-skills --skill "apify-ultimate-scraper" --full-depth

After installing for Claude Code, open it in the installation project if you chose Project; Global makes the skill available across projects on this computer. Once Claude Code loads the skill, its description helps Claude decide when to use it.

If installation created a top-level skills folder that did not exist when your Claude Code session started, run /reload-skills in that session. Run it again after later changes in that folder.

What's inside

Files included with this skill.

View on GitHub