A crawl that can only run to completion means any interruption — a rate limit, a bad selector spotted halfway, a credit ceiling — costs you the whole run. Being able to pause, look at the pages collected so far, and pick the job back up turns crawling into something you can supervise. The monitors are the other half: a watch that reports activity is how you track a page over time without re-scraping it on a cron and paying for the unchanged copies.
ScrapeGraphAI's hosted endpoint. Beyond one-shot scraping it covers crawls as long-running jobs and monitors as standing watches.
- Scrape, extract and search for single-page work
- Crawls across their whole lifecycle — start, get, list pages, stop, resume, delete
- Monitors — create, list, get, update, pause, resume, delete, and read their activity
- Account tools for credits and history
OAuth through Google is their recommended path; an API key as a bearer token is the alternative. Fully hosted — there is no local package.
Paste a URL, then authorize — add the endpoint to your client, then approve the OAuth consent screen