SocialCrawl Web Scraping & Browser Automation
socialcrawl_webFull web scraping, search, and browser automation (Firecrawl-backed). Sync reads: 'scrape' (URL → markdown/HTML/screenshot/links), 'search' (web search with page content), 'map' (discover a site's URLs), 'extract' (LLM structured data from a page). Async jobs (submit, then poll with job_get/job_list, stop with job_cancel): 'crawl' a whole site, 'batch_scrape' many URLs, 'agent' (autonomous multi-step web task). Change detection: monitor_create/list/get/update/delete/checks (re-check a URL on a cadence → webhook). Interactive browser: session_create/get/list, session_execute (run code in the live page), session_close. Pricing varies by action (scrape 1cr, search 2cr, extract/session_create 5cr, agent 25cr; jobs/monitors/sessions management 0cr) — see the 'web' get_docs topic. Requires a valid SOCIALCRAWL_API_KEY.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Job / monitor / session id. Required for *_get, *_cancel, *_delete, *_update, *_checks, *_execute, and job_errors actions. Returned by the matching *_create / *_list action. | |
| input | No | Operation parameters. For sync/GET actions these are query params (e.g. { url: 'https://example.com', formats: 'markdown,screenshot' } for scrape; { query: 'ai agents', limit: 10 } for search). For POST/PATCH actions this is the JSON body (e.g. { url, prompt, model } for agent; { url, cadence_minutes, webhook_url } for monitor_create; { code, language } for session_execute). Use socialcrawl_list_endpoints for platform 'web', or the 'web' get_docs topic, for the full per-action parameter list. | |
| action | Yes | Web operation. Sync (returns data now): scrape, search, map, extract. Async jobs: crawl, batch_scrape, agent → then job_get/job_list/job_cancel to poll, and job_errors for a job's per-page failure feed. crawl_preview dry-runs a crawl's parameters for free before you pay for it. Change detection: monitor_create/list/get/update/delete/checks. Interactive browser: session_create/list/get/execute/close. | |
| idempotencyKey | No | Optional Idempotency-Key for the async job submitters (crawl, batch_scrape). Replays return the original job and deduct 0 credits. |