Scrape 1-3 known URLs and extract their content with AI. This tool WAITS for the result (typically 10-60 seconds) and returns the extracted content directly.

IMPORTANT: with multiple URLs, their content is COMBINED and the AI produces ONE answer across all of them (the per-URL raw pages are still returned in "pages"). Use several URLs here when you want to compare or synthesize across pages — e.g. "compare the pricing on these two pages". If instead you want the SAME extraction run separately on each URL (own result per URL), use spidra_batch_scrape even for just 2 URLs.

Best for: one URL, or one combined answer drawn from 2-3 related URLs. Not for: per-URL independent results (use spidra_batch_scrape) or discovering pages on a site (use spidra_crawl).

Behavior notes:

• Omit "prompt" and "schema" to get the raw page content as markdown.

• Pass "prompt" for free-form AI extraction, and add "schema" when you need a guaranteed JSON shape. Define every field in the schema — untyped objects come back empty.

• Use "actions" to interact with the page first (dismiss cookie banners, type into search boxes, scroll, or loop over elements with forEach).

• Use "useProxy" with "proxyCountry" for geo-restricted or bot-protected sites.

• Costs: 2 credits per URL plus AI tokens; CAPTCHA solves cost 10 credits each.

Usage example:

{
  "name": "spidra_scrape",
  "arguments": {
    "urls": ["https://example.com/pricing"],
    "prompt": "Extract all pricing plans with name, price, and included features",
    "output": "json"
  }
}

Returns: extracted content plus token/credit stats. If the wait window is exceeded, the job keeps running — poll spidra_check_scrape_status with the returned jobId.

Check the status of a scrape job by jobId. Only needed when spidra_scrape reported that its wait window was exceeded. Job statuses: waiting/active/running are in progress; completed, failed, and cancelled are terminal.

Scrape a list of 2-50 known URLs in parallel with the same extraction prompt/schema. Each URL is processed INDEPENDENTLY and gets its OWN result (unlike spidra_scrape, which merges multiple URLs into one combined answer). This tool returns IMMEDIATELY with a batchId — it does not wait.

Best for: running the same extraction on each of many similar pages (product pages, listings, articles) where you need separate data per URL — even for just 2 URLs. Workflow: call this, then poll spidra_check_batch_status with the batchId every 10-15 seconds until the batch reaches a terminal state. Do NOT resubmit while a batch is pending.

Costs: 2 credits per URL plus AI tokens. Failed items can be retried from the dashboard or cancelled with spidra_cancel_batch (credits for unprocessed items are refunded).

Check the status of a batch scrape by batchId. Returns per-URL statuses and results for finished items. Batch statuses: pending/running are in progress; completed, failed, and cancelled are terminal. A completed batch can still contain failed items — check failedCount.

Cancel a pending or running batch scrape. Credits for unprocessed items are refunded; already-finished items keep their results.

Crawl a website starting from one URL: Spidra discovers pages by following links according to your plain-language instruction, and optionally extracts structured data from every page. Returns IMMEDIATELY with a jobId — it does not wait.

Best for: extracting from many pages when you do NOT know their URLs upfront (docs sites, blogs, product catalogs). Not for: URLs you already know (use spidra_scrape or spidra_batch_scrape — cheaper and faster). Workflow: call this, then poll spidra_check_crawl_status with the jobId every 10-15 seconds until terminal. Do NOT resubmit while a crawl is pending. Cancel a mistake with spidra_cancel_crawl.

Behavior notes:

• "crawlInstruction" controls which links are followed (e.g. "Follow blog post links only, skip tag pages").

• "transformInstruction" controls what is extracted per page; omit it (and schema) for raw markdown with no AI token cost.

• Keep "maxPages" small (default 5, max 50) — every page costs credits.

Usage example:

{
  "name": "spidra_crawl",
  "arguments": {
    "baseUrl": "https://example.com/blog",
    "crawlInstruction": "Follow blog post links only, skip tag and category pages",
    "transformInstruction": "Extract the title, author, and publish date",
    "maxPages": 10
  }
}

Check the status of a crawl job by jobId. While running, returns progress (pagesCrawled/maxPages). When completed, returns the extracted data for every page. Job statuses: waiting/active/running are in progress; completed, failed, and cancelled are terminal.

Get per-page results for a crawl, including signed download URLs for each page's raw HTML and markdown (links expire after 1 hour). Works on completed crawls and on cancelled crawls (returns the pages processed before cancellation).

Run a NEW extraction prompt over an already-completed crawl without re-crawling any pages — much cheaper than crawling again (only AI token credits are charged). Returns a new jobId immediately; poll spidra_check_crawl_status with it. The source crawl must have status "completed".

Cancel a queued or running crawl job. Pages already processed are preserved and retrievable with spidra_crawl_pages; credits for unprocessed pages are refunded.

List past scrape jobs for this account with optional filters. Useful for finding a previous job's result, debugging failures, or checking what a key has been used for. Fetch a single log's full AI output by passing its uuid.

Get this account's request/credit/token usage broken down by day or week. Use it to answer "how many credits have I used" style questions or to check remaining headroom before a large batch/crawl.