Skip to main content
Glama
spidra-mcp-server

spidra-mcp-server

Official
by spidra-io

spidra_batch_scrape

Read-only

Scrape 2 to 50 URLs in one batch with a shared prompt or schema, returning separate results per URL for product pages, listings, or article comparisons.

Instructions

Read 2 to 50 known URLs, each on its own with its own result, using the same prompt or schema for all.

Use it for anything with more than one URL: product pages, listings, articles, or just two pages to compare. Waiting: with "wait": true and 10 URLs or fewer, it waits (up to a few minutes) and returns the finished batch. Otherwise it returns a "batchId" straight away; check it with spidra_get_job (type "batch") every 10 to 15 seconds until "status" is completed or cancelled. Don't start the batch again while it runs.

  • "output" is "markdown" (default) or "json"; a schema sets "json". Popular-site pages also get "structuredData".

  • A completed batch can still have failed items: check "failedCount" and each item's "error". spidra_retry_batch re-runs only the failed ones.

  • Browser actions aren't supported in a batch; use spidra_scrape for pages that need them.

Cost: per URL, the same as spidra_scrape. Failed URLs cost nothing.

Example: {"urls": ["https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html", "https://books.toscrape.com/catalogue/tipping-the-velvet_999/index.html"], "prompt": "Title and price", "output": "json", "wait": true}

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "urls": "<urls>"
    },
    "name": "spidra_batch_scrape"
  }
}

Input Schema

NameRequiredDescriptionDefault
urlsYes2 to 50 URLs
waitNoWith 10 URLs or fewer, wait and return the finished batch instead of a batchId
outputNo"markdown" (default) or "json"
promptNoWhat to extract from each page. Leave out for Markdown.
schemaNoA JSON Schema for the answer. You get exactly these fields and types, with null for anything the page doesn't show. Define every field you want: an object with no properties comes back empty. The root must be an object.
cookiesNoCookies from the user's own signed-in browser session, for pages behind a login, e.g. "session=abc; token=xyz". Never stored.
useProxyNoStealth Mode: browse through residential proxies, for sites that block or vary by country. Billed by bandwidth, not credits.
scrapeModeNo"default" always uses a browser; "fast" fetches pages and opens a browser only when needed
proxyCountryNoWith useProxy, the country to browse from: a two-letter code such as "us", "gb" or "de", or "global", "eu" or "asia"
extractContentOnlyNoLeave out menus, headers, footers and ads

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed12 schema fields changedv0.7.0
    • addedInput schema / properties / cookies / description
      Added value: +"Cookies from the user's own signed-in browser session, for pages behind a login, e.g. \"session=abc; token=xyz\". Never stored."
    • addedInput schema / properties / extractContentOnly / description
      Added value: +"Leave out menus, headers, footers and ads"
    • addedInput schema / properties / output / description
      Added value: +"\"markdown\" (default) or \"json\""
    • changedInput schema / properties / output / enum
      Previous value: -[
      -  "json",
      -  "markdown",
      -  "text",
      -  "table"
      -]New value: +[
      +  "markdown",
      +  "json"
      +]
    • changedInput schema / properties / prompt / description
      Previous value: -"What to extract from each page. Omit for raw markdown."New value: +"What to extract from each page. Leave out for Markdown."
    • changedInput schema / properties / proxyCountry / description
      Previous value: -"Two-letter country code for the proxy, e.g. \"us\", \"de\", \"jp\", or \"eu\"/\"global\""New value: +"With useProxy, the country to browse from: a two-letter code such as \"us\", \"gb\" or \"de\", or \"global\", \"eu\" or \"asia\""
    • changedInput schema / properties / schema / additionalProperties
      Previous value: -falseNew value: +{}
    • changedInput schema / properties / schema / description
      Previous value: -"JSON Schema enforcing the exact output shape. Define EVERY field you want extracted — an untyped object with no properties comes back empty. Missing fields return null instead of hallucinated values."New value: +"A JSON Schema for the answer. You get exactly these fields and types, with null for anything the page doesn't show. Define every field you want: an object with no properties comes back empty. The root must be an object."
    • addedInput schema / properties / scrapeMode / description
      Added value: +"\"default\" always uses a browser; \"fast\" fetches pages and opens a browser only when needed"
    • changedInput schema / properties / urls / description
      Previous value: -"2-50 URLs to scrape in parallel (plain strings)"New value: +"2 to 50 URLs"
    • changedInput schema / properties / useProxy / description
      Previous value: -"Route through a residential proxy (for blocked/geo-restricted sites)"New value: +"Stealth Mode: browse through residential proxies, for sites that block or vary by country. Billed by bandwidth, not credits."
    • addedInput schema / properties / wait
      Added value: +{
      +  "description": "With 10 URLs or fewer, wait and return the finished batch instead of a batchId",
      +  "type": "boolean"
      +}
  2. First observedv0.2.2

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnly/openWorld annotations: explains the wait-vs-batchId async behavior with a threshold (10 URLs), the 10-15s poll interval, the 'don't restart while running' rule, partial-failure semantics (failedCount, per-item error), and cost (per URL, same as spidra_scrape, failures free). This is rich operational context the annotations cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the core purpose, then uses bolded labels (Use it for, Waiting, Cost) and bullets so an agent can scan. Despite its length, every section (async behavior, failure handling, cost) earns its place and no sentence is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter async tool with no output schema, the description covers what an agent most needs: return-shape behavior (batchId vs finished batch), polling procedure, partial failure handling, cost, and output format options. Nothing critical to correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description still adds value by tying 'a schema sets json' and noting popular-site pages also return 'structuredData', which the schema does not mention. The example object reinforces correct parameter combination, though most field detail is already in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Read 2 to 50 known URLs'), defines the scope (each with its own result, shared prompt/schema), and immediately differentiates from siblings by pointing to spidra_scrape for browser-action pages. An agent can distinguish it from spidra_crawl and spidra_scrape without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit 'Use it for anything with more than one URL' plus concrete alternatives: spidra_scrape for browser-action pages, spidra_get_job to poll status, spidra_retry_batch to re-run failures. It even states the exclusion ('Browser actions aren't supported in a batch') and the polling cadence, leaving nothing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.