Skip to main content
Glama
spidra-mcp-server

spidra-mcp-server

Official
by spidra-io

spidra_scrape

Read one URL and return its content as Markdown or extract requested data as JSON, including PDF and Office files, with browser actions and proxy support for blocked pages.

Instructions

Read one known URL and get its content as Markdown, or the data you describe as JSON. Waits and returns the result, usually in a few seconds (up to a minute for hard pages).

Use it for one URL. Links to PDF, Word, Excel and PowerPoint files work too. Don't use it for more than one URL, even to compare two: spidra_batch_scrape gives each URL its own result. To find pages on a site, use spidra_map or spidra_crawl.

  • No "prompt" or "schema": the page as Markdown.

  • "prompt": what you want, in plain words; the AI picks the shape. Add "schema" for exact fields and types (null where the page has nothing).

  • Popular sites (Amazon, Google Maps, LinkedIn, YouTube, Zillow, Indeed and others): with "output": "json" and no prompt, "content" is the page's data in named fields, no AI needed. Otherwise that data is in "structuredData".

  • "actions" runs browser steps first (click, type, scroll, forEach to go through a list); "instruction" does one task described in plain words instead. Never both. Steps run once and are never retried automatically, since a step like a form submission has effects.

  • If the result has "lowContent": true, the page came back nearly empty, usually because the site blocks automated visits. Call again with "useProxy": true.

Cost: 1 credit, plus AI credits for the tokens used with a prompt or schema (often 1 or 2), plus 5 per CAPTCHA solved. A page that can't be read costs nothing.

Browser actions: up to 50 steps (click, type, check, uncheck, hover, select, press, wait, waitFor, scroll, scrollUntilEnd, evaluate, screenshot, forEach), with waits adding up to 60 seconds at most. "selector" is CSS, XPath, or plain words ("the Sign in button"); prefer CSS when you know it. A failed step fails the job (free) with "Action N (...) failed: "; mark steps that may not be there, like cookie banners, "optional": true. forEach goes through a list of items and must be the single, last action. "actionResults" reports each step as ok, skipped or failed.

Example: {"url": "https://books.toscrape.com", "prompt": "Return the title and price of every book", "output": "json"} Example with actions: {"url": "https://quotes.toscrape.com/login", "actions": [{"type": "type", "selector": "#username", "value": "demo"}, {"type": "type", "selector": "#password", "value": "demo"}, {"type": "click", "selector": "input[type='submit']"}, {"type": "waitFor", "selector": "a[href='/logout']"}], "prompt": "List the first 3 quotes with their authors", "output": "json"}

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "url": "<url>"
    },
    "name": "spidra_scrape"
  }
}

Input Schema

NameRequiredDescriptionDefault
urlYesThe page to read
outputNo"markdown" (default) or "json"
promptNoWhat to extract, in plain words. Leave out for the page as Markdown.
schemaNoA JSON Schema for the answer. You get exactly these fields and types, with null for anything the page doesn't show. Define every field you want: an object with no properties comes back empty. The root must be an object.
actionsNoBrowser steps to run in order before the page is read. A forEach must be the single, last one. Not with instruction.
cookiesNoCookies from the user's own signed-in browser session, for pages behind a login, e.g. "session=abc; token=xyz". Never stored.
useProxyNoStealth Mode: browse through residential proxies, for sites that block or vary by country. Billed by bandwidth, not credits.
scrapeModeNo"fast" fetches the page and opens a browser only if the page needs one; "default" always uses a browser. Unset: "fast", or a browser when there are actions or an instruction.
screenshotNoTake a screenshot of the visible page (a link comes back)
deliveryZipNoAmazon and Walmart product pages: the delivery postcode to check price and stock for, e.g. "90210"
instructionNoOne task in plain words to do before the page is read, e.g. "search for wireless headphones and open the first result". Not with actions.
proxyCountryNoWith useProxy, the country to browse from: a two-letter code such as "us", "gb" or "de", or "global", "eu" or "asia"
extractContentOnlyNoLeave out menus, headers, footers and ads
fullPageScreenshotNoTake a screenshot of the whole page

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed25 schema fields changedv0.7.0
    • removedInput schema / definitions
      Removed value: -{
      -  "__schema0": {
      -    "additionalProperties": {},
      -    "properties": {
      -      "actions": {
      -        "description": "forEach: per-element actions after click/navigate",
      -        "items": {
      -          "$ref": "#/definitions/__schema0"
      -        },
      -        "type": "array"
      -      },
      -      "captureSelector": {
      -        "description": "forEach: CSS selector of content to capture per item",
      -        "type": "string"
      -      },
      -      "duration": {
      -        "description": "Milliseconds to pause — for the wait action",
      -        "type": "number"
      -      },
      -      "itemPrompt": {
      -        "description": "forEach: per-element extraction prompt",
      -        "type": "string"
      -      },
      -      "maxItems": {
      -        "description": "forEach: max elements to process (cap 50)",
      -        "type": "number"
      -      },
      -      "mode": {
      -        "description": "forEach interaction mode",
      -        "enum": [
      -          "click",
      -          "inline",
      -          "navigate"
      -        ],
      -        "type": "string"
      -      },
      -      "observe": {
      -        "description": "forEach: which elements to find, e.g. \"Find all product cards\"",
      -        "type": "string"
      -      },
      -      "pagination": {
      -        "properties": {
      -          "maxPages": {
      -            "description": "Max extra pages to paginate through (cap 10)",
      -            "type": "number"
      -          },
      -          "nextSelector": {
      -            "description": "Selector or description of the \"next page\" link",
      -            "type": "string"
      -          }
      -        },
      -        "required": [
      -          "nextSelector"
      -        ],
      -        "type": "object"
      -      },
      -      "selector": {
      -        "description": "CSS selector or XPath of the target element",
      -        "type": "string"
      -      },
      -      "to": {
      -        "description": "Scroll destination as a percentage, e.g. \"80%\"",
      -        "type": "string"
      -      },
      -      "type": {
      -        "enum": [
      -          "click",
      -          "type",
      -          "check",
      -          "uncheck",
      -          "wait",
      -          "scroll",
      -          "forEach"
      -        ],
      -        "type": "string"
      -      },
      -      "value": {
      -        "description": "Plain-language element description (AI locates it), or the text to type",
      -        "type": "string"
      -      },
      -      "waitAfterClick": {
      -        "description": "forEach: ms to wait after click/navigate before capture",
      -        "type": "number"
      -      }
      -    },
      -    "required": [
      -      "type"
      -    ],
      -    "type": "object"
      -  }
      -}
    • changedInput schema / properties / actions / description
      Previous value: -"Browser actions to run on each URL before extraction, in order"New value: +"Browser steps to run in order before the page is read. A forEach must be the single, last one. Not with instruction."
    • removedInput schema / properties / actions / items / $ref
      Removed value: -"#/definitions/__schema0"
    • addedInput schema / properties / actions / items / additionalProperties
      Added value: +{}
    • addedInput schema / properties / actions / items / properties
      Added value: +{
      +  "actions": {
      +    "description": "forEach: steps to run on each item after it is clicked or opened (no forEach)",
      +    "items": {
      +      "additionalProperties": {},
      +      "description": "A step, with the same fields as a top-level action (no forEach)",
      +      "properties": {
      +        "clickSelector": {
      +          "type": "string"
      +        },
      +        "duration": {
      +          "maximum": 30000,
      +          "minimum": 1,
      +          "type": "number"
      +        },
      +        "fullPage": {
      +          "type": "boolean"
      +        },
      +        "key": {
      +          "type": "string"
      +        },
      +        "maxScrolls": {
      +          "maximum": 30,
      +          "minimum": 1,
      +          "type": "integer"
      +        },
      +        "optional": {
      +          "type": "boolean"
      +        },
      +        "pause": {
      +          "maximum": 5000,
      +          "minimum": 200,
      +          "type": "number"
      +        },
      +        "pixels": {
      +          "type": "number"
      +        },
      +        "script": {
      +          "maxLength": 10000,
      +          "type": "string"
      +        },
      +        "selector": {
      +          "type": "string"
      +        },
      +        "state": {
      +          "enum": [
      +            "visible",
      +            "attached",
      +            "hidden",
      +            "detached"
      +          ],
      +          "type": "string"
      +        },
      +        "timeout": {
      +          "maximum": 30000,
      +          "minimum": 100,
      +          "type": "number"
      +        },
      +        "to": {
      +          "type": [
      +            "number",
      +            "string"
      +          ]
      +        },
      +        "type": {
      +          "enum": [
      +            "click",
      +            "type",
      +            "check",
      +            "uncheck",
      +            "hover",
      +            "select",
      +            "press",
      +            "wait",
      +            "waitFor",
      +            "scroll",
      +            "scrollUntilEnd",
      +            "evaluate",
      +            "screenshot"
      +          ],
      +          "type": "string"
      +        },
      +        "value": {
      +          "anyOf": [
      +            {
      +              "type": "string"
      +            },
      +            {
      +              "items": {
      +                "type": "string"
      +              },
      +              "type": "array"
      +            }
      +          ]
      +        }
      +      },
      +      "required": [
      +        "type"
      +      ],
      +      "type": "object"
      +    },
      +    "maxItems": 10,
      +    "type": "array"
      +  },
      +  "captureSelector": {
      +    "description": "forEach: what to capture for each item (CSS selector, XPath or description)",
      +    "type": "string"
      +  },
      +  "clickSelector": {
      +    "description": "scrollUntilEnd: a \"Load more\" button to click each round instead of scrolling",
      +    "type": "string"
      +  },
      +  "duration": {
      +    "description": "wait: milliseconds to pause",
      +    "maximum": 30000,
      +    "minimum": 1,
      +    "type": "number"
      +  },
      +  "fullPage": {
      +    "description": "screenshot: capture the whole page instead of the viewport",
      +    "type": "boolean"
      +  },
      +  "itemPrompt": {
      +    "description": "forEach: an extraction prompt run on each item on its own",
      +    "maxLength": 2000,
      +    "type": "string"
      +  },
      +  "key": {
      +    "description": "press: a key name such as \"Enter\", \"Tab\", \"Escape\", \"ArrowDown\" or \"Control+A\"",
      +    "type": "string"
      +  },
      +  "maxItems": {
      +    "description": "forEach: most items across all pages (default 50)",
      +    "maximum": 50,
      +    "minimum": 1,
      +    "type": "integer"
      +  },
      +  "maxScrolls": {
      +    "description": "scrollUntilEnd: most rounds (default 10)",
      +    "maximum": 30,
      +    "minimum": 1,
      +    "type": "integer"
      +  },
      +  "mode": {
      +    "description": "forEach: \"click\" (default) clicks each item and captures what appears, \"inline\" reads each item in place, \"navigate\" opens each item's link",
      +    "enum": [
      +      "click",
      +      "inline",
      +      "navigate"
      +    ],
      +    "type": "string"
      +  },
      +  "observe": {
      +    "description": "forEach (required): which elements to go through, in plain English, e.g. \"each product card\"",
      +    "type": "string"
      +  },
      +  "optional": {
      +    "description": "true: if this step fails, skip it (reported as skipped) and run the rest. Without it, a failed step fails the whole job. Use it for things that may not be there, like cookie banners or popups.",
      +    "type": "boolean"
      +  },
      +  "pagination": {
      +    "additionalProperties": false,
      +    "description": "forEach: move through results pages after the items on the current one",
      +    "properties": {
      +      "maxPages": {
      +        "description": "Pages in total, counting the first (default 5)",
      +        "maximum": 10,
      +        "minimum": 1,
      +        "type": "integer"
      +      },
      +      "nextSelector": {
      +        "description": "The \"next page\" button or link (CSS selector, XPath or description)",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "nextSelector"
      +    ],
      +    "type": "object"
      +  },
      +  "pause": {
      +    "description": "scrollUntilEnd: milliseconds to wait after each round (default 1500)",
      +    "maximum": 5000,
      +    "minimum": 200,
      +    "type": "number"
      +  },
      +  "pixels": {
      +    "description": "scroll: scroll by this many pixels instead of using to (negative scrolls up)",
      +    "type": "number"
      +  },
      +  "script": {
      +    "description": "evaluate: JavaScript run in the page. An expression (\"document.title\") is returned as is; statements need return. The value comes back in actionResults.",
      +    "maxLength": 10000,
      +    "type": "string"
      +  },
      +  "selector": {
      +    "description": "The element: a CSS selector (\"#login\"), an XPath, or plain English (\"the Sign in button\"). Required for click, check, uncheck, hover, select and waitFor; optional for type (default: the first visible text field) and press (default: the page).",
      +    "type": "string"
      +  },
      +  "state": {
      +    "description": "waitFor: the state to wait for (default \"visible\")",
      +    "enum": [
      +      "visible",
      +      "attached",
      +      "hidden",
      +      "detached"
      +    ],
      +    "type": "string"
      +  },
      +  "timeout": {
      +    "description": "Milliseconds to look for the element (default 10000). Raise it for slow pages, e.g. waitFor on content that loads late.",
      +    "maximum": 30000,
      +    "minimum": 100,
      +    "type": "number"
      +  },
      +  "to": {
      +    "description": "scroll: a percentage of the page height (80 or \"80%\"), \"top\", \"bottom\", or an element to scroll into view",
      +    "type": [
      +      "number",
      +      "string"
      +    ]
      +  },
      +  "type": {
      +    "description": "The action. forEach must be the single, last action.",
      +    "enum": [
      +      "click",
      +      "type",
      +      "check",
      +      "uncheck",
      +      "hover",
      +      "select",
      +      "press",
      +      "wait",
      +      "waitFor",
      +      "scroll",
      +      "scrollUntilEnd",
      +      "evaluate",
      +      "screenshot",
      +      "forEach"
      +    ],
      +    "type": "string"
      +  },
      +  "value": {
      +    "anyOf": [
      +      {
      +        "type": "string"
      +      },
      +      {
      +        "items": {
      +          "type": "string"
      +        },
      +        "type": "array"
      +      }
      +    ],
      +    "description": "type: the text to type. select: the option to choose (by label or value), or several for a multi-select."
      +  },
      +  "waitAfterClick": {
      +    "description": "forEach: ms to wait after clicking or opening an item before capturing it (default 2500)",
      +    "maximum": 10000,
      +    "minimum": 0,
      +    "type": "number"
      +  }
      +}
    • addedInput schema / properties / actions / items / required
      Added value: +[
      +  "type"
      +]
    • addedInput schema / properties / actions / items / type
      Added value: +"object"
    • addedInput schema / properties / actions / maxItems
      Added value: +50
    • changedInput schema / properties / cookies / description
      Previous value: -"Raw Cookie header string for pages behind a login, e.g. \"session=abc\""New value: +"Cookies from the user's own signed-in browser session, for pages behind a login, e.g. \"session=abc; token=xyz\". Never stored."
    • addedInput schema / properties / deliveryZip
      Added value: +{
      +  "description": "Amazon and Walmart product pages: the delivery postcode to check price and stock for, e.g. \"90210\"",
      +  "type": "string"
      +}
    • changedInput schema / properties / extractContentOnly / description
      Previous value: -"Strip navigation/ads/boilerplate before extraction"New value: +"Leave out menus, headers, footers and ads"
    • addedInput schema / properties / fullPageScreenshot
      Added value: +{
      +  "description": "Take a screenshot of the whole page",
      +  "type": "boolean"
      +}
    • addedInput schema / properties / instruction
      Added value: +{
      +  "description": "One task in plain words to do before the page is read, e.g. \"search for wireless headphones and open the first result\". Not with actions.",
      +  "maxLength": 2000,
      +  "type": "string"
      +}
    • changedInput schema / properties / output / description
      Previous value: -"Output format (default \"markdown\")"New value: +"\"markdown\" (default) or \"json\""
    • changedInput schema / properties / output / enum
      Previous value: -[
      -  "json",
      -  "markdown",
      -  "text",
      -  "table"
      -]New value: +[
      +  "markdown",
      +  "json"
      +]
    • changedInput schema / properties / prompt / description
      Previous value: -"What to extract, in plain English. Omit for raw markdown."New value: +"What to extract, in plain words. Leave out for the page as Markdown."
    • changedInput schema / properties / proxyCountry / description
      Previous value: -"Two-letter country code for the proxy, e.g. \"us\", \"de\", \"jp\", or \"eu\"/\"global\""New value: +"With useProxy, the country to browse from: a two-letter code such as \"us\", \"gb\" or \"de\", or \"global\", \"eu\" or \"asia\""
    • changedInput schema / properties / schema / additionalProperties
      Previous value: -falseNew value: +{}
    • changedInput schema / properties / schema / description
      Previous value: -"JSON Schema enforcing the exact output shape. Define EVERY field you want extracted — an untyped object with no properties comes back empty. Missing fields return null instead of hallucinated values."New value: +"A JSON Schema for the answer. You get exactly these fields and types, with null for anything the page doesn't show. Define every field you want: an object with no properties comes back empty. The root must be an object."
    • changedInput schema / properties / scrapeMode / description
      Previous value: -"\"fast\" = HTTP only (no browser), cheaper but less capable"New value: +"\"fast\" fetches the page and opens a browser only if the page needs one; \"default\" always uses a browser. Unset: \"fast\", or a browser when there are actions or an instruction."
    • changedInput schema / properties / screenshot / description
      Previous value: -"Capture a viewport screenshot (URL returned)"New value: +"Take a screenshot of the visible page (a link comes back)"
    • addedInput schema / properties / url
      Added value: +{
      +  "description": "The page to read",
      +  "type": "string"
      +}
    • removedInput schema / properties / urls
      Removed value: -{
      -  "description": "1-3 URLs to scrape in parallel",
      -  "items": {
      -    "type": "string"
      -  },
      -  "maxItems": 3,
      -  "minItems": 1,
      -  "type": "array"
      -}
    • changedInput schema / properties / useProxy / description
      Previous value: -"Route through a residential proxy (for blocked/geo-restricted sites)"New value: +"Stealth Mode: browse through residential proxies, for sites that block or vary by country. Billed by bandwidth, not credits."
    • changedInput schema / required
      Previous value: -[
      -  "urls"
      -]New value: +[
      +  "url"
      +]
  2. First observedv0.2.2

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well past the annotations by disclosing latency expectations (seconds, up to a minute), the full cost model (1 credit + AI tokens + 5 per CAPTCHA), that unreadable pages are free, and that action steps are never auto-retried because they can have side effects. These are exactly the operational facts an agent needs before invoking a browser-driven scrape.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose, then organized into bolded Use/Don't-use blocks, cost, action semantics and two worked examples. It's long, but the length is largely earned by a 14-parameter nested schema; a few lines (e.g. the exhaustive action-type list) partially duplicate the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex open-world scraper with no output schema, the description covers invocation, cost, failure semantics (optional steps, actionResults ok/skipped/failed), empty-page recovery and concrete examples, leaving no significant gap for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description still adds real meaning by explaining the interplay of prompt/schema/output (no prompt = Markdown, prompt = AI picks shape, schema = exact fields with null), that output:json on popular sites returns named fields in content, and that actions and instruction are mutually exclusive. It doesn't restate every nested action field, which the schema already covers.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ('Read one known URL') and states both possible return shapes (Markdown or described JSON). It explicitly distinguishes itself from spidra_batch_scrape, spidra_map and spidra_crawl, so an agent can route without opening other schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use ('one URL', files like PDF/Word/Excel) and when-not-to-use ('more than one URL, even to compare two'), naming the exact alternative tools for each case. Also tells the agent what to do on a blocked page (retry with useProxy when lowContent is true).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.