Skip to main content
Glama
Oxylabs Web API MCP Server

Oxylabs Web API MCP Server

Official

Run a dedicated scraper

run_scraper
Read-only

Run a site-specific scrape endpoint with custom parameters to extract structured JSON or Markdown for search, product, marketplace, video, or AI chat pages, including batch URLs.

Instructions

Run a dedicated scrape endpoint with its own parameters.

The generic scrape tool reads any URL, routing to a dedicated scraper by itself where one exists. Use this when at least one of these holds: you need parameters scrape lacks (pagination, sort order, store or delivery location and the like), or you need the content a dedicated parser returns structured, described below. Dedicated scrapers exist only for major sites (big search engines, marketplaces, retailers, video platforms and AI chats); most sites have none. Look the endpoint up with list_scrapers(), read its parameters with list_scrapers(endpoint), then call it here.

An endpoint with a dedicated parser returns its parsed JSON by default; set output in params (e.g. ["markdown"]) to read the page instead. Other endpoints return Markdown wherever they offer it, unless output says otherwise; for structured data from them, describe the fields in json_prompt. Endpoints with a dedicated parser return typical page content in a predefined shape: search results (organic links, ads, images, news, videos, SERP extras), product listings and details (title, price, stock, seller, reviews), seller profiles, bestsellers, hotel offers, AI chat answers (prompt, response, citations), YouTube video and channel metadata. That runs first; with no dedicated parser, or one that failed or returned nothing, the AI parser extracts what json_prompt describes. The result's parser says which one answered, and parser_note carries the dedicated parser's own verdict (e.g. PARSE_PARTIAL_SUCCESS_SOME_FIELDS_DEFAULT). If a dedicated result lacks what you need, call again with parser="ai".

To run several URLs or queries with the same parameters, give url or query in params as a list: a few run side by side, more go to the endpoint's batch in one request, and the call waits for them. The reply has an item per URL or query, as scrape gives for a list. A batch cannot fall back from the dedicated parser to the AI parser item by item: run the ones it failed again with parser="ai".

Calls run synchronously. One that times out is sent to the corresponding asynchronous endpoint (/v1/async/...) and returns a request_id to collect with check_requests; an async/... endpoint, or one list_scrapers lists only in async form, is always queued.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "endpoint": "<endpoint>",
      "params": "<params>"
    },
    "name": "run_scraper"
  }
}

Input Schema

NameRequiredDescriptionDefault
paramsYesThe endpoint's own request body. Get its JSON Schema from `list_scrapers(endpoint)` rather than guessing the keys. Give `url` or `query` as a list to run several that share every other parameter.
parserNoauto uses the endpoint's dedicated parser when it has one and falls back to the AI parser when that fails or returns nothing. ai goes straight to the AI parser — use it when a dedicated result lacked the fields you need.auto
endpointYesA scrape endpoint path from `list_scrapers`, without the /v1/ prefix, e.g. 'scrape/amazon/search'.
structuredNoReturn parsed JSON instead of page content. Implied by `json_prompt` or `json_schema`, and by an endpoint with a dedicated parser unless `output` in `params` asks for content. The dedicated parser runs first when there is one.
json_promptNoFields to pull out as JSON, in plain words: name them and say what shape you want. Used by the AI parser, which runs when the endpoint has no dedicated parser or its dedicated parser did not return them. Pass this or `json_schema`, not both. Leave unset to read the content yourself.
json_schemaNoA JSON Schema the AI parser's output must follow, instead of `json_prompt`: the two cannot be combined. Use it when you need the exact same shape off many pages.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.3.2

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations only say readOnlyHint=true and openWorldHint=true; the description adds substantial behavior beyond that. It explains the dedicated-parser-then-AI-parser fallback order, the `parser`/`parser_note` fields in the response, batch-vs-parallel execution for list inputs, the inability to fall back per item in a batch, and the synchronous-call timeout path to `/v1/async/...` with a `request_id` collected via `check_requests`.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well front-loaded — purpose and when-to-use come first, with mechanics after — but it is five dense paragraphs and repeats some schema-level material, such as the `parser="ai"` fallback being explained both in `parser`'s schema description and again in prose twice. The catalog of parser output shapes is long, though it does help an agent recognize which endpoints return what.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with six parameters, nested free-form input, and no output schema, the description covers what an agent actually needs: endpoint discovery, parser selection and fallback, structured-vs-markdown output, multi-URL batching, and the sync/async boundary with the tool to collect async results. Return shape is described well enough given no output schema exists.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning the schema cannot: `params` is a free-form object whose keys must be discovered via `list_scrapers(endpoint)`, `output` inside `params` overrides structured mode, and `url`/`query` as lists changes execution semantics. It stops short of giving concrete examples of `params` keys, but it correctly routes the agent to where they live.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: run a dedicated scrape endpoint with its own parameters. It immediately distinguishes itself from the generic `scrape` sibling, which is a separate named tool, so an agent can tell which one to pick without comparing schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit inclusion criteria ('use this when at least one of these holds: you need parameters `scrape` lacks ... or you need the content structured'), a negative condition ('most sites have none; dedicated scrapers exist only for major sites'), and the discovery workflow: `list_scrapers()`, then `list_scrapers(endpoint)`, then call. Alternatives and their selection conditions are named outright.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.