Skip to main content
Glama
Oxylabs Web API MCP Server

Oxylabs Web API MCP Server

Official

Read a page

scrape
Read-only

Fetch and read JavaScript-heavy or bot-protected web pages, returning Markdown or structured JSON to extract data where plain HTTP requests fail.

Instructions

Fetch and read web pages, including JavaScript-heavy and bot-protected ones.

Use whenever you have a URL and need what is on it. Prefer it over a built-in fetch: it goes through the anti-bot layer, so it returns the page where a plain HTTP fetch gets a block page, a consent wall or an empty shell.

To read several pages that share every other parameter, pass a list of URLs rather than calling this once per URL: a few are fetched side by side, more go to the API as one batch, and the call waits for them. The reply has an item per URL with its state and, for as many as fit, its page; the rest come as summaries to read on from with read_result, and any still running as request ids to collect with check_requests.

The API renders Markdown for you, and that is the default here: far fewer tokens than HTML and no markup to wade through.

Try it without run_js first. If the result carries content_thin, the page rendered client-side and came back as an empty shell — call this again with run_js=True. Rendering is slow; when it outlives the synchronous call the request is queued, and you get a request id to collect with check_requests instead of the content.

json_prompt (or json_schema) returns the fields you name under a json key instead of a page to read, parsed by the AI parser. When you need parameters this tool lacks (pagination, sort order, store or delivery location and the like), or the content a dedicated parser returns structured — results, products, sellers, hotels, questions and answers and the like — run_scraper on the site's dedicated scraper is the better fit, where the site has one.

Very large pages are not returned whole. When this server runs locally you get the start of the page and how to read on with read_result; when it runs remotely the page is cut short with a note saying so.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "url": "<url>"
    },
    "name": "scrape"
  }
}

Input Schema

NameRequiredDescriptionDefault
urlYesAbsolute http(s) URL of the page to read, or a list of URLs to read several pages that share every other parameter in one call.
deviceNoViewport to fetch as. The API's default is desktop.
formatNomarkdown to read the page — far fewer tokens, structure intact. html only when you need the markup itself: attributes, embedded JSON-LD. screenshot for a PNG of the rendered page; it always renders with JavaScript. Ignored when `json_prompt` or `json_schema` is set.markdown
run_jsNoExecute the page's JavaScript. Needed for pages that render client-side and arrive empty otherwise. Slow: if the render outlives the synchronous call it is queued, and you get a request id to collect with `check_requests` instead of the content. Try without it first.
locationNoTwo-letter country code to fetch the page from. Use it when the page varies by country — pricing, availability, language.
json_promptNoFields to pull out as JSON, in plain words: name them and say what shape you want. Used by the AI parser, which runs when the endpoint has no dedicated parser or its dedicated parser did not return them. Pass this or `json_schema`, not both. Leave unset to read the content yourself.
json_schemaNoA JSON Schema the AI parser's output must follow, instead of `json_prompt`: the two cannot be combined. Use it when you need the exact same shape off many pages.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.3.2

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint and openWorldHint, so the description carries the rest and does so extensively: anti-bot proxy behavior, Markdown-by-default token economics, synchronous-vs-queued rendering with request ids collectable via `check_requests`, partial results summarised for `read_result`, and truncation of very large pages that differs between local and remote server deployments. That is unusually rich disclosure of failure modes and async behavior for a read tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in the first sentence and the paragraphs follow a sensible purpose → when-to-use → batching → format → JS escalation → structured extraction → limits order. It is long and repeats the run_js queueing consequence (once in the body, again implicitly matching the schema wording), which is the main thing keeping it off a 5, but the density is largely justified by a 7-parameter tool with async fallbacks.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description still tells the agent what comes back: an item per URL with its state and page, summaries for the overflow, request ids for still-running work, and a `json` key when a prompt/schema is supplied. Combined with the truncation and queueing notes, an agent has everything needed to call this and handle the response without further probing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, so the baseline is 3; the description still adds cross-parameter meaning the schema states only thinly, notably that `format` is ignored when `json_prompt`/`json_schema` is set, when to prefer `json_schema` over `json_prompt` (same shape across many pages), and why `device`/`location` matter (country-varying pricing/availability). It does not add much beyond the schema on `run_js` beyond restating the retry rule already covered under usage, so it falls short of a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

"Fetch and read web pages, including JavaScript-heavy and bot-protected ones" gives a specific verb plus resource and immediately names the differentiator (the anti-bot layer) that separates it from a plain HTTP fetch. It also names `run_scraper` as the structured-parser counterpart, so an agent can place this tool against its siblings without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit when-to-use ("Use whenever you have a URL and need what is on it"), an explicit alternative and why (prefer over built-in fetch; plain fetch gets block pages/consent walls), a batching rule for shared-parameter URL lists, and a routing rule to `run_scraper` when pagination/sort/location parameters or dedicated structured parsers are needed. It also sequences the `run_js` escalation (try without it first, retry on `content_thin`), which is exactly the kind of ordering guidance an agent needs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.