ScrapeUnblocker MCP Server lets you fetch web pages and run Google searches via an anti-bot bypass API, returning HTML or structured JSON.
fetch_html: Fetches fully rendered HTML of any URL, bypassing anti-bot protections (Cloudflare, DataDome, Akamai, etc.). Supports waiting for a CSS selector or JS expression before capturing, proxy country selection (ISO 3166-1 alpha-2), extra sleep after load, and custom timeouts.
fetch_parsed: Fetches a URL and returns AI-parsed structured JSON (e.g., product details, article content) instead of raw HTML. Accepts an optional natural-language hint to guide data extraction.
google_search: Runs a Google search and returns organic results as structured JSON. Supports up to 10 pages of results and optional proxy country selection.
Bypasses Akamai's anti-bot protection to fetch fully rendered HTML of web pages.
Bypasses Cloudflare's anti-bot protection to fetch fully rendered HTML of web pages.
Runs Google searches and returns organic results as JSON.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@ScrapeUnblocker MCP Server Get the HTML of https://example.com which is blocked by Cloudflare."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
A Model Context Protocol server that lets Claude (and any other MCP client) fetch any web page's HTML through the ScrapeUnblocker scraping API, bypassing anti-bot protection (Cloudflare, DataDome, PerimeterX, Akamai, Shape).
You bring your own API key. Nothing is shared or proxied through us.
Tool | What it does |
| Fetch the fully rendered HTML of a URL (optionally after running interactive browser |
| List a page's notable elements with a ready-to-use selector for each (selector discovery). |
| Fetch a page and return AI-parsed structured JSON. |
| Run a Google search and return organic results as JSON. |
Some pages only reveal what you need after you interact with them - accept a
cookie banner, click a tab, type into a search box and submit, or scroll to
trigger lazy loading. Pass an optional steps array to fetch_html and those
actions run in a real browser, in order, after the page loads; the resulting
HTML is then returned.
Available actions:
Action | Fields |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Steps are not idempotent - they run once per call. If a step fails,
fetch_html returns which step failed, why, and the page HTML at that moment so
you can fix the selector and retry.
list_elements loads a page and returns a JSON list of its notable elements
(links, inputs, buttons, selects, ...), each with a ready-to-use selector plus
tag, text and useful attributes (name, id, type, placeholder,
aria_label, href, ...):
{ "url": "https://example.com", "count": 42, "elements": [ { "tag": "input", "selector": "#search", "type": "text", "placeholder": "Search", "aria_label": "Search" } ] }The natural workflow is discover, then act: call list_elements to find the
selectors you need, then pass matching steps to fetch_html to click/type/
select and capture the resulting HTML.
Related MCP server: SteadyFetch
Sign up and grab your key at app.scrapeunblocker.com. The server
reads it from the SCRAPEUNBLOCKER_KEY environment variable.
The easiest route is the official plugin, which installs this server for you,
prompts for your API key (stored in your OS keychain rather than an environment
variable), and adds a /scrape-url command plus reference skills:
/plugin marketplace add ScrapeUnblocker/claude-code-plugin
/plugin install scrapeunblocker@scrapeunblockerSee ScrapeUnblocker/claude-code-plugin.
To add the bare server instead:
claude mcp add scrapeunblocker \
--env SCRAPEUNBLOCKER_KEY=your_api_key_here \
-- npx -y scrapeunblocker-mcpAdd this to your claude_desktop_config.json
(Settings → Developer → Edit Config):
{
"mcpServers": {
"scrapeunblocker": {
"command": "npx",
"args": ["-y", "scrapeunblocker-mcp"],
"env": {
"SCRAPEUNBLOCKER_KEY": "your_api_key_here"
}
}
}
}Restart Claude Desktop and the ScrapeUnblocker tools appear.
Run the server over stdio:
SCRAPEUNBLOCKER_KEY=your_api_key_here npx -y scrapeunblocker-mcp"Fetch the HTML of https://www.example-shop.com/product/123 and list the price."
"This page keeps blocking me: . Use fetch_html to get it."
"List the elements on , then use fetch_html steps to type 'laptop' into the search box, press Enter, wait for the results, and give me the HTML."
"Search Google for 'best running shoes 2026' and give me the top 5 links."
npm install
npm run build # bundles to dist/ with tsup
npm run typecheck
SCRAPEUNBLOCKER_KEY=... node dist/index.js # run the serverMIT
Fetch the fully rendered HTML of any web page through ScrapeUnblocker, bypassing anti-bot protection (Cloudflare, DataDome, PerimeterX, Akamai, Shape). Use this when a normal fetch is blocked (403/429, captcha, 'access denied') or when the page needs a real browser to render. Returns the raw HTML as text.
For pages that need interaction (accept a cookie banner, click a tab, type into a search box, scroll to trigger lazy loading) pass steps: an ordered list of browser actions run in a real browser AFTER the page loads, then the resulting HTML is returned. Workflow: first call the list_elements tool to discover the real selectors on the page, then build steps against them. Steps are NOT idempotent - they run once per call. If a step fails, this tool returns which step failed, why, and the page HTML at that moment so you can fix the selector and retry.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The absolute URL to fetch (http/https). | |
| steps | No | Optional ordered browser actions to run in a real browser after the page loads, before the HTML is captured. Use `list_elements` first to find selectors. Runs once (not idempotent). | |
| wait_value | No | The CSS selector or JS expression paired with wait_method (e.g. '#price' or 'document.readyState==="complete"'). | |
| wait_method | No | Optional render-wait strategy: 'css' waits for a selector, 'js' waits for a JS expression to be truthy. | |
| proxy_country | No | Optional ISO 3166-1 alpha-2 country code to route through, e.g. 'US', 'GB', 'DE'. | |
| sleep_seconds | No | Extra seconds to wait after load before capturing the HTML. | |
| method_timeout_seconds | No | Cap in seconds for the render-wait method. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses important behaviors: bypasses anti-bot protection, steps are not idempotent and run once per call, and failed steps return the failing step, reason, and page HTML at that moment. This is strong behavioral disclosure, though it omits potential costs, rate limits, or output size caveats.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence serves a purpose: the main behavior, the bypass value, the step workflow, the selector discovery guidance, and the failure semantics. It front-loads the core purpose and keeps related operational details together. No filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no annotations and no output schema, the description is remarkably complete. It covers the main use case, the fallback trigger, step semantics, failure behavior, return format, and the recommended workflow with list_elements. The 100% schema coverage for parameters fills the remaining gaps, so nothing critical is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds meaningful semantics beyond the schema by explaining that `steps` are an ordered list run after page load, that selectors should be discovered via list_elements, and that steps are non-idempotent. Other parameters like `wait_method` and `proxy_country` are adequately covered by the schema itself.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Fetch the fully rendered HTML of any web page') and immediately distinguishes itself by naming the anti-bot bypass use case. The scope is unambiguous and clearly separates it from the sibling tools like fetch_parsed and google_search.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells the agent when to use it: when a normal fetch is blocked (403/429, captcha, 'access denied') or when a real browser is needed for rendering. It also gives a workflow for interactive pages (use list_elements first, then build steps). It doesn't explicitly name the alternative tool for parsed content or state when not to use it, but the usage context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetch a web page through ScrapeUnblocker and return AI-parsed structured JSON instead of raw HTML (e.g. product details, article content). Best for extracting fields from product, listing or article pages without writing your own HTML parsing. If the page holds no structured data, the result says so (that call is not billed) - use fetch_html for the page itself.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The absolute URL to fetch and parse. | |
| rules_hint | No | Optional natural-language hint about what to extract, to guide parsing. | |
| proxy_country | No | Optional ISO country code to route through, e.g. 'US'. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden and does well: it discloses the ScrapeUnblocker routing layer, the output form, and a non-obvious billing rule for empty results. It does not address authentication, rate limits, or latency, so it falls short of full disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, zero padding, and the core differentiator (structured JSON vs raw HTML) is front-loaded before the usage guidance and the fallback instruction.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description usefully characterizes the return value (structured JSON with product/article fields) and the empty-result case. It is nearly complete, though it leaves the exact response envelope unspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so url, rules_hint and proxy_country are already documented in the schema. The description hints at the parsing intent behind rules_hint but adds no syntax or format detail beyond what the schema provides, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource (fetch a web page) and the distinctive output (AI-parsed structured JSON instead of raw HTML), with concrete examples of the content types returned. It explicitly distinguishes itself from the sibling fetch_html.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Names the ideal scenarios (product, listing, article pages) and names the alternative tool (fetch_html) with the condition that selects it. It also covers the failure case - no structured data present - and notes that call is unbilled, which removes ambiguity about retry behavior.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run a Google search through ScrapeUnblocker and return the organic results as structured JSON. Use this to discover URLs before fetching them.
| Name | Required | Description | Default |
|---|---|---|---|
| keyword | Yes | The search query. | |
| proxy_country | No | Optional ISO country code to search from, e.g. 'US'. | |
| pages_to_check | No | How many result pages to collect (default 1). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It adds useful context by mentioning ScrapeUnblocker as the execution path and 'organic results' as an output filter. However, it omits potential behavioral details like rate limits, pagination behavior, error responses, or any caveats about scraping Google.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the action and output, the second states the use case. Every clause contributes value, and it is front-loaded with the core purpose. No redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers what the tool does, how it does it, and when to use it. Since there is no output schema, it would benefit from specifying more about the returned JSON structure, but 'discover URLs' implies the key output. Given the simple parameter set and clear sibling relationship, this is largely complete but could be more detailed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All three parameters (keyword, proxy_country, pages_to_check) have complete schema descriptions (100% coverage), so the baseline is 3. The tool description adds no parameter-specific nuance, but it doesn't need to because the schema already explains them well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action and resource: 'Run a Google search through ScrapeUnblocker and return the organic results as structured JSON.' It clearly distinguishes this from sibling tools (fetch_html, fetch_parsed) by emphasizing search results rather than page content.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The line 'Use this to discover URLs before fetching them' gives clear usage context and positions this tool as a discovery step relative to fetch tools. It does not explicitly name alternatives or state when not to use, but the guidance is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Load a page through ScrapeUnblocker and return a JSON list of its notable elements (links, inputs, buttons, selects, etc.) with a ready-to-use selector for each, plus tag, text and useful attributes (name, id, type, placeholder, aria_label, href, ...). Read-only and does not interact with the page. This is the discovery half of interactive scraping: call list_elements to find the selectors you need, then pass matching steps to fetch_html to click/type/select and capture the resulting HTML.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The absolute URL whose elements you want to list. | |
| proxy_country | No | Optional ISO country code to route through, e.g. 'US'. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the read-only nature, that it does not interact with the page, and that it loads the page through ScrapeUnblocker. It does not mention rate limits or how 'notable' elements are selected, but for a read-only discovery tool this is reasonable coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with no filler: the first states core purpose and output, the second emphasizes read-only behavior, and the third gives the workflow context. All information earns its place and the most important details are front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Since there is no output schema, the description explains the return value (JSON list with selector, tag, text, attributes) sufficiently. It also connects the tool to the broader fetch_html workflow. Minor gaps such as empty results or blocked pages are not covered, but they are not essential for correct selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%: url is described as 'The absolute URL whose elements you want to list' and proxy_country as 'Optional ISO country code to route through'. The tool description adds no extra parameter context, but with full schema coverage the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: loads a page and returns a JSON list of notable elements with ready-to-use selectors and attributes. It also distinguishes itself from the sibling fetch_html by positioning itself as the 'discovery half of interactive scraping' and explicitly noting it is read-only.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a direct usage workflow: 'call list_elements to find the selectors you need, then pass matching steps to fetch_html'. It also clarifies that the tool does not interact with the page, implying that fetch_html is for interaction, which serves as an alternative and exclusion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.2.0fetch_html1 field changedInput schema / properties / stepsAdded value: +{
+ "description": "Optional ordered browser actions to run in a real browser after the page loads, before the HTML is captured. Use `list_elements` first to find selectors. Runs once (not idempotent).",
+ "items": {
+ "anyOf": [
+ {
+ "additionalProperties": false,
+ "description": "Wait until an element matching `selector` exists.",
+ "properties": {
+ "action": {
+ "const": "wait_for",
+ "type": "string"
+ },
+ "selector": {
+ "description": "Selector to wait for (until present).",
+ "type": "string"
+ },
+ "selector_type": {
+ "description": "How `selector` is interpreted. Defaults to 'css'.",
+ "enum": [
+ "css",
+ "xPath",
+ "className",
+ "tagName"
+ ],
+ "type": "string"
+ },
+ "timeout_ms": {
+ "exclusiveMinimum": 0,
+ "type": "integer"
+ }
+ },
+ "required": [
+ "action",
+ "selector"
+ ],
+ "type": "object"
+ },
+ {
+ "additionalProperties": false,
+ "description": "Wait until the given text appears on the page.",
+ "properties": {
+ "action": {
+ "const": "wait_for_text",
+ "type": "string"
+ },
+ "timeout_ms": {
+ "exclusiveMinimum": 0,
+ "type": "integer"
+ },
+ "value": {
+ "description": "Text to wait for anywhere on the page.",
+ "type": "string"
+ }
+ },
+ "required": [
+ "action",
+ "value"
+ ],
+ "type": "object"
+ },
+ {
+ "additionalProperties": false,
+ "description": "Pause for a fixed number of milliseconds.",
+ "properties": {
+ "action": {
+ "const": "wait",
+ "type": "string"
+ },
+ "value": {
+ "description": "Fixed pause in milliseconds.",
+ "minimum": 0,
+ "type": "integer"
+ }
+ },
+ "required": [
+ "action",
+ "value"
+ ],
+ "type": "object"
+ },
+ {
+ "additionalProperties": false,
+ "description": "Click the element matching `selector`.",
+ "properties": {
+ "action": {
+ "const": "click",
+ "type": "string"
+ },
+ "selector": {
+ "description": "Selector of the element to click.",
+ "type": "string"
+ },
+ "selector_type": {
+ "$ref": "#/properties/steps/items/anyOf/0/properties/selector_type",
+ "description": "How `selector` is interpreted. Defaults to 'css'."
+ },
+ "timeout_ms": {
+ "exclusiveMinimum": 0,
+ "type": "integer"
+ }
+ },
+ "required": [
+ "action",
+ "selector"
+ ],
+ "type": "object"
+ },
+ {
+ "additionalProperties": false,
+ "description": "Type text into an input, character by character.",
+ "properties": {
+ "action": {
+ "const": "type",
+ "type": "string"
+ },
+ "clear": {
+ "description": "Clear the field before typing.",
+ "type": "boolean"
+ },
+ "selector": {
+ "description": "Selector of the field to type into.",
+ "type": "string"
+ },
+ "selector_type": {
+ "$ref": "#/properties/steps/items/anyOf/0/properties/selector_type",
+ "description": "How `selector` is interpreted. Defaults to 'css'."
+ },
+ "timeout_ms": {
+ "exclusiveMinimum": 0,
+ "type": "integer"
+ },
+ "value": {
+ "description": "Text to type (entered human-like).",
+ "type": "string"
+ }
+ },
+ "required": [
+ "action",
+ "selector",
+ "value"
+ ],
+ "type": "object"
+ },
+ {
+ "additionalProperties": false,
+ "description": "Choose an option in a <select> dropdown by value.",
+ "properties": {
+ "action": {
+ "const": "select",
+ "type": "string"
+ },
+ "selector": {
+ "description": "Selector of the <select> element.",
+ "type": "string"
+ },
+ "selector_type": {
+ "$ref": "#/properties/steps/items/anyOf/0/properties/selector_type",
+ "description": "How `selector` is interpreted. Defaults to 'css'."
+ },
+ "timeout_ms": {
+ "exclusiveMinimum": 0,
+ "type": "integer"
+ },
+ "value": {
+ "description": "The option value to select.",
+ "type": "string"
+ }
+ },
+ "required": [
+ "action",
+ "selector",
+ "value"
+ ],
+ "type": "object"
+ },
+ {
+ "additionalProperties": false,
+ "description": "Press a single keyboard key.",
+ "properties": {
+ "action": {
+ "const": "press_key",
+ "type": "string"
+ },
+ "value": {
+ "description": "The key to press.",
+ "enum": [
+ "Enter",
+ "Tab",
+ "Escape",
+ "Backspace",
+ "Delete",
+ "Space",
+ "ArrowUp",
+ "ArrowDown",
+ "ArrowLeft",
+ "ArrowRight",
+ "Home",
+ "End",
+ "PageUp",
+ "PageDown"
+ ],
+ "type": "string"
+ }
+ },
+ "required": [
+ "action",
+ "value"
+ ],
+ "type": "object"
+ },
+ {
+ "additionalProperties": false,
+ "description": "Scroll the page to the bottom or by a pixel amount.",
+ "properties": {
+ "action": {
+ "const": "scroll",
+ "type": "string"
+ },
+ "value": {
+ "anyOf": [
+ {
+ "const": "bottom",
+ "type": "string"
+ },
+ {
+ "type": "integer"
+ }
+ ],
+ "description": "'bottom' to scroll to the end, or a pixel offset."
+ }
+ },
+ "required": [
+ "action",
+ "value"
+ ],
+ "type": "object"
+ }
+ ]
+ },
+ "type": "array"
+}list_elementsv0.1.4fetch_htmlfetch_parsedgoogle_searchv0.1.3fetch_htmlfetch_parsedgoogle_searchv0.1.2fetch_htmlfetch_parsedgoogle_searchScored across 4 tools
Each tool targets a distinct action: fetch_html (raw HTML), fetch_parsed (structured JSON), google_search (URL discovery), and list_elements (read-only selector discovery). The descriptions explicitly cross-reference each other (e.g. 'use fetch_html for the page itself'), making selection unambiguous.
All four names are snake_case and follow a verb_noun pattern (fetch_html, fetch_parsed, list_elements, google_search). The convention is uniform and predictable.
Four tools is lean and each earns its place for a scraping service, covering fetch, parse, search, and element discovery. It sits at the low end of the ideal range but is well-scoped rather than thin.
The surface covers the core scraping lifecycle: discovery (google_search), selector inspection (list_elements), raw fetch, and parsed extraction with interactive steps. Minor gaps like multi-page crawling or pagination/session handling exist but core workflows are covered.
Fetch and extract data from any public web page, even JS-rendered or anti-bot protected
Web search, browser automation, scraping, crawling and CAPTCHA solving for AI agents.
Fetch pages as markdown, search web and news, extract structured data. For AI agents.
Fetch any web page's HTML, AI-parsed JSON, or Google results via the ScrapeUnblocker anti-bot API
