WebScraping-AI MCP Server
OfficialWebScraping.AI MCP 服务器
模型上下文协议 (MCP) 服务器实现与WebScraping.AI集成,实现 Web 数据提取功能。
特征
关于网页内容的问答
从网页中提取结构化数据
使用 JavaScript 渲染的 HTML 内容检索
从网页中提取纯文本
基于 CSS 选择器的内容提取
多种代理类型(数据中心、住宅)和国家选择
使用无头 Chrome/Chromium 进行 JavaScript 渲染
具有速率限制的并发请求管理
在目标页面上执行自定义 JavaScript
设备模拟(台式机、手机、平板电脑)
账户使用情况监控
Related MCP server: Scrapfly MCP
安装
使用 npx 运行
env WEBSCRAPING_AI_API_KEY=your_api_key npx -y webscraping-ai-mcp手动安装
# Clone the repository
git clone https://github.com/webscraping-ai/webscraping-ai-mcp-server.git
cd webscraping-ai-mcp-server
# Install dependencies
npm install
# Run
npm start在光标中配置
注意:需要 Cursor 版本 0.45.6+
WebScraping.AI MCP 服务器可以在 Cursor 中以两种方式配置:
项目特定配置(推荐用于团队项目):在项目目录中创建一个
.cursor/mcp.json文件:{ "servers": { "webscraping-ai": { "type": "command", "command": "npx -y webscraping-ai-mcp", "env": { "WEBSCRAPING_AI_API_KEY": "your-api-key", "WEBSCRAPING_AI_CONCURRENCY_LIMIT": "5" } } } }全局配置(供所有项目个人使用):在您的主目录中创建一个
~/.cursor/mcp.json文件,其配置格式与上述相同。
如果您使用的是 Windows 并且遇到问题,请尝试使用
cmd /c "set WEBSCRAPING_AI_API_KEY=your-api-key && npx -y webscraping-ai-mcp"作为命令。
此配置将使 WebScraping.AI 工具在与网页抓取任务相关时自动可供 Cursor 的 AI 代理使用。
在 Claude Desktop 上运行
将其添加到您的claude_desktop_config.json中:
{
"mcpServers": {
"mcp-server-webscraping-ai": {
"command": "npx",
"args": ["-y", "webscraping-ai-mcp"],
"env": {
"WEBSCRAPING_AI_API_KEY": "YOUR_API_KEY_HERE",
"WEBSCRAPING_AI_CONCURRENCY_LIMIT": "5"
}
}
}
}配置
环境变量
必需的
WEBSCRAPING_AI_API_KEY:您的 WebScraping.AI API 密钥所有操作均必需
从WebScraping.AI获取您的 API 密钥
可选配置
WEBSCRAPING_AI_CONCURRENCY_LIMIT:最大并发请求数(默认值:5)WEBSCRAPING_AI_DEFAULT_PROXY_TYPE:使用的代理类型(默认值:residential)WEBSCRAPING_AI_DEFAULT_JS_RENDERING:启用/禁用 JavaScript 渲染(默认值:true)WEBSCRAPING_AI_DEFAULT_TIMEOUT:最大网页检索时间(毫秒)(默认值:15000,最大值:30000)WEBSCRAPING_AI_DEFAULT_JS_TIMEOUT:最大 JavaScript 渲染时间(毫秒)(默认值:2000)
配置示例
对于标准用法:
# Required
export WEBSCRAPING_AI_API_KEY=your-api-key
# Optional - customize behavior (default values)
export WEBSCRAPING_AI_CONCURRENCY_LIMIT=5
export WEBSCRAPING_AI_DEFAULT_PROXY_TYPE=residential # datacenter or residential
export WEBSCRAPING_AI_DEFAULT_JS_RENDERING=true
export WEBSCRAPING_AI_DEFAULT_TIMEOUT=15000
export WEBSCRAPING_AI_DEFAULT_JS_TIMEOUT=2000可用工具
1.问题工具( webscraping_ai_question )
询问有关网页内容的问题。
{
"name": "webscraping_ai_question",
"arguments": {
"url": "https://example.com",
"question": "What is the main topic of this page?",
"timeout": 30000,
"js": true,
"js_timeout": 2000,
"wait_for": ".content-loaded",
"proxy": "datacenter",
"country": "us"
}
}响应示例:
{
"content": [
{
"type": "text",
"text": "The main topic of this page is examples and documentation for HTML and web standards."
}
],
"isError": false
}2. 字段工具( webscraping_ai_fields )
根据指令从网页中提取结构化数据。
{
"name": "webscraping_ai_fields",
"arguments": {
"url": "https://example.com/product",
"fields": {
"title": "Extract the product title",
"price": "Extract the product price",
"description": "Extract the product description"
},
"js": true,
"timeout": 30000
}
}响应示例:
{
"content": [
{
"type": "text",
"text": {
"title": "Example Product",
"price": "$99.99",
"description": "This is an example product description."
}
}
],
"isError": false
}3. HTML工具( webscraping_ai_html )
获取带有 JavaScript 渲染的网页的完整 HTML。
{
"name": "webscraping_ai_html",
"arguments": {
"url": "https://example.com",
"js": true,
"timeout": 30000,
"wait_for": "#content-loaded"
}
}响应示例:
{
"content": [
{
"type": "text",
"text": "<html>...[full HTML content]...</html>"
}
],
"isError": false
}4.文本工具( webscraping_ai_text )
从网页中提取可见的文本内容。
{
"name": "webscraping_ai_text",
"arguments": {
"url": "https://example.com",
"js": true,
"timeout": 30000
}
}响应示例:
{
"content": [
{
"type": "text",
"text": "Example Domain\nThis domain is for use in illustrative examples in documents..."
}
],
"isError": false
}5.选定工具( webscraping_ai_selected )
使用 CSS 选择器从特定元素中提取内容。
{
"name": "webscraping_ai_selected",
"arguments": {
"url": "https://example.com",
"selector": "div.main-content",
"js": true,
"timeout": 30000
}
}响应示例:
{
"content": [
{
"type": "text",
"text": "<div class=\"main-content\">This is the main content of the page.</div>"
}
],
"isError": false
}6. 选定多个工具( webscraping_ai_selected_multiple )
使用 CSS 选择器从多个元素中提取内容。
{
"name": "webscraping_ai_selected_multiple",
"arguments": {
"url": "https://example.com",
"selectors": ["div.header", "div.product-list", "div.footer"],
"js": true,
"timeout": 30000
}
}响应示例:
{
"content": [
{
"type": "text",
"text": [
"<div class=\"header\">Header content</div>",
"<div class=\"product-list\">Product list content</div>",
"<div class=\"footer\">Footer content</div>"
]
}
],
"isError": false
}7.帐户工具( webscraping_ai_account )
获取有关您的 WebScraping.AI 帐户的信息。
{
"name": "webscraping_ai_account",
"arguments": {}
}响应示例:
{
"content": [
{
"type": "text",
"text": {
"requests": 5000,
"remaining": 4500,
"limit": 10000,
"resets_at": "2023-12-31T23:59:59Z"
}
}
],
"isError": false
}所有工具的通用选项
以下选项可与所有抓取工具一起使用:
timeout:网页检索的最大时间(毫秒)(默认为 15000,最大值为 30000)js:使用无头浏览器执行页面 JavaScript(默认为 true)js_timeout:最大 JavaScript 渲染时间(毫秒)(默认为 2000)wait_for:返回页面内容之前等待的 CSS 选择器proxy:代理类型,数据中心或住宅(默认为住宅)country:代理服务器所在国家/地区(默认为美国)。支持的国家/地区:美国、英国、德国、意大利、法国、加拿大、西班牙、俄罗斯、日本、韩国、印度custom_proxy:您自己的代理 URL,格式为“ http://user:password@host:port ”device:设备仿真类型。支持的值:桌面设备、移动设备、平板电脑error_on_404:在目标页面上返回 404 HTTP 状态错误(默认为 false)error_on_redirect:在目标页面重定向时返回错误(默认为 false)js_script:在目标页面上执行的自定义 JavaScript 代码
错误处理
服务器提供了强大的错误处理:
暂时性错误自动重试
带退避的速率限制处理
详细错误消息
网络弹性
错误响应示例:
{
"content": [
{
"type": "text",
"text": "API Error: 429 Too Many Requests"
}
],
"isError": true
}与法学硕士 (LLM) 的整合
此服务器实现了模型上下文协议 (MCP) ,使其与任何支持 MCP 的 LLM 平台兼容。您可以配置您的 LLM,以使用这些工具执行 Web 抓取任务。
示例:使用 MCP 配置 Claude
const { Claude } = require('@anthropic-ai/sdk');
const { Client } = require('@modelcontextprotocol/sdk/client/index.js');
const { StdioClientTransport } = require('@modelcontextprotocol/sdk/client/stdio.js');
const claude = new Claude({
apiKey: process.env.ANTHROPIC_API_KEY
});
const transport = new StdioClientTransport({
command: 'npx',
args: ['-y', 'webscraping-ai-mcp'],
env: {
WEBSCRAPING_AI_API_KEY: 'your-api-key'
}
});
const client = new Client({
name: 'claude-client',
version: '1.0.0'
});
await client.connect(transport);
// Now you can use Claude with WebScraping.AI tools
const tools = await client.listTools();
const response = await claude.complete({
prompt: 'What is the main topic of example.com?',
tools: tools
});发展
# Clone the repository
git clone https://github.com/webscraping-ai/webscraping-ai-mcp-server.git
cd webscraping-ai-mcp-server
# Install dependencies
npm install
# Run tests
npm test
# Add your .env file
cp .env.example .env
# Start the inspector
npx @modelcontextprotocol/inspector node src/index.js贡献
分叉存储库
创建你的功能分支
运行测试:
npm test提交拉取请求
执照
MIT 许可证 - 详情请参阅许可证文件
Available Tools
9 toolswebscraping_ai_accountGet Account StatusARead-only
Get your WebScraping.AI account status: email, remaining API credits, and concurrency. API docs: https://webscraping.ai/docs#account
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, openWorldHint=false, and destructiveHint=false, so safety is covered. The description adds value beyond them by naming the concrete returned fields (email, credits, concurrency), which is important because no output schema exists to reveal the response shape.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences: the purpose and returned fields come first, the docs link trails. Every element earns its place and nothing is padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only status tool with full annotation coverage this is close to complete: the agent knows what it does, that it is safe, and what data comes back even without an output schema. Only minor behavioral notes (credits are consumed quota, error behavior when auth is missing) are absent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters and the schema is an empty object, so there is nothing for the description to disambiguate. Baseline of 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Get your WebScraping.AI account status') and enumerates exactly what is returned: email, remaining credits, concurrency. This is unmistakably distinct from the scraping-oriented siblings (html, text, serp, data), so no schema opening is needed to disambiguate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the tool's nature but never stated: there is no guidance such as 'call this before batch jobs to verify remaining credits' and no mention of when not to use it. With one obviously non-overlapping sibling set, the implied context is adequate but thin.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_dataGet Structured Site DataARead-only
Get structured JSON for a public page on a supported site from its normal URL, e.g. a YouTube video, channel or playlist, a TikTok video or profile, an X post or profile, a LinkedIn company, job or profile, an Instagram post, reel or profile, or a Reddit post, subreddit or user. Returns {request_parameters: {url, provider, type}, parse_status, data}: provider (site) and type (page kind) are detected from the URL, parse_status is ok, parse_failed or not_found, and data holds snake_case fields whose shape depends on provider and type (null fields for values the page doesn't expose; data itself can be null when parsing fails). More sites are added on the server over time: an unsupported URL or page type returns a 400 error, not charged, whose message lists what is supported. For other sites, use webscraping_ai_fields. Priced per site (see https://webscraping.ai/docs#data), including parse_failed and not_found results; failed fetches are not charged.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Normal URL of a public page on a supported site, e.g. https://www.youtube.com/watch?v=dQw4w9WgXcQ. Sent as-is; the site and page type are detected from it. | |
| params | No | Extra site-specific query parameters sent to the API as-is, as an object of string, number or boolean values, for parameters added after this tool was released. Must not contain url, api_key or the parameters above. | |
| country | No | Two-letter country code of the proxy used to fetch the page, e.g. us, gb, de (us by default). | |
| transcript | No | YouTube videos only. Also fetch the video's transcript into data.transcript (null when no matching captions are available; false by default). If the transcript fetch fails, the whole request fails with a 500 and is not charged. | |
| transcript_language | No | YouTube videos only, with transcript: true. Caption language to pick, e.g. en, de. Without it, English is preferred, then the first available track; if the video has no captions in that language, data.transcript is null. | |
| disable_content_sandboxing | No | Return the raw result without the external-content security boundaries that guard against prompt injection (false by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With annotations covering read-only/open-world/destructive, the description adds substantial behavioral detail: the exact return envelope, parse_status values, null-field and null-data behavior, 400 and 500 error conditions, and pricing rules including that failed fetches are not charged. This goes well beyond what annotations alone provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but front-loaded: purpose, supported examples, return shape, error handling, alternative, and pricing follow in order. Every sentence carries operational information, and there is no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex, open-world tool with no output schema and six parameters, the description supplies the return structure, status semantics, failure behavior, cost model, and sibling routing. It gives the agent everything needed to decide whether and how to call it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description reinforces that provider and type are detected from the URL, but it does not add parameter-level meaning beyond the schema, nor does it discuss country, transcript, or sandboxing options.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Get structured JSON for a public page on a supported site from its normal URL.' It enumerates supported providers and page types, and explicitly names the sibling for unsupported sites ('use webscraping_ai_fields'), making it easy to distinguish from alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It states the condition for use (supported site, normal URL) and gives an explicit alternative for other sites: 'For other sites, use webscraping_ai_fields.' It also explains what happens for unsupported URLs (400 error, not charged, message lists support), which tells the agent when the tool will not work.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_fieldsExtract Fields from a PageARead-only
Scrape a web page and have an LLM extract named values from it, using only the page's own content. Returns JSON of the form {"result": {: }} with exactly the requested keys; each value is a string, or null when the page doesn't contain it. Values are never nested objects or arrays, so request a list as a single delimited string. Costs 5 credits on top of the scrape's normal cost. API docs: https://webscraping.ai/docs#ai-fields
| Name | Required | Description | Default |
|---|---|---|---|
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| url | Yes | URL of the target page. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| device | No | Type of device emulation. | |
| fields | Yes | Dictionary of field names with instructions for extraction. | |
| country | No | Country of the proxy to use (US by default). | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| js_script | No | Custom JavaScript code to execute on the target page. | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, openWorldHint=true, so the safety profile is covered. The description goes beyond that with real behavioral context: the credit surcharge on top of scrape cost, the exact response envelope, the null-when-absent rule, and the flat-string constraint on values. It does not mention rate limits or failure modes when the LLM cannot find a field.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, front-loaded with purpose and immediately followed by the response contract and the value constraint. The cost disclosure and docs link are placed last. No filler or restatement of the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description compensates well by specifying the return envelope, key set, and value types. For a 13-parameter tool whose schema is fully described, this is close to complete; the only real gap is routing guidance relative to siblings, which matters because the sibling set contains several closely related extraction tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3 and the schema already documents all 13 parameters including proxy, country, and timeout. The description nonetheless adds meaning the schema lacks: that field values are flat strings or null, never nested, and that lists must be requested as one delimited string. That is genuine elaboration on the `fields` parameter's semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific compound operation: scrape a page, then have an LLM extract named values from the page's own content. That is a clear verb+resource and is distinguishable in spirit from question-answering siblings. It stops short of naming any sibling (e.g., webscraping_ai_question or webscraping_ai_selected), so an agent must infer the boundary itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is useful in-band guidance on how to shape requests (request a list as a single delimited string) and a cost note, but no explicit when-to-use versus the other eight webscraping_ai tools and no stated exclusions. Usage is implied by the 'named values, one value per key' framing rather than spelled out.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_htmlGet Page HTMLCRead-only
Get the full rendered HTML of a web page (with JavaScript execution by default). API docs: https://webscraping.ai/docs#html
| Name | Required | Description | Default |
|---|---|---|---|
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| url | Yes | URL of the target page. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| device | No | Type of device emulation. | |
| format | No | Response format (json or text). | |
| country | No | Country of the proxy to use (US by default). | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| js_script | No | Custom JavaScript code to execute on the target page. | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). | |
| return_script_result | No | Return result of the custom JavaScript code execution. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=true, so the safety profile is covered. The description adds almost no behavioral context (no mention of fetch cost, blocking/anti-bot behavior, or failure modes), and the parenthetical 'with JavaScript execution by default' actively contradicts the schema, where js is documented as false by default.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core capability. Slightly wasteful only in that the second sentence is a bare documentation link rather than information the agent can act on.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 14-parameter fetch tool with 100% schema coverage and no output schema, the description is barely sufficient: it conveys what comes back at a high level but omits any sibling routing and states a default (JS on) that conflicts with the schema. Functional, but with clear gaps around selection and accurate defaults.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% across all 14 parameters, including enum explanations and defaults, so the schema carries the full parameter burden. The description adds nothing beyond the schema (no syntax or format hints for url, wait_for, or js_script), so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a concrete verb and resource: 'Get the full rendered HTML of a web page.' That is enough to distinguish it in spirit from siblings like webscraping_ai_text or webscraping_ai_fields, but the description never names or contrasts those alternatives, so the differentiation rests on the agent inferring 'HTML' vs 'text' on its own.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no exclusions, and no routing to the alternative siblings (text, selected, fields, question, serp) that all fetch from the same site. The only non-purpose sentence is a link to external API docs, which is a pointer rather than guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_questionAsk a Question About a PageARead-only
Scrape a web page and have an LLM answer a question about it, using only the page's own content. Returns the answer as plain text. If the page lacks the information or the question can't be answered from it, the tool returns an error result whose body.error starts with "Invalid page or question", and the credits are refunded. Costs 5 credits on top of the scrape's normal cost. Suited to a single fact or short answer; use webscraping_ai_fields for several named values at once, or webscraping_ai_text to read or reason over the whole page yourself. API docs: https://webscraping.ai/docs#ai-question
| Name | Required | Description | Default |
|---|---|---|---|
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| url | Yes | URL of the target page. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| device | No | Type of device emulation. | |
| country | No | Country of the proxy to use (US by default). | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| question | Yes | Question or instructions to ask the LLM model about the target page. | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| js_script | No | Custom JavaScript code to execute on the target page. | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations by disclosing the exact error contract (body.error starts with 'Invalid page or question'), that credits are refunded on failure, and the specific cost model (5 credits on top of the scrape's normal cost). These are behavioral traits an agent cannot infer from readOnlyHint/openWorldHint alone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads what the tool does and its return type, then routing/error/cost details, then the docs link. Every sentence conveys distinct, actionable information with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 13 parameters and no output schema, the description covers return format ('plain text'), the error shape and refund behavior, cost, and sibling routing. Nothing an agent needs to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all 13 parameters are already documented in the schema; the description adds no per-parameter detail (e.g., js, proxy, or question syntax). Baseline 3 is appropriate when the schema carries the full parameter burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb chain (scrape a page, have an LLM answer a question using only the page's own content) and the return type (plain text). The scope constraint 'using only the page's own content' distinguishes it from tools that read or manipulate the page, letting an agent separate it from siblings without opening schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly names when to use it ('a single fact or short answer') and names two alternatives with their selecting conditions: webscraping_ai_fields for several named values at once, webscraping_ai_text to read/reason over the whole page yourself. It also states the failure condition that produces an error.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_selectedGet Selected Element HTMLBRead-only
Get the HTML of the first page element matching a CSS selector. API docs: https://webscraping.ai/docs#selected
| Name | Required | Description | Default |
|---|---|---|---|
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| url | Yes | URL of the target page. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| device | No | Type of device emulation. | |
| format | No | Response format (json or text). | json |
| country | No | Country of the proxy to use (US by default). | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| selector | Yes | CSS selector to extract content for. | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| js_script | No | Custom JavaScript code to execute on the target page. | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=true, so safety is covered. The description does add one genuinely useful behavioral fact — only the FIRST matching element is returned, so extra matches are silently ignored — but says nothing about cost, rate limits, or failure behavior for a paid scraping API.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core behavior and with zero redundancy. The trailing docs URL is a reasonable pointer but occupies half the text without adding inline value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 14-parameter, mutation-free read tool with a fully documented schema, the description is adequate but minimal. It omits the first-match caveat's practical implication (no error when zero matches), and gives no sense of the response shape or cost, though the scope claim is correct.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with detailed prose on proxy tiers, timeouts, js flags, and enum options, so the schema carries the parameter burden. The description adds no syntax or default information beyond it, which makes the baseline 3 appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Get), resource (HTML of the first matching page element), and a precise selection mechanism (CSS selector), which implicitly separates it from the whole-page sibling webscraping_ai_html and the plural webscraping_ai_selected_multiple. It stops short of naming those siblings explicitly, so an agent must infer the routing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to prefer this over webscraping_ai_html, webscraping_ai_text, or webscraping_ai_selected_multiple, and no mention of prerequisites such as JS rendering being needed. The only pointer is a documentation URL, which is a reference rather than usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_selected_multipleGet Multiple Selected Elements HTMLBRead-only
Get the HTML of all page elements matching a list of CSS selectors. API docs: https://webscraping.ai/docs#selected
| Name | Required | Description | Default |
|---|---|---|---|
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| url | Yes | URL of the target page. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| device | No | Type of device emulation. | |
| country | No | Country of the proxy to use (US by default). | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| js_script | No | Custom JavaScript code to execute on the target page. | |
| selectors | Yes | Array of CSS selectors to extract content for. | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=true, so the safety profile is covered. The description adds only that all matching elements are returned as HTML; it omits rate limits, auth needs, or behavior when a selector matches nothing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences, zero waste, with the core action front-loaded and the doc link appended last. Nothing repeats the title or restates schema fields.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only extraction tool with 13 fully documented params and no output schema, the description covers the essential purpose. It could note return structure or empty-match behavior, but the rich schema and annotations carry most of the burden.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all 13 parameters including proxy, timeout, and js are fully documented in the schema. The description adds no syntax, format, or selector-array semantics beyond that, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource: 'Get the HTML of all page elements matching a list of CSS selectors.' The multi-selector scope is clear, but it never distinguishes itself from siblings like webscraping_ai_selected (singular) or webscraping_ai_html, so an agent must infer the split.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance and no exclusions. With siblings named webscraping_ai_selected, webscraping_ai_html, and webscraping_ai_text, the description should say when to prefer this over those, but it offers nothing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_serpSearch GoogleARead-only
Search Google and get parsed results as JSON: organic_results (position, title, link, domain, displayed_link, snippet, date), related_searches, search_information (including spelling corrections) and pagination. 10 results per page; position restarts at 1 on every page. Priced per search (see https://webscraping.ai/docs#serp); failed searches are not charged. Use it to find pages, then read them with webscraping_ai_text or the other tools.
| Name | Required | Description | Default |
|---|---|---|---|
| q | Yes | Search query. | |
| gl | No | Two-letter country code for the search, e.g. us, gb, de (us by default). | |
| hl | No | Two-letter language code for the results, e.g. en, de, fr (en by default). | |
| page | No | Results page number, 10 results per page (1 by default). | |
| engine | No | Search engine to query (google by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnly, non-destructive, openWorld). The description adds real behavioral context beyond them: 10 results per page, position restarting at 1 on each page, per-search pricing, and the fact that failed searches are not charged. It does not describe rate limits or result freshness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences: the return shape is front-loaded, then pagination/pricing constraints, then the routing hint. Every clause carries information an agent needs, with no filler or restated tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description takes on the burden of describing the return payload and does so by enumerating the top-level fields and key sub-fields, plus pagination and cost behavior. Nothing needed to call this tool correctly appears to be missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds semantics the schema does not carry — notably the 10-results-per-page size and the position restart behavior that governs how the 'page' parameter should be used. That is meaningful added value over the schema's bare 'Results page number' text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb+resource ('Search Google') and names the exact return payload (organic_results with position, title, link, snippet, etc.), which no sibling tool produces. An agent can distinguish it from webscraping_ai_text or webscraping_ai_html without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit usage context — 'Use it to find pages, then read them with webscraping_ai_text or the other tools' — which routes the agent to a specific downstream sibling. It stops short of stating when NOT to use it or which sibling to prefer for a direct-page read, so it is clear but not exhaustive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_textGet Page TextARead-only
Get a web page's main content as Markdown, with boilerplate stripped and structure (headings, lists, tables, links) kept. Best for reading a page or passing it to an LLM. API docs: https://webscraping.ai/docs#text
| Name | Required | Description | Default |
|---|---|---|---|
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| url | Yes | URL of the target page. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| device | No | Type of device emulation. | |
| country | No | Country of the proxy to use (US by default). | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| js_script | No | Custom JavaScript code to execute on the target page. | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| text_format | No | Format of the text response. | json |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| return_links | No | Return links from the page body text. | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=true, so safety is covered. The description adds useful transformation behavior (boilerplate removal, Markdown structure retention), but says nothing about authentication, rate limits, or how failures surface, which would matter for a network tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences plus a docs link. Purpose and output format are front-loaded, and every sentence earns its place with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, but the description compensates by stating the return format (Markdown with structure preserved). With annotations covering safety and the schema covering all 14 params, the definition is nearly complete; only the option to change response format via text_format goes unmentioned in prose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all 14 parameters (js, proxy, timeout, text_format, etc.) are already documented in the schema, including enum guidance and defaults. The description adds nothing parameter-specific, so baseline 3 is correct.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Get a web page's main content as Markdown') and describes the output shape (boilerplate stripped, structure kept). This implicitly distinguishes it from siblings like webscraping_ai_html or webscraping_ai_data, but it never names those alternatives explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Best for reading a page or passing it to an LLM' implies a use context but gives no when-not-to-use guidance and does not name the sibling tools (question, fields, html, data) that would be the alternatives for other extraction needs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
8 tool updates
v1.2.2- Added
webscraping_ai_data - Changed
webscraping_ai_fields2 fields changed- added
Input schema / additionalPropertiesAdded value: +false - removed
Input schema / properties / fields / propertyNamesRemoved value: -{ - "type": "string" -}
- Changed
webscraping_ai_html1 field changed- added
Input schema / additionalPropertiesAdded value: +false
- Changed
webscraping_ai_question1 field changed- added
Input schema / additionalPropertiesAdded value: +false
- Changed
webscraping_ai_selected1 field changed- added
Input schema / additionalPropertiesAdded value: +false
- Changed
webscraping_ai_selected_multiple1 field changed- added
Input schema / additionalPropertiesAdded value: +false
- Added
webscraping_ai_serp - Changed
webscraping_ai_text1 field changed- added
Input schema / additionalPropertiesAdded value: +false
6 tool updates
v1.0.5- Changed
webscraping_ai_fields1 field changed- changed
Input schema / properties / country / enumPrevious value: -[ - "us", - "gb", - "de", - "it", - "fr", - "ca", - "es", - "ru", - "jp", - "kr", - "in" -]New value: +[ + "us", + "gb", + "de", + "it", + "fr", + "ca", + "es", + "ru", + "jp", + "kr", + "in", + "hk", + "tr" +]
- Changed
webscraping_ai_html1 field changed- changed
Input schema / properties / country / enumPrevious value: -[ - "us", - "gb", - "de", - "it", - "fr", - "ca", - "es", - "ru", - "jp", - "kr", - "in" -]New value: +[ + "us", + "gb", + "de", + "it", + "fr", + "ca", + "es", + "ru", + "jp", + "kr", + "in", + "hk", + "tr" +]
- Changed
webscraping_ai_question1 field changed- changed
Input schema / properties / country / enumPrevious value: -[ - "us", - "gb", - "de", - "it", - "fr", - "ca", - "es", - "ru", - "jp", - "kr", - "in" -]New value: +[ + "us", + "gb", + "de", + "it", + "fr", + "ca", + "es", + "ru", + "jp", + "kr", + "in", + "hk", + "tr" +]
- Changed
webscraping_ai_selected1 field changed- changed
Input schema / properties / country / enumPrevious value: -[ - "us", - "gb", - "de", - "it", - "fr", - "ca", - "es", - "ru", - "jp", - "kr", - "in" -]New value: +[ + "us", + "gb", + "de", + "it", + "fr", + "ca", + "es", + "ru", + "jp", + "kr", + "in", + "hk", + "tr" +]
- Changed
webscraping_ai_selected_multiple1 field changed- changed
Input schema / properties / country / enumPrevious value: -[ - "us", - "gb", - "de", - "it", - "fr", - "ca", - "es", - "ru", - "jp", - "kr", - "in" -]New value: +[ + "us", + "gb", + "de", + "it", + "fr", + "ca", + "es", + "ru", + "jp", + "kr", + "in", + "hk", + "tr" +]
- Changed
webscraping_ai_text1 field changed- changed
Input schema / properties / country / enumPrevious value: -[ - "us", - "gb", - "de", - "it", - "fr", - "ca", - "es", - "ru", - "jp", - "kr", - "in" -]New value: +[ + "us", + "gb", + "de", + "it", + "fr", + "ca", + "es", + "ru", + "jp", + "kr", + "in", + "hk", + "tr" +]
6 tool updates
v1.0.4- Changed
webscraping_ai_fields2 fields changed- changed
Input schema / properties / proxy / descriptionPrevious value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page." - changed
Input schema / properties / proxy / enumPrevious value: -[ - "datacenter", - "residential" -]New value: +[ + "datacenter", + "residential", + "stealth" +]
- Changed
webscraping_ai_html2 fields changed- changed
Input schema / properties / proxy / descriptionPrevious value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page." - changed
Input schema / properties / proxy / enumPrevious value: -[ - "datacenter", - "residential" -]New value: +[ + "datacenter", + "residential", + "stealth" +]
- Changed
webscraping_ai_question2 fields changed- changed
Input schema / properties / proxy / descriptionPrevious value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page." - changed
Input schema / properties / proxy / enumPrevious value: -[ - "datacenter", - "residential" -]New value: +[ + "datacenter", + "residential", + "stealth" +]
- Changed
webscraping_ai_selected2 fields changed- changed
Input schema / properties / proxy / descriptionPrevious value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page." - changed
Input schema / properties / proxy / enumPrevious value: -[ - "datacenter", - "residential" -]New value: +[ + "datacenter", + "residential", + "stealth" +]
- Changed
webscraping_ai_selected_multiple2 fields changed- changed
Input schema / properties / proxy / descriptionPrevious value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page." - changed
Input schema / properties / proxy / enumPrevious value: -[ - "datacenter", - "residential" -]New value: +[ + "datacenter", + "residential", + "stealth" +]
- Changed
webscraping_ai_text2 fields changed- changed
Input schema / properties / proxy / descriptionPrevious value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page." - changed
Input schema / properties / proxy / enumPrevious value: -[ - "datacenter", - "residential" -]New value: +[ + "datacenter", + "residential", + "stealth" +]
7 tool updates
v1.0.2- Added
webscraping_ai_account - Added
webscraping_ai_fields - Added
webscraping_ai_html - Added
webscraping_ai_question - Added
webscraping_ai_selected - Added
webscraping_ai_selected_multiple - Added
webscraping_ai_text
TDQS
Scored across 9 tools
Most tools have clearly distinct purposes: html/text/selected return page content, serp searches, data handles known provider pages, and account handles billing status. The AI tools (question, fields, text) and the selector tools (selected vs selected_multiple) overlap somewhat, but the descriptions explicitly cross-reference each other and clarify when to use which.
All tools share a predictable webscraping_ai_ prefix and use a consistent noun/adjective style (question, fields, html, text, selected, serp, data, account). There is no verb_noun pattern, and selected vs selected_multiple is a minor deviation, but the set is highly consistent overall.
Nine tools is well-scoped for a web scraping API client, covering retrieval, AI extraction, structured data, search, and account status without redundancy. Each tool maps to a distinct endpoint or capability.
The surface covers the core scraping lifecycle: raw HTML, rendered text, CSS selection, AI question/field extraction, SERP search, provider-specific structured data, and account status. Minor gaps exist, such as screenshot or other advanced scraping options, but agents can accomplish most scraping tasks with this set.
Maintenance
Related MCP Connectors
Scrape any web page, search Google/Amazon/Walmart/YouTube and extract data via the ScrapingBee API.
181Fetch and process content from specified URLs & sources using the Oxylabs Web Scraper API.
The most accurate web access API. Stop getting blocked.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables web scraping and document processing with JavaScript execution, anti-detection measures, batch processing, and structured data extraction. Supports multiple formats including markdown, HTML, screenshots, and handles PDFs with OCR capabilities.4MIT
- AlicenseAqualityBmaintenanceGives AI assistants real-time access to web data with the ability to scrape any website, bypass anti-bot systems, extract structured data, and capture screenshots through enterprise-grade infrastructure.6290 npm14ISC

ScrapeOps MCP Serverofficial
AlicenseAqualityCmaintenanceEnables AI agents to browse the web, bypass anti-bots, render JavaScript, take screenshots, and perform structured data extraction using the ScrapeOps Proxy API.313 npmMIT- AlicenseNot gradedqualityBmaintenanceEnables headless-Chromium web automation, including rendering fully-executed HTML, extracting structured data with CSS selectors, capturing screenshots, and AI-assisted smart scraping of web pages.371 npmMIT