WebScraping-AI MCP Server
OfficialWebScraping.AI MCPサーバー
Web データ抽出機能のためにWebScraping.AIと統合するモデル コンテキスト プロトコル (MCP) サーバー実装。
特徴
ウェブページの内容に関する質問への回答
ウェブページからの構造化データ抽出
JavaScript レンダリングによる HTML コンテンツの取得
ウェブページからのプレーンテキスト抽出
CSSセレクタベースのコンテンツ抽出
国を選択できる複数のプロキシタイプ(データセンター、住宅)
ヘッドレス Chrome/Chromium を使用した JavaScript レンダリング
レート制限による同時リクエスト管理
対象ページでのカスタム JavaScript 実行
デバイスエミュレーション(デスクトップ、モバイル、タブレット)
アカウント使用状況の監視
Related MCP server: Scrapfly MCP
インストール
npxで実行
env WEBSCRAPING_AI_API_KEY=your_api_key npx -y webscraping-ai-mcp手動インストール
# Clone the repository
git clone https://github.com/webscraping-ai/webscraping-ai-mcp-server.git
cd webscraping-ai-mcp-server
# Install dependencies
npm install
# Run
npm startカーソルでの設定
注: カーソルバージョン 0.45.6 以上が必要です
WebScraping.AI MCP サーバーは、Cursor で 2 つの方法で構成できます。
プロジェクト固有の構成(チーム プロジェクトに推奨): プロジェクト ディレクトリに
.cursor/mcp.jsonファイルを作成します。{ "servers": { "webscraping-ai": { "type": "command", "command": "npx -y webscraping-ai-mcp", "env": { "WEBSCRAPING_AI_API_KEY": "your-api-key", "WEBSCRAPING_AI_CONCURRENCY_LIMIT": "5" } } } }グローバル構成(すべてのプロジェクトでの個人使用): 上記と同じ構成形式で、ホーム ディレクトリに
~/.cursor/mcp.jsonファイルを作成します。
Windows を使用していて問題が発生している場合は、コマンドとして
cmd /c "set WEBSCRAPING_AI_API_KEY=your-api-key && npx -y webscraping-ai-mcp"を使用してみてください。
この構成により、Web スクレイピング タスクに関連する場合に、WebScraping.AI ツールが Cursor の AI エージェントで自動的に使用できるようになります。
Claude Desktopで実行中
これをclaude_desktop_config.jsonに追加します:
{
"mcpServers": {
"mcp-server-webscraping-ai": {
"command": "npx",
"args": ["-y", "webscraping-ai-mcp"],
"env": {
"WEBSCRAPING_AI_API_KEY": "YOUR_API_KEY_HERE",
"WEBSCRAPING_AI_CONCURRENCY_LIMIT": "5"
}
}
}
}構成
環境変数
必須
WEBSCRAPING_AI_API_KEY: WebScraping.AI APIキーすべての操作に必要
WebScraping.AIからAPIキーを取得する
オプション構成
WEBSCRAPING_AI_CONCURRENCY_LIMIT: 同時リクエストの最大数(デフォルト:5)WEBSCRAPING_AI_DEFAULT_PROXY_TYPE: 使用するプロキシの種類(デフォルト:residential)WEBSCRAPING_AI_DEFAULT_JS_RENDERING: JavaScript レンダリングを有効/無効にする (デフォルト:true)WEBSCRAPING_AI_DEFAULT_TIMEOUT: ウェブページの最大取得時間(ミリ秒)(デフォルト:15000、最大:30000)WEBSCRAPING_AI_DEFAULT_JS_TIMEOUT: JavaScript レンダリングの最大時間(ミリ秒)(デフォルト:2000)
設定例
標準的な使用法:
# Required
export WEBSCRAPING_AI_API_KEY=your-api-key
# Optional - customize behavior (default values)
export WEBSCRAPING_AI_CONCURRENCY_LIMIT=5
export WEBSCRAPING_AI_DEFAULT_PROXY_TYPE=residential # datacenter or residential
export WEBSCRAPING_AI_DEFAULT_JS_RENDERING=true
export WEBSCRAPING_AI_DEFAULT_TIMEOUT=15000
export WEBSCRAPING_AI_DEFAULT_JS_TIMEOUT=2000利用可能なツール
1. 質問ツール ( webscraping_ai_question )
Web ページのコンテンツについて質問します。
{
"name": "webscraping_ai_question",
"arguments": {
"url": "https://example.com",
"question": "What is the main topic of this page?",
"timeout": 30000,
"js": true,
"js_timeout": 2000,
"wait_for": ".content-loaded",
"proxy": "datacenter",
"country": "us"
}
}応答例:
{
"content": [
{
"type": "text",
"text": "The main topic of this page is examples and documentation for HTML and web standards."
}
],
"isError": false
}2. フィールドツール( webscraping_ai_fields )
指示に基づいて Web ページから構造化データを抽出します。
{
"name": "webscraping_ai_fields",
"arguments": {
"url": "https://example.com/product",
"fields": {
"title": "Extract the product title",
"price": "Extract the product price",
"description": "Extract the product description"
},
"js": true,
"timeout": 30000
}
}応答例:
{
"content": [
{
"type": "text",
"text": {
"title": "Example Product",
"price": "$99.99",
"description": "This is an example product description."
}
}
],
"isError": false
}3. HTMLツール( webscraping_ai_html )
JavaScript レンダリングを使用して Web ページの完全な HTML を取得します。
{
"name": "webscraping_ai_html",
"arguments": {
"url": "https://example.com",
"js": true,
"timeout": 30000,
"wait_for": "#content-loaded"
}
}応答例:
{
"content": [
{
"type": "text",
"text": "<html>...[full HTML content]...</html>"
}
],
"isError": false
}4. テキストツール ( webscraping_ai_text )
Web ページから表示されているテキスト コンテンツを抽出します。
{
"name": "webscraping_ai_text",
"arguments": {
"url": "https://example.com",
"js": true,
"timeout": 30000
}
}応答例:
{
"content": [
{
"type": "text",
"text": "Example Domain\nThis domain is for use in illustrative examples in documents..."
}
],
"isError": false
}5. 選択されたツール ( webscraping_ai_selected )
CSS セレクターを使用して特定の要素からコンテンツを抽出します。
{
"name": "webscraping_ai_selected",
"arguments": {
"url": "https://example.com",
"selector": "div.main-content",
"js": true,
"timeout": 30000
}
}応答例:
{
"content": [
{
"type": "text",
"text": "<div class=\"main-content\">This is the main content of the page.</div>"
}
],
"isError": false
}6. 複数選択ツール ( webscraping_ai_selected_multiple )
CSS セレクターを使用して複数の要素からコンテンツを抽出します。
{
"name": "webscraping_ai_selected_multiple",
"arguments": {
"url": "https://example.com",
"selectors": ["div.header", "div.product-list", "div.footer"],
"js": true,
"timeout": 30000
}
}応答例:
{
"content": [
{
"type": "text",
"text": [
"<div class=\"header\">Header content</div>",
"<div class=\"product-list\">Product list content</div>",
"<div class=\"footer\">Footer content</div>"
]
}
],
"isError": false
}7. アカウントツール( webscraping_ai_account )
WebScraping.AI アカウントに関する情報を取得します。
{
"name": "webscraping_ai_account",
"arguments": {}
}応答例:
{
"content": [
{
"type": "text",
"text": {
"requests": 5000,
"remaining": 4500,
"limit": 10000,
"resets_at": "2023-12-31T23:59:59Z"
}
}
],
"isError": false
}すべてのツールに共通するオプション
次のオプションは、すべてのスクレイピング ツールで使用できます。
timeout: ウェブページの最大取得時間(ミリ秒)(デフォルトは15000、最大値は30000)js: ヘッドレスブラウザを使用してページ上の JavaScript を実行します(デフォルトでは true)js_timeout: JavaScript レンダリングの最大時間(ミリ秒)(デフォルトは 2000)wait_for: ページコンテンツを返す前に待機する CSS セレクタproxy: プロキシの種類、データセンターまたは住宅(デフォルトは住宅)country: 使用するプロキシの国(デフォルトは米国)。サポートされている国: us、gb、de、it、fr、ca、es、ru、jp、kr、incustom_proxy: 「 http://user:password@host:port 」形式の独自のプロキシ URLdevice: デバイスエミュレーションの種類。サポートされる値: デスクトップ、モバイル、タブレットerror_on_404: 対象ページの 404 HTTP ステータスでエラーを返します (デフォルトでは false)error_on_redirect: 対象ページでのリダイレクト時にエラーを返す(デフォルトは false)js_script: 対象ページで実行するカスタム JavaScript コード
エラー処理
サーバーは堅牢なエラー処理を提供します。
一時的なエラーの自動再試行
バックオフによるレート制限の処理
詳細なエラーメッセージ
ネットワークの回復力
エラー応答の例:
{
"content": [
{
"type": "text",
"text": "API Error: 429 Too Many Requests"
}
],
"isError": true
}LLMとの統合
このサーバーはモデルコンテキストプロトコル(MCP)を実装しており、MCP対応のLLMプラットフォームと互換性があります。これらのツールをWebスクレイピングタスクに使用するようにLLMを設定できます。
例: MCP を使用した Claude の設定
const { Claude } = require('@anthropic-ai/sdk');
const { Client } = require('@modelcontextprotocol/sdk/client/index.js');
const { StdioClientTransport } = require('@modelcontextprotocol/sdk/client/stdio.js');
const claude = new Claude({
apiKey: process.env.ANTHROPIC_API_KEY
});
const transport = new StdioClientTransport({
command: 'npx',
args: ['-y', 'webscraping-ai-mcp'],
env: {
WEBSCRAPING_AI_API_KEY: 'your-api-key'
}
});
const client = new Client({
name: 'claude-client',
version: '1.0.0'
});
await client.connect(transport);
// Now you can use Claude with WebScraping.AI tools
const tools = await client.listTools();
const response = await claude.complete({
prompt: 'What is the main topic of example.com?',
tools: tools
});発達
# Clone the repository
git clone https://github.com/webscraping-ai/webscraping-ai-mcp-server.git
cd webscraping-ai-mcp-server
# Install dependencies
npm install
# Run tests
npm test
# Add your .env file
cp .env.example .env
# Start the inspector
npx @modelcontextprotocol/inspector node src/index.js貢献
リポジトリをフォークする
機能ブランチを作成する
テストを実行:
npm testプルリクエストを送信する
ライセンス
MITライセンス - 詳細はLICENSEファイルを参照
Available Tools
9 toolswebscraping_ai_accountGet Account StatusARead-only
Get your WebScraping.AI account status: email, remaining API credits, and concurrency. API docs: https://webscraping.ai/docs#account
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, openWorldHint=false, and destructiveHint=false, so safety is covered. The description adds value beyond them by naming the concrete returned fields (email, credits, concurrency), which is important because no output schema exists to reveal the response shape.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences: the purpose and returned fields come first, the docs link trails. Every element earns its place and nothing is padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only status tool with full annotation coverage this is close to complete: the agent knows what it does, that it is safe, and what data comes back even without an output schema. Only minor behavioral notes (credits are consumed quota, error behavior when auth is missing) are absent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters and the schema is an empty object, so there is nothing for the description to disambiguate. Baseline of 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Get your WebScraping.AI account status') and enumerates exactly what is returned: email, remaining credits, concurrency. This is unmistakably distinct from the scraping-oriented siblings (html, text, serp, data), so no schema opening is needed to disambiguate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the tool's nature but never stated: there is no guidance such as 'call this before batch jobs to verify remaining credits' and no mention of when not to use it. With one obviously non-overlapping sibling set, the implied context is adequate but thin.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_dataGet Structured Site DataARead-only
Get structured JSON for a public page on a supported site from its normal URL, e.g. a YouTube video, channel or playlist, a TikTok video or profile, an X post or profile, a LinkedIn company, job or profile, an Instagram post, reel or profile, or a Reddit post, subreddit or user. Returns {request_parameters: {url, provider, type}, parse_status, data}: provider (site) and type (page kind) are detected from the URL, parse_status is ok, parse_failed or not_found, and data holds snake_case fields whose shape depends on provider and type (null fields for values the page doesn't expose; data itself can be null when parsing fails). More sites are added on the server over time: an unsupported URL or page type returns a 400 error, not charged, whose message lists what is supported. For other sites, use webscraping_ai_fields. Priced per site (see https://webscraping.ai/docs#data), including parse_failed and not_found results; failed fetches are not charged.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Normal URL of a public page on a supported site, e.g. https://www.youtube.com/watch?v=dQw4w9WgXcQ. Sent as-is; the site and page type are detected from it. | |
| params | No | Extra site-specific query parameters sent to the API as-is, as an object of string, number or boolean values, for parameters added after this tool was released. Must not contain url, api_key or the parameters above. | |
| country | No | Two-letter country code of the proxy used to fetch the page, e.g. us, gb, de (us by default). | |
| transcript | No | YouTube videos only. Also fetch the video's transcript into data.transcript (null when no matching captions are available; false by default). If the transcript fetch fails, the whole request fails with a 500 and is not charged. | |
| transcript_language | No | YouTube videos only, with transcript: true. Caption language to pick, e.g. en, de. Without it, English is preferred, then the first available track; if the video has no captions in that language, data.transcript is null. | |
| disable_content_sandboxing | No | Return the raw result without the external-content security boundaries that guard against prompt injection (false by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With annotations covering read-only/open-world/destructive, the description adds substantial behavioral detail: the exact return envelope, parse_status values, null-field and null-data behavior, 400 and 500 error conditions, and pricing rules including that failed fetches are not charged. This goes well beyond what annotations alone provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but front-loaded: purpose, supported examples, return shape, error handling, alternative, and pricing follow in order. Every sentence carries operational information, and there is no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex, open-world tool with no output schema and six parameters, the description supplies the return structure, status semantics, failure behavior, cost model, and sibling routing. It gives the agent everything needed to decide whether and how to call it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description reinforces that provider and type are detected from the URL, but it does not add parameter-level meaning beyond the schema, nor does it discuss country, transcript, or sandboxing options.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Get structured JSON for a public page on a supported site from its normal URL.' It enumerates supported providers and page types, and explicitly names the sibling for unsupported sites ('use webscraping_ai_fields'), making it easy to distinguish from alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It states the condition for use (supported site, normal URL) and gives an explicit alternative for other sites: 'For other sites, use webscraping_ai_fields.' It also explains what happens for unsupported URLs (400 error, not charged, message lists support), which tells the agent when the tool will not work.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_fieldsExtract Fields from a PageARead-only
Scrape a web page and have an LLM extract named values from it, using only the page's own content. Returns JSON of the form {"result": {: }} with exactly the requested keys; each value is a string, or null when the page doesn't contain it. Values are never nested objects or arrays, so request a list as a single delimited string. Costs 5 credits on top of the scrape's normal cost. API docs: https://webscraping.ai/docs#ai-fields
| Name | Required | Description | Default |
|---|---|---|---|
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| url | Yes | URL of the target page. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| device | No | Type of device emulation. | |
| fields | Yes | Dictionary of field names with instructions for extraction. | |
| country | No | Country of the proxy to use (US by default). | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| js_script | No | Custom JavaScript code to execute on the target page. | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, openWorldHint=true, so the safety profile is covered. The description goes beyond that with real behavioral context: the credit surcharge on top of scrape cost, the exact response envelope, the null-when-absent rule, and the flat-string constraint on values. It does not mention rate limits or failure modes when the LLM cannot find a field.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, front-loaded with purpose and immediately followed by the response contract and the value constraint. The cost disclosure and docs link are placed last. No filler or restatement of the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description compensates well by specifying the return envelope, key set, and value types. For a 13-parameter tool whose schema is fully described, this is close to complete; the only real gap is routing guidance relative to siblings, which matters because the sibling set contains several closely related extraction tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3 and the schema already documents all 13 parameters including proxy, country, and timeout. The description nonetheless adds meaning the schema lacks: that field values are flat strings or null, never nested, and that lists must be requested as one delimited string. That is genuine elaboration on the `fields` parameter's semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific compound operation: scrape a page, then have an LLM extract named values from the page's own content. That is a clear verb+resource and is distinguishable in spirit from question-answering siblings. It stops short of naming any sibling (e.g., webscraping_ai_question or webscraping_ai_selected), so an agent must infer the boundary itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is useful in-band guidance on how to shape requests (request a list as a single delimited string) and a cost note, but no explicit when-to-use versus the other eight webscraping_ai tools and no stated exclusions. Usage is implied by the 'named values, one value per key' framing rather than spelled out.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_htmlGet Page HTMLCRead-only
Get the full rendered HTML of a web page (with JavaScript execution by default). API docs: https://webscraping.ai/docs#html
| Name | Required | Description | Default |
|---|---|---|---|
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| url | Yes | URL of the target page. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| device | No | Type of device emulation. | |
| format | No | Response format (json or text). | |
| country | No | Country of the proxy to use (US by default). | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| js_script | No | Custom JavaScript code to execute on the target page. | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). | |
| return_script_result | No | Return result of the custom JavaScript code execution. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=true, so the safety profile is covered. The description adds almost no behavioral context (no mention of fetch cost, blocking/anti-bot behavior, or failure modes), and the parenthetical 'with JavaScript execution by default' actively contradicts the schema, where js is documented as false by default.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core capability. Slightly wasteful only in that the second sentence is a bare documentation link rather than information the agent can act on.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 14-parameter fetch tool with 100% schema coverage and no output schema, the description is barely sufficient: it conveys what comes back at a high level but omits any sibling routing and states a default (JS on) that conflicts with the schema. Functional, but with clear gaps around selection and accurate defaults.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% across all 14 parameters, including enum explanations and defaults, so the schema carries the full parameter burden. The description adds nothing beyond the schema (no syntax or format hints for url, wait_for, or js_script), so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a concrete verb and resource: 'Get the full rendered HTML of a web page.' That is enough to distinguish it in spirit from siblings like webscraping_ai_text or webscraping_ai_fields, but the description never names or contrasts those alternatives, so the differentiation rests on the agent inferring 'HTML' vs 'text' on its own.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no exclusions, and no routing to the alternative siblings (text, selected, fields, question, serp) that all fetch from the same site. The only non-purpose sentence is a link to external API docs, which is a pointer rather than guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_questionAsk a Question About a PageARead-only
Scrape a web page and have an LLM answer a question about it, using only the page's own content. Returns the answer as plain text. If the page lacks the information or the question can't be answered from it, the tool returns an error result whose body.error starts with "Invalid page or question", and the credits are refunded. Costs 5 credits on top of the scrape's normal cost. Suited to a single fact or short answer; use webscraping_ai_fields for several named values at once, or webscraping_ai_text to read or reason over the whole page yourself. API docs: https://webscraping.ai/docs#ai-question
| Name | Required | Description | Default |
|---|---|---|---|
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| url | Yes | URL of the target page. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| device | No | Type of device emulation. | |
| country | No | Country of the proxy to use (US by default). | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| question | Yes | Question or instructions to ask the LLM model about the target page. | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| js_script | No | Custom JavaScript code to execute on the target page. | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations by disclosing the exact error contract (body.error starts with 'Invalid page or question'), that credits are refunded on failure, and the specific cost model (5 credits on top of the scrape's normal cost). These are behavioral traits an agent cannot infer from readOnlyHint/openWorldHint alone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads what the tool does and its return type, then routing/error/cost details, then the docs link. Every sentence conveys distinct, actionable information with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 13 parameters and no output schema, the description covers return format ('plain text'), the error shape and refund behavior, cost, and sibling routing. Nothing an agent needs to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all 13 parameters are already documented in the schema; the description adds no per-parameter detail (e.g., js, proxy, or question syntax). Baseline 3 is appropriate when the schema carries the full parameter burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb chain (scrape a page, have an LLM answer a question using only the page's own content) and the return type (plain text). The scope constraint 'using only the page's own content' distinguishes it from tools that read or manipulate the page, letting an agent separate it from siblings without opening schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly names when to use it ('a single fact or short answer') and names two alternatives with their selecting conditions: webscraping_ai_fields for several named values at once, webscraping_ai_text to read/reason over the whole page yourself. It also states the failure condition that produces an error.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_selectedGet Selected Element HTMLBRead-only
Get the HTML of the first page element matching a CSS selector. API docs: https://webscraping.ai/docs#selected
| Name | Required | Description | Default |
|---|---|---|---|
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| url | Yes | URL of the target page. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| device | No | Type of device emulation. | |
| format | No | Response format (json or text). | json |
| country | No | Country of the proxy to use (US by default). | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| selector | Yes | CSS selector to extract content for. | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| js_script | No | Custom JavaScript code to execute on the target page. | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=true, so safety is covered. The description does add one genuinely useful behavioral fact — only the FIRST matching element is returned, so extra matches are silently ignored — but says nothing about cost, rate limits, or failure behavior for a paid scraping API.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core behavior and with zero redundancy. The trailing docs URL is a reasonable pointer but occupies half the text without adding inline value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 14-parameter, mutation-free read tool with a fully documented schema, the description is adequate but minimal. It omits the first-match caveat's practical implication (no error when zero matches), and gives no sense of the response shape or cost, though the scope claim is correct.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with detailed prose on proxy tiers, timeouts, js flags, and enum options, so the schema carries the parameter burden. The description adds no syntax or default information beyond it, which makes the baseline 3 appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Get), resource (HTML of the first matching page element), and a precise selection mechanism (CSS selector), which implicitly separates it from the whole-page sibling webscraping_ai_html and the plural webscraping_ai_selected_multiple. It stops short of naming those siblings explicitly, so an agent must infer the routing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to prefer this over webscraping_ai_html, webscraping_ai_text, or webscraping_ai_selected_multiple, and no mention of prerequisites such as JS rendering being needed. The only pointer is a documentation URL, which is a reference rather than usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_selected_multipleGet Multiple Selected Elements HTMLBRead-only
Get the HTML of all page elements matching a list of CSS selectors. API docs: https://webscraping.ai/docs#selected
| Name | Required | Description | Default |
|---|---|---|---|
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| url | Yes | URL of the target page. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| device | No | Type of device emulation. | |
| country | No | Country of the proxy to use (US by default). | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| js_script | No | Custom JavaScript code to execute on the target page. | |
| selectors | Yes | Array of CSS selectors to extract content for. | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=true, so the safety profile is covered. The description adds only that all matching elements are returned as HTML; it omits rate limits, auth needs, or behavior when a selector matches nothing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences, zero waste, with the core action front-loaded and the doc link appended last. Nothing repeats the title or restates schema fields.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only extraction tool with 13 fully documented params and no output schema, the description covers the essential purpose. It could note return structure or empty-match behavior, but the rich schema and annotations carry most of the burden.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all 13 parameters including proxy, timeout, and js are fully documented in the schema. The description adds no syntax, format, or selector-array semantics beyond that, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource: 'Get the HTML of all page elements matching a list of CSS selectors.' The multi-selector scope is clear, but it never distinguishes itself from siblings like webscraping_ai_selected (singular) or webscraping_ai_html, so an agent must infer the split.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance and no exclusions. With siblings named webscraping_ai_selected, webscraping_ai_html, and webscraping_ai_text, the description should say when to prefer this over those, but it offers nothing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_serpSearch GoogleARead-only
Search Google and get parsed results as JSON: organic_results (position, title, link, domain, displayed_link, snippet, date), related_searches, search_information (including spelling corrections) and pagination. 10 results per page; position restarts at 1 on every page. Priced per search (see https://webscraping.ai/docs#serp); failed searches are not charged. Use it to find pages, then read them with webscraping_ai_text or the other tools.
| Name | Required | Description | Default |
|---|---|---|---|
| q | Yes | Search query. | |
| gl | No | Two-letter country code for the search, e.g. us, gb, de (us by default). | |
| hl | No | Two-letter language code for the results, e.g. en, de, fr (en by default). | |
| page | No | Results page number, 10 results per page (1 by default). | |
| engine | No | Search engine to query (google by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnly, non-destructive, openWorld). The description adds real behavioral context beyond them: 10 results per page, position restarting at 1 on each page, per-search pricing, and the fact that failed searches are not charged. It does not describe rate limits or result freshness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences: the return shape is front-loaded, then pagination/pricing constraints, then the routing hint. Every clause carries information an agent needs, with no filler or restated tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description takes on the burden of describing the return payload and does so by enumerating the top-level fields and key sub-fields, plus pagination and cost behavior. Nothing needed to call this tool correctly appears to be missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds semantics the schema does not carry — notably the 10-results-per-page size and the position restart behavior that governs how the 'page' parameter should be used. That is meaningful added value over the schema's bare 'Results page number' text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb+resource ('Search Google') and names the exact return payload (organic_results with position, title, link, snippet, etc.), which no sibling tool produces. An agent can distinguish it from webscraping_ai_text or webscraping_ai_html without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit usage context — 'Use it to find pages, then read them with webscraping_ai_text or the other tools' — which routes the agent to a specific downstream sibling. It stops short of stating when NOT to use it or which sibling to prefer for a direct-page read, so it is clear but not exhaustive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
webscraping_ai_textGet Page TextARead-only
Get a web page's main content as Markdown, with boilerplate stripped and structure (headings, lists, tables, links) kept. Best for reading a page or passing it to an LLM. API docs: https://webscraping.ai/docs#text
| Name | Required | Description | Default |
|---|---|---|---|
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| url | Yes | URL of the target page. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| device | No | Type of device emulation. | |
| country | No | Country of the proxy to use (US by default). | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| js_script | No | Custom JavaScript code to execute on the target page. | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| text_format | No | Format of the text response. | json |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| return_links | No | Return links from the page body text. | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=true, so safety is covered. The description adds useful transformation behavior (boilerplate removal, Markdown structure retention), but says nothing about authentication, rate limits, or how failures surface, which would matter for a network tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences plus a docs link. Purpose and output format are front-loaded, and every sentence earns its place with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, but the description compensates by stating the return format (Markdown with structure preserved). With annotations covering safety and the schema covering all 14 params, the definition is nearly complete; only the option to change response format via text_format goes unmentioned in prose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all 14 parameters (js, proxy, timeout, text_format, etc.) are already documented in the schema, including enum guidance and defaults. The description adds nothing parameter-specific, so baseline 3 is correct.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Get a web page's main content as Markdown') and describes the output shape (boilerplate stripped, structure kept). This implicitly distinguishes it from siblings like webscraping_ai_html or webscraping_ai_data, but it never names those alternatives explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Best for reading a page or passing it to an LLM' implies a use context but gives no when-not-to-use guidance and does not name the sibling tools (question, fields, html, data) that would be the alternatives for other extraction needs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
8 tool updates
v1.2.2- Added
webscraping_ai_data - Changed
webscraping_ai_fields2 fields changed- added
Input schema / additionalPropertiesAdded value: +false - removed
Input schema / properties / fields / propertyNamesRemoved value: -{ - "type": "string" -}
- Changed
webscraping_ai_html1 field changed- added
Input schema / additionalPropertiesAdded value: +false
- Changed
webscraping_ai_question1 field changed- added
Input schema / additionalPropertiesAdded value: +false
- Changed
webscraping_ai_selected1 field changed- added
Input schema / additionalPropertiesAdded value: +false
- Changed
webscraping_ai_selected_multiple1 field changed- added
Input schema / additionalPropertiesAdded value: +false
- Added
webscraping_ai_serp - Changed
webscraping_ai_text1 field changed- added
Input schema / additionalPropertiesAdded value: +false
6 tool updates
v1.0.5- Changed
webscraping_ai_fields1 field changed- changed
Input schema / properties / country / enumPrevious value: -[ - "us", - "gb", - "de", - "it", - "fr", - "ca", - "es", - "ru", - "jp", - "kr", - "in" -]New value: +[ + "us", + "gb", + "de", + "it", + "fr", + "ca", + "es", + "ru", + "jp", + "kr", + "in", + "hk", + "tr" +]
- Changed
webscraping_ai_html1 field changed- changed
Input schema / properties / country / enumPrevious value: -[ - "us", - "gb", - "de", - "it", - "fr", - "ca", - "es", - "ru", - "jp", - "kr", - "in" -]New value: +[ + "us", + "gb", + "de", + "it", + "fr", + "ca", + "es", + "ru", + "jp", + "kr", + "in", + "hk", + "tr" +]
- Changed
webscraping_ai_question1 field changed- changed
Input schema / properties / country / enumPrevious value: -[ - "us", - "gb", - "de", - "it", - "fr", - "ca", - "es", - "ru", - "jp", - "kr", - "in" -]New value: +[ + "us", + "gb", + "de", + "it", + "fr", + "ca", + "es", + "ru", + "jp", + "kr", + "in", + "hk", + "tr" +]
- Changed
webscraping_ai_selected1 field changed- changed
Input schema / properties / country / enumPrevious value: -[ - "us", - "gb", - "de", - "it", - "fr", - "ca", - "es", - "ru", - "jp", - "kr", - "in" -]New value: +[ + "us", + "gb", + "de", + "it", + "fr", + "ca", + "es", + "ru", + "jp", + "kr", + "in", + "hk", + "tr" +]
- Changed
webscraping_ai_selected_multiple1 field changed- changed
Input schema / properties / country / enumPrevious value: -[ - "us", - "gb", - "de", - "it", - "fr", - "ca", - "es", - "ru", - "jp", - "kr", - "in" -]New value: +[ + "us", + "gb", + "de", + "it", + "fr", + "ca", + "es", + "ru", + "jp", + "kr", + "in", + "hk", + "tr" +]
- Changed
webscraping_ai_text1 field changed- changed
Input schema / properties / country / enumPrevious value: -[ - "us", - "gb", - "de", - "it", - "fr", - "ca", - "es", - "ru", - "jp", - "kr", - "in" -]New value: +[ + "us", + "gb", + "de", + "it", + "fr", + "ca", + "es", + "ru", + "jp", + "kr", + "in", + "hk", + "tr" +]
6 tool updates
v1.0.4- Changed
webscraping_ai_fields2 fields changed- changed
Input schema / properties / proxy / descriptionPrevious value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page." - changed
Input schema / properties / proxy / enumPrevious value: -[ - "datacenter", - "residential" -]New value: +[ + "datacenter", + "residential", + "stealth" +]
- Changed
webscraping_ai_html2 fields changed- changed
Input schema / properties / proxy / descriptionPrevious value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page." - changed
Input schema / properties / proxy / enumPrevious value: -[ - "datacenter", - "residential" -]New value: +[ + "datacenter", + "residential", + "stealth" +]
- Changed
webscraping_ai_question2 fields changed- changed
Input schema / properties / proxy / descriptionPrevious value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page." - changed
Input schema / properties / proxy / enumPrevious value: -[ - "datacenter", - "residential" -]New value: +[ + "datacenter", + "residential", + "stealth" +]
- Changed
webscraping_ai_selected2 fields changed- changed
Input schema / properties / proxy / descriptionPrevious value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page." - changed
Input schema / properties / proxy / enumPrevious value: -[ - "datacenter", - "residential" -]New value: +[ + "datacenter", + "residential", + "stealth" +]
- Changed
webscraping_ai_selected_multiple2 fields changed- changed
Input schema / properties / proxy / descriptionPrevious value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page." - changed
Input schema / properties / proxy / enumPrevious value: -[ - "datacenter", - "residential" -]New value: +[ + "datacenter", + "residential", + "stealth" +]
- Changed
webscraping_ai_text2 fields changed- changed
Input schema / properties / proxy / descriptionPrevious value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page." - changed
Input schema / properties / proxy / enumPrevious value: -[ - "datacenter", - "residential" -]New value: +[ + "datacenter", + "residential", + "stealth" +]
7 tool updates
v1.0.2- Added
webscraping_ai_account - Added
webscraping_ai_fields - Added
webscraping_ai_html - Added
webscraping_ai_question - Added
webscraping_ai_selected - Added
webscraping_ai_selected_multiple - Added
webscraping_ai_text
TDQS
Scored across 9 tools
Most tools have clearly distinct purposes: html/text/selected return page content, serp searches, data handles known provider pages, and account handles billing status. The AI tools (question, fields, text) and the selector tools (selected vs selected_multiple) overlap somewhat, but the descriptions explicitly cross-reference each other and clarify when to use which.
All tools share a predictable webscraping_ai_ prefix and use a consistent noun/adjective style (question, fields, html, text, selected, serp, data, account). There is no verb_noun pattern, and selected vs selected_multiple is a minor deviation, but the set is highly consistent overall.
Nine tools is well-scoped for a web scraping API client, covering retrieval, AI extraction, structured data, search, and account status without redundancy. Each tool maps to a distinct endpoint or capability.
The surface covers the core scraping lifecycle: raw HTML, rendered text, CSS selection, AI question/field extraction, SERP search, provider-specific structured data, and account status. Minor gaps exist, such as screenshot or other advanced scraping options, but agents can accomplish most scraping tasks with this set.
Maintenance
Related MCP Connectors
Scrape any web page, search Google/Amazon/Walmart/YouTube and extract data via the ScrapingBee API.
181Fetch and process content from specified URLs & sources using the Oxylabs Web Scraper API.
The most accurate web access API. Stop getting blocked.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables web scraping and document processing with JavaScript execution, anti-detection measures, batch processing, and structured data extraction. Supports multiple formats including markdown, HTML, screenshots, and handles PDFs with OCR capabilities.4MIT
- AlicenseAqualityBmaintenanceGives AI assistants real-time access to web data with the ability to scrape any website, bypass anti-bot systems, extract structured data, and capture screenshots through enterprise-grade infrastructure.6290 npm14ISC

ScrapeOps MCP Serverofficial
AlicenseAqualityCmaintenanceEnables AI agents to browse the web, bypass anti-bots, render JavaScript, take screenshots, and perform structured data extraction using the ScrapeOps Proxy API.313 npmMIT- AlicenseNot gradedqualityBmaintenanceEnables headless-Chromium web automation, including rendering fully-executed HTML, extracting structured data with CSS selectors, capturing screenshots, and AI-assisted smart scraping of web pages.371 npmMIT