en es ja ko zh

WebScraping-AI MCP Server

Name: WebScraping-AI MCP Server
Author: webscraping-ai

Official

by webscraping-ai

Overview Schema Source Related Servers Discussions Admin

JavaScript

Remote

WebScraping.AI MCPサーバー

Web データ抽出機能のためにWebScraping.AIと統合するモデルコンテキストプロトコル (MCP) サーバー実装。

特徴

ウェブページの内容に関する質問への回答
ウェブページからの構造化データ抽出
JavaScript レンダリングによる HTML コンテンツの取得
ウェブページからのプレーンテキスト抽出
CSSセレクタベースのコンテンツ抽出
国を選択できる複数のプロキシタイプ（データセンター、住宅）
ヘッドレス Chrome/Chromium を使用した JavaScript レンダリング
レート制限による同時リクエスト管理
対象ページでのカスタム JavaScript 実行
デバイスエミュレーション（デスクトップ、モバイル、タブレット）
アカウント使用状況の監視

Related MCP server: Scrapfly MCP

インストール

npxで実行

env WEBSCRAPING_AI_API_KEY=your_api_key npx -y webscraping-ai-mcp

手動インストール

# Clone the repository
git clone https://github.com/webscraping-ai/webscraping-ai-mcp-server.git
cd webscraping-ai-mcp-server

# Install dependencies
npm install

# Run
npm start

カーソルでの設定

注: カーソルバージョン 0.45.6 以上が必要です

WebScraping.AI MCP サーバーは、Cursor で 2 つの方法で構成できます。

プロジェクト固有の構成(チームプロジェクトに推奨): プロジェクトディレクトリに.cursor/mcp.jsonファイルを作成します。

{
  "servers": {
    "webscraping-ai": {
      "type": "command",
      "command": "npx -y webscraping-ai-mcp",
      "env": {
        "WEBSCRAPING_AI_API_KEY": "your-api-key",
        "WEBSCRAPING_AI_CONCURRENCY_LIMIT": "5"
      }
    }
  }
}

グローバル構成(すべてのプロジェクトでの個人使用): 上記と同じ構成形式で、ホームディレクトリに~/.cursor/mcp.jsonファイルを作成します。

Windows を使用していて問題が発生している場合は、コマンドとしてcmd /c "set WEBSCRAPING_AI_API_KEY=your-api-key && npx -y webscraping-ai-mcp"を使用してみてください。

この構成により、Web スクレイピングタスクに関連する場合に、WebScraping.AI ツールが Cursor の AI エージェントで自動的に使用できるようになります。

Claude Desktopで実行中

これをclaude_desktop_config.jsonに追加します:

{
  "mcpServers": {
    "mcp-server-webscraping-ai": {
      "command": "npx",
      "args": ["-y", "webscraping-ai-mcp"],
      "env": {
        "WEBSCRAPING_AI_API_KEY": "YOUR_API_KEY_HERE",
        "WEBSCRAPING_AI_CONCURRENCY_LIMIT": "5"
      }
    }
  }
}

構成

環境変数

必須

WEBSCRAPING_AI_API_KEY : WebScraping.AI APIキー
- すべての操作に必要
- WebScraping.AIからAPIキーを取得する

オプション構成

WEBSCRAPING_AI_CONCURRENCY_LIMIT : 同時リクエストの最大数（デフォルト: 5 ）
WEBSCRAPING_AI_DEFAULT_PROXY_TYPE : 使用するプロキシの種類（デフォルト: residential ）
WEBSCRAPING_AI_DEFAULT_JS_RENDERING : JavaScript レンダリングを有効/無効にする (デフォルト: true )
WEBSCRAPING_AI_DEFAULT_TIMEOUT : ウェブページの最大取得時間（ミリ秒）（デフォルト: 15000 、最大: 30000 ）
WEBSCRAPING_AI_DEFAULT_JS_TIMEOUT : JavaScript レンダリングの最大時間（ミリ秒）（デフォルト: 2000 ）

設定例

標準的な使用法:

# Required
export WEBSCRAPING_AI_API_KEY=your-api-key

# Optional - customize behavior (default values)
export WEBSCRAPING_AI_CONCURRENCY_LIMIT=5
export WEBSCRAPING_AI_DEFAULT_PROXY_TYPE=residential # datacenter or residential
export WEBSCRAPING_AI_DEFAULT_JS_RENDERING=true
export WEBSCRAPING_AI_DEFAULT_TIMEOUT=15000
export WEBSCRAPING_AI_DEFAULT_JS_TIMEOUT=2000

利用可能なツール

1. 質問ツール ( `webscraping_ai_question` )

Web ページのコンテンツについて質問します。

{
  "name": "webscraping_ai_question",
  "arguments": {
    "url": "https://example.com",
    "question": "What is the main topic of this page?",
    "timeout": 30000,
    "js": true,
    "js_timeout": 2000,
    "wait_for": ".content-loaded",
    "proxy": "datacenter",
    "country": "us"
  }
}

応答例:

{
  "content": [
    {
      "type": "text",
      "text": "The main topic of this page is examples and documentation for HTML and web standards."
    }
  ],
  "isError": false
}

2. フィールドツール（ `webscraping_ai_fields` ）

指示に基づいて Web ページから構造化データを抽出します。

{
  "name": "webscraping_ai_fields",
  "arguments": {
    "url": "https://example.com/product",
    "fields": {
      "title": "Extract the product title",
      "price": "Extract the product price",
      "description": "Extract the product description"
    },
    "js": true,
    "timeout": 30000
  }
}

応答例:

{
  "content": [
    {
      "type": "text",
      "text": {
        "title": "Example Product",
        "price": "$99.99",
        "description": "This is an example product description."
      }
    }
  ],
  "isError": false
}

3. HTMLツール（ `webscraping_ai_html` ）

JavaScript レンダリングを使用して Web ページの完全な HTML を取得します。

{
  "name": "webscraping_ai_html",
  "arguments": {
    "url": "https://example.com",
    "js": true,
    "timeout": 30000,
    "wait_for": "#content-loaded"
  }
}

応答例:

{
  "content": [
    {
      "type": "text",
      "text": "<html>...[full HTML content]...</html>"
    }
  ],
  "isError": false
}

4. テキストツール ( `webscraping_ai_text` )

Web ページから表示されているテキストコンテンツを抽出します。

{
  "name": "webscraping_ai_text",
  "arguments": {
    "url": "https://example.com",
    "js": true,
    "timeout": 30000
  }
}

応答例:

{
  "content": [
    {
      "type": "text",
      "text": "Example Domain\nThis domain is for use in illustrative examples in documents..."
    }
  ],
  "isError": false
}

5. 選択されたツール ( `webscraping_ai_selected` )

CSS セレクターを使用して特定の要素からコンテンツを抽出します。

{
  "name": "webscraping_ai_selected",
  "arguments": {
    "url": "https://example.com",
    "selector": "div.main-content",
    "js": true,
    "timeout": 30000
  }
}

応答例:

{
  "content": [
    {
      "type": "text",
      "text": "<div class=\"main-content\">This is the main content of the page.</div>"
    }
  ],
  "isError": false
}

6. 複数選択ツール ( `webscraping_ai_selected_multiple` )

CSS セレクターを使用して複数の要素からコンテンツを抽出します。

{
  "name": "webscraping_ai_selected_multiple",
  "arguments": {
    "url": "https://example.com",
    "selectors": ["div.header", "div.product-list", "div.footer"],
    "js": true,
    "timeout": 30000
  }
}

応答例:

{
  "content": [
    {
      "type": "text",
      "text": [
        "<div class=\"header\">Header content</div>",
        "<div class=\"product-list\">Product list content</div>",
        "<div class=\"footer\">Footer content</div>"
      ]
    }
  ],
  "isError": false
}

7. アカウントツール（ `webscraping_ai_account` ）

WebScraping.AI アカウントに関する情報を取得します。

{
  "name": "webscraping_ai_account",
  "arguments": {}
}

応答例:

{
  "content": [
    {
      "type": "text",
      "text": {
        "requests": 5000,
        "remaining": 4500,
        "limit": 10000,
        "resets_at": "2023-12-31T23:59:59Z"
      }
    }
  ],
  "isError": false
}

すべてのツールに共通するオプション

次のオプションは、すべてのスクレイピングツールで使用できます。

timeout : ウェブページの最大取得時間（ミリ秒）（デフォルトは15000、最大値は30000）
js : ヘッドレスブラウザを使用してページ上の JavaScript を実行します（デフォルトでは true）
js_timeout : JavaScript レンダリングの最大時間（ミリ秒）（デフォルトは 2000）
wait_for : ページコンテンツを返す前に待機する CSS セレクタ
proxy : プロキシの種類、データセンターまたは住宅（デフォルトは住宅）
country : 使用するプロキシの国（デフォルトは米国）。サポートされている国: us、gb、de、it、fr、ca、es、ru、jp、kr、in
custom_proxy : 「 http://user:password@host:port 」形式の独自のプロキシ URL
device : デバイスエミュレーションの種類。サポートされる値: デスクトップ、モバイル、タブレット
error_on_404 : 対象ページの 404 HTTP ステータスでエラーを返します (デフォルトでは false)
error_on_redirect : 対象ページでのリダイレクト時にエラーを返す（デフォルトは false）
js_script : 対象ページで実行するカスタム JavaScript コード

エラー処理

サーバーは堅牢なエラー処理を提供します。

一時的なエラーの自動再試行
バックオフによるレート制限の処理
詳細なエラーメッセージ
ネットワークの回復力

エラー応答の例:

{
  "content": [
    {
      "type": "text",
      "text": "API Error: 429 Too Many Requests"
    }
  ],
  "isError": true
}

LLMとの統合

このサーバーはモデルコンテキストプロトコル（MCP）を実装しており、MCP対応のLLMプラットフォームと互換性があります。これらのツールをWebスクレイピングタスクに使用するようにLLMを設定できます。

例: MCP を使用した Claude の設定

const { Claude } = require('@anthropic-ai/sdk');
const { Client } = require('@modelcontextprotocol/sdk/client/index.js');
const { StdioClientTransport } = require('@modelcontextprotocol/sdk/client/stdio.js');

const claude = new Claude({
  apiKey: process.env.ANTHROPIC_API_KEY
});

const transport = new StdioClientTransport({
  command: 'npx',
  args: ['-y', 'webscraping-ai-mcp'],
  env: {
    WEBSCRAPING_AI_API_KEY: 'your-api-key'
  }
});

const client = new Client({
  name: 'claude-client',
  version: '1.0.0'
});

await client.connect(transport);

// Now you can use Claude with WebScraping.AI tools
const tools = await client.listTools();
const response = await claude.complete({
  prompt: 'What is the main topic of example.com?',
  tools: tools
});

発達

# Clone the repository
git clone https://github.com/webscraping-ai/webscraping-ai-mcp-server.git
cd webscraping-ai-mcp-server

# Install dependencies
npm install

# Run tests
npm test

# Add your .env file
cp .env.example .env

# Start the inspector
npx @modelcontextprotocol/inspector node src/index.js

貢献

リポジトリをフォークする
機能ブランチを作成する
テストを実行: npm test
プルリクエストを送信する

ライセンス

MITライセンス - 詳細はLICENSEファイルを参照

Available Tools

7 tools

webscraping_ai_accountD

ParametersJSON Schema

Name	Required	Description	Default
No parameters

TDQS

D1/5.0

Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Tool has no description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness1/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Tool has no description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Tool has no description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has no description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose1/5

Does the description clearly state what the tool does and how it differs from similar tools?

Tool has no description.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Tool has no description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

webscraping_ai_fieldsD

ParametersJSON Schema

Name	Required	Description	Default
`js`	No	Execute on-page JavaScript using a headless browser (false by default).
`url`	Yes	URL of the target page.
`proxy`	No	Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page.	datacenter
`device`	No	Type of device emulation.
`fields`	Yes	Dictionary of field names with instructions for extraction.
`country`	No	Country of the proxy to use (US by default).
`timeout`	No	Maximum web page retrieval time in ms (20000 by default, maximum is 30000).
`wait_for`	No	CSS selector to wait for before returning the page content.
`js_script`	No	Custom JavaScript code to execute on the target page.
`js_timeout`	No	Maximum JavaScript rendering time in ms (3000 by default).
`custom_proxy`	No	Your own proxy URL in "http://user:password@host:port" format.
`error_on_404`	No	Return error on 404 HTTP status on the target page (false by default).
`error_on_redirect`	No	Return error on redirect on the target page (false by default).

TDQS

D1/5.0

Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Tool has no description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness1/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Tool has no description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Tool has no description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has no description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose1/5

Does the description clearly state what the tool does and how it differs from similar tools?

Tool has no description.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Tool has no description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

webscraping_ai_htmlD

ParametersJSON Schema

Name	Required	Description	Default
`js`	No	Execute on-page JavaScript using a headless browser (false by default).
`url`	Yes	URL of the target page.
`proxy`	No	Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page.	datacenter
`device`	No	Type of device emulation.
`format`	No	Response format (json or text).
`country`	No	Country of the proxy to use (US by default).
`timeout`	No	Maximum web page retrieval time in ms (20000 by default, maximum is 30000).
`wait_for`	No	CSS selector to wait for before returning the page content.
`js_script`	No	Custom JavaScript code to execute on the target page.
`js_timeout`	No	Maximum JavaScript rendering time in ms (3000 by default).
`custom_proxy`	No	Your own proxy URL in "http://user:password@host:port" format.
`error_on_404`	No	Return error on 404 HTTP status on the target page (false by default).
`error_on_redirect`	No	Return error on redirect on the target page (false by default).
`return_script_result`	No	Return result of the custom JavaScript code execution.

TDQS

D1/5.0

Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Tool has no description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness1/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Tool has no description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Tool has no description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has no description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose1/5

Does the description clearly state what the tool does and how it differs from similar tools?

Tool has no description.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Tool has no description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

webscraping_ai_questionD

ParametersJSON Schema

Name	Required	Description	Default
`js`	No	Execute on-page JavaScript using a headless browser (false by default).
`url`	Yes	URL of the target page.
`proxy`	No	Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page.	datacenter
`device`	No	Type of device emulation.
`country`	No	Country of the proxy to use (US by default).
`timeout`	No	Maximum web page retrieval time in ms (20000 by default, maximum is 30000).
`question`	Yes	Question or instructions to ask the LLM model about the target page.
`wait_for`	No	CSS selector to wait for before returning the page content.
`js_script`	No	Custom JavaScript code to execute on the target page.
`js_timeout`	No	Maximum JavaScript rendering time in ms (3000 by default).
`custom_proxy`	No	Your own proxy URL in "http://user:password@host:port" format.
`error_on_404`	No	Return error on 404 HTTP status on the target page (false by default).
`error_on_redirect`	No	Return error on redirect on the target page (false by default).

TDQS

D1/5.0

Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Tool has no description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness1/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Tool has no description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Tool has no description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has no description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose1/5

Does the description clearly state what the tool does and how it differs from similar tools?

Tool has no description.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Tool has no description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

webscraping_ai_selectedD

ParametersJSON Schema

Name	Required	Description	Default
`js`	No	Execute on-page JavaScript using a headless browser (false by default).
`url`	Yes	URL of the target page.
`proxy`	No	Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page.	datacenter
`device`	No	Type of device emulation.
`format`	No	Response format (json or text).	json
`country`	No	Country of the proxy to use (US by default).
`timeout`	No	Maximum web page retrieval time in ms (20000 by default, maximum is 30000).
`selector`	Yes	CSS selector to extract content for.
`wait_for`	No	CSS selector to wait for before returning the page content.
`js_script`	No	Custom JavaScript code to execute on the target page.
`js_timeout`	No	Maximum JavaScript rendering time in ms (3000 by default).
`custom_proxy`	No	Your own proxy URL in "http://user:password@host:port" format.
`error_on_404`	No	Return error on 404 HTTP status on the target page (false by default).
`error_on_redirect`	No	Return error on redirect on the target page (false by default).

TDQS

D1/5.0

Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Tool has no description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness1/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Tool has no description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Tool has no description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has no description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose1/5

Does the description clearly state what the tool does and how it differs from similar tools?

Tool has no description.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Tool has no description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

webscraping_ai_selected_multipleD

ParametersJSON Schema

Name	Required	Description	Default
`js`	No	Execute on-page JavaScript using a headless browser (false by default).
`url`	Yes	URL of the target page.
`proxy`	No	Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page.	datacenter
`device`	No	Type of device emulation.
`country`	No	Country of the proxy to use (US by default).
`timeout`	No	Maximum web page retrieval time in ms (20000 by default, maximum is 30000).
`wait_for`	No	CSS selector to wait for before returning the page content.
`js_script`	No	Custom JavaScript code to execute on the target page.
`selectors`	Yes	Array of CSS selectors to extract content for.
`js_timeout`	No	Maximum JavaScript rendering time in ms (3000 by default).
`custom_proxy`	No	Your own proxy URL in "http://user:password@host:port" format.
`error_on_404`	No	Return error on 404 HTTP status on the target page (false by default).
`error_on_redirect`	No	Return error on redirect on the target page (false by default).

TDQS

D1/5.0

Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Tool has no description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness1/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Tool has no description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Tool has no description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has no description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose1/5

Does the description clearly state what the tool does and how it differs from similar tools?

Tool has no description.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Tool has no description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

webscraping_ai_textD

ParametersJSON Schema

Name	Required	Description	Default
`js`	No	Execute on-page JavaScript using a headless browser (false by default).
`url`	Yes	URL of the target page.
`proxy`	No	Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page.	datacenter
`device`	No	Type of device emulation.
`country`	No	Country of the proxy to use (US by default).
`timeout`	No	Maximum web page retrieval time in ms (20000 by default, maximum is 30000).
`wait_for`	No	CSS selector to wait for before returning the page content.
`js_script`	No	Custom JavaScript code to execute on the target page.
`js_timeout`	No	Maximum JavaScript rendering time in ms (3000 by default).
`text_format`	No	Format of the text response.	json
`custom_proxy`	No	Your own proxy URL in "http://user:password@host:port" format.
`error_on_404`	No	Return error on 404 HTTP status on the target page (false by default).
`return_links`	No	Return links from the page body text.
`error_on_redirect`	No	Return error on redirect on the target page (false by default).

TDQS

D1/5.0

Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Tool has no description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness1/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Tool has no description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Tool has no description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has no description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose1/5

Does the description clearly state what the tool does and how it differs from similar tools?

Tool has no description.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Tool has no description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

6 tool updatesv1.0.5Jun 24, 2026

Changedwebscraping_ai_fields1 field changed

changedInput schema / properties / country / enum

Previous value: -[
-  "us",
-  "gb",
-  "de",
-  "it",
-  "fr",
-  "ca",
-  "es",
-  "ru",
-  "jp",
-  "kr",
-  "in"
-]New value: +[
+  "us",
+  "gb",
+  "de",
+  "it",
+  "fr",
+  "ca",
+  "es",
+  "ru",
+  "jp",
+  "kr",
+  "in",
+  "hk",
+  "tr"
+]

Changedwebscraping_ai_html1 field changed

changedInput schema / properties / country / enum

Previous value: -[
-  "us",
-  "gb",
-  "de",
-  "it",
-  "fr",
-  "ca",
-  "es",
-  "ru",
-  "jp",
-  "kr",
-  "in"
-]New value: +[
+  "us",
+  "gb",
+  "de",
+  "it",
+  "fr",
+  "ca",
+  "es",
+  "ru",
+  "jp",
+  "kr",
+  "in",
+  "hk",
+  "tr"
+]

Changedwebscraping_ai_question1 field changed

changedInput schema / properties / country / enum

Previous value: -[
-  "us",
-  "gb",
-  "de",
-  "it",
-  "fr",
-  "ca",
-  "es",
-  "ru",
-  "jp",
-  "kr",
-  "in"
-]New value: +[
+  "us",
+  "gb",
+  "de",
+  "it",
+  "fr",
+  "ca",
+  "es",
+  "ru",
+  "jp",
+  "kr",
+  "in",
+  "hk",
+  "tr"
+]

Changedwebscraping_ai_selected1 field changed

changedInput schema / properties / country / enum

Previous value: -[
-  "us",
-  "gb",
-  "de",
-  "it",
-  "fr",
-  "ca",
-  "es",
-  "ru",
-  "jp",
-  "kr",
-  "in"
-]New value: +[
+  "us",
+  "gb",
+  "de",
+  "it",
+  "fr",
+  "ca",
+  "es",
+  "ru",
+  "jp",
+  "kr",
+  "in",
+  "hk",
+  "tr"
+]

Changedwebscraping_ai_selected_multiple1 field changed

changedInput schema / properties / country / enum

Previous value: -[
-  "us",
-  "gb",
-  "de",
-  "it",
-  "fr",
-  "ca",
-  "es",
-  "ru",
-  "jp",
-  "kr",
-  "in"
-]New value: +[
+  "us",
+  "gb",
+  "de",
+  "it",
+  "fr",
+  "ca",
+  "es",
+  "ru",
+  "jp",
+  "kr",
+  "in",
+  "hk",
+  "tr"
+]

Changedwebscraping_ai_text1 field changed

changedInput schema / properties / country / enum

Previous value: -[
-  "us",
-  "gb",
-  "de",
-  "it",
-  "fr",
-  "ca",
-  "es",
-  "ru",
-  "jp",
-  "kr",
-  "in"
-]New value: +[
+  "us",
+  "gb",
+  "de",
+  "it",
+  "fr",
+  "ca",
+  "es",
+  "ru",
+  "jp",
+  "kr",
+  "in",
+  "hk",
+  "tr"
+]

6 tool updatesv1.0.4May 17, 2026

Changedwebscraping_ai_fields2 fields changed

changedInput schema / properties / proxy / description

Previous value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page."

changedInput schema / properties / proxy / enum

Previous value: -[
-  "datacenter",
-  "residential"
-]New value: +[
+  "datacenter",
+  "residential",
+  "stealth"
+]

Changedwebscraping_ai_html2 fields changed

changedInput schema / properties / proxy / description

Previous value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page."

changedInput schema / properties / proxy / enum

Previous value: -[
-  "datacenter",
-  "residential"
-]New value: +[
+  "datacenter",
+  "residential",
+  "stealth"
+]

Changedwebscraping_ai_question2 fields changed

changedInput schema / properties / proxy / description

Previous value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page."

changedInput schema / properties / proxy / enum

Previous value: -[
-  "datacenter",
-  "residential"
-]New value: +[
+  "datacenter",
+  "residential",
+  "stealth"
+]

Changedwebscraping_ai_selected2 fields changed

changedInput schema / properties / proxy / description

Previous value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page."

changedInput schema / properties / proxy / enum

Previous value: -[
-  "datacenter",
-  "residential"
-]New value: +[
+  "datacenter",
+  "residential",
+  "stealth"
+]

Changedwebscraping_ai_selected_multiple2 fields changed

changedInput schema / properties / proxy / description

Previous value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page."

changedInput schema / properties / proxy / enum

Previous value: -[
-  "datacenter",
-  "residential"
-]New value: +[
+  "datacenter",
+  "residential",
+  "stealth"
+]

Changedwebscraping_ai_text2 fields changed

changedInput schema / properties / proxy / description

Previous value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page."

changedInput schema / properties / proxy / enum

Previous value: -[
-  "datacenter",
-  "residential"
-]New value: +[
+  "datacenter",
+  "residential",
+  "stealth"
+]

7 tool updatesv1.0.2May 3, 2026
- Addedwebscraping_ai_account
- Addedwebscraping_ai_fields
- Addedwebscraping_ai_html
- Addedwebscraping_ai_question
- Addedwebscraping_ai_selected
- Addedwebscraping_ai_selected_multiple
- Addedwebscraping_ai_text

Deploy Server

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

D1.8/5.0

Scored 2026-09-10 02:05 across 7 tools

Disambiguation3/5

The names suggest distinct output modes, but several tools overlap conceptually (html vs. text vs. fields vs. question), and selected vs. selected_multiple could easily confuse an agent without descriptions. The lack of descriptions worsens this ambiguity.

Naming Consistency4/5

All tools share the webscraping_ai_ prefix and snake_case convention, making the set look cohesive. The suffixes are mostly nouns/adjectives rather than verb_noun actions, but the pattern is predictable and consistent.

Tool Count4/5

Seven tools is a reasonable size for a scraping-focused serverched. The count covers several output modes without feeling bloated, though selected and selected_multiple could arguably be one parameterized tool.

Completeness4/5

The domain is inferable from the names: HTML/text/field extraction, selected elements, and question-answering over scraped content. It lacks some common extras like link extraction or screenshot, but the core scraping workflow appears covered.

Maintenance

ActivityStale

ResponsivenessNo issues

Resources

Need Help?

Reddit Discussion

Related Servers

Related MCP Connectors

scrapeunblocker-mcp-remote A
com.scrapeunblocker
Fetch any web page's HTML, AI-parsed JSON, or Google results via the ScrapeUnblocker anti-bot API
Web Scraping Browser Automation
42
oxylabs-mcp B
io.oxylabs
Fetch and process content from specified URLs & sources using the Oxylabs Web Scraper API.
Web Scraping Browser Automation
101106
web-access AOAuth
ai.usestring
The most accurate web access API. Stop getting blocked.
Browser Automation Web Scraping
22
WyndPath Web DataOAuth
com.wyndpath
Fetch and extract data from any public web page, even JS-rendered or anti-bot protected
Browser Automation Web Scraping
1

Related MCP Servers

FreeCrawl MCP Server
Web Scraping Browser Automation Documentation Access
dylan-gluck
A
license
Not graded
quality
D
maintenance
Enables web scraping and document processing with JavaScript execution, anti-detection measures, batch processing, and structured data extraction. Supports multiple formats including markdown, HTML, screenshots, and handles PDFs with OCR capabilities.
Updated a year ago (2025-08-20 03:34 UTC)
4
MIT
Scrapfly MCP
Web Scraping
scrapfly
A
license
A
quality
B
maintenance
Gives AI assistants real-time access to web data with the ability to scrape any website, bypass anti-bot systems, extract structured data, and capture screenshots through enterprise-grade infrastructure.
Updated 14 days ago (2026-09-12 03:43 UTC)
2
6
96 npm
14
ISC
ScrapeOps MCP Server
ScrapeOps
A
license
A
quality
D
maintenance
Enables AI agents to browse the web, bypass anti-bots, render JavaScript, take screenshots, and perform structured data extraction using the ScrapeOps Proxy API.
Updated 7 months ago (2026-02-24 10:53 UTC)
3
3 npm
MIT
browserless
Browser Automation Web Scraping
pipeworx-io
A
license
Not graded
quality
B
maintenance
Enables headless-Chromium web automation, including rendering fully-executed HTML, extracting structured data with CSS selectors, capturing screenshots, and AI-assisted smart scraping of web pages.
Updated a day ago (2026-09-24 22:06 UTC)
MIT

WebScraping.AI MCPサーバー

特徴

インストール

npxで実行

手動インストール

カーソルでの設定

Claude Desktopで実行中

構成

環境変数

必須

オプション構成

設定例

利用可能なツール

1. 質問ツール ( webscraping_ai_question )

2. フィールドツール（ webscraping_ai_fields ）

3. HTMLツール（ webscraping_ai_html ）

4. テキストツール ( webscraping_ai_text )

5. 選択されたツール ( webscraping_ai_selected )

6. 複数選択ツール ( webscraping_ai_selected_multiple )

7. アカウントツール（ webscraping_ai_account ）

すべてのツールに共通するオプション

エラー処理

LLMとの統合

例: MCP を使用した Claude の設定

発達

貢献

ライセンス

Available Tools

Tool Schema Changelog

Maintenance

Resources

Related MCP Connectors

Related MCP Servers

Appeared in Searches

1. 質問ツール ( `webscraping_ai_question` )

2. フィールドツール（ `webscraping_ai_fields` ）

3. HTMLツール（ `webscraping_ai_html` ）

4. テキストツール ( `webscraping_ai_text` )

5. 選択されたツール ( `webscraping_ai_selected` )

6. 複数選択ツール ( `webscraping_ai_selected_multiple` )

7. アカウントツール（ `webscraping_ai_account` ）