Skip to main content
Glama
enesjakozh
amotivv

Web Content MCP Server

by amotivv
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Remote

Cloudflareブラウザレンダリング実験とMCPサーバー

このプロジェクトでは、Cloudflare Browser Renderingを用いてLLMコンテキスト用のWebコンテンツを抽出する方法を紹介します。REST APIとWorkers Binding APIを用いた実験に加え、LLMにWebコンテキストを提供するために使用できるMCPサーバー実装も含まれています。

プロジェクト構造

cloudflare-browser-rendering/
├── examples/                   # Example implementations and utilities
│   ├── basic-worker-example.js # Basic Worker with Browser Rendering
│   ├── minimal-worker-example.js # Minimal implementation
│   ├── debugging-tools/        # Tools for debugging
│   │   └── debug-test.js       # Debug test utility
│   └── testing/                # Testing utilities
│       └── content-test.js     # Content testing utility
├── experiments/                # Educational experiments
│   ├── basic-rest-api/         # REST API tests
│   ├── puppeteer-binding/      # Workers Binding API tests
│   └── content-extraction/     # Content processing tests
├── src/                        # MCP server source code
│   ├── index.ts                # Main entry point
│   ├── server.ts               # MCP server implementation
│   ├── browser-client.ts       # Browser Rendering client
│   └── content-processor.ts    # Content processing utilities
├── puppeteer-worker.js         # Cloudflare Worker with Browser Rendering binding
├── test-puppeteer.js           # Tests for the main implementation
├── wrangler.toml               # Wrangler configuration for the Worker
├── cline_mcp_settings.json.example # Example MCP settings for Cline
├── .gitignore                  # Git ignore file
└── LICENSE                     # MIT License

Related MCP server: MCP Web Tools Server

前提条件

  • Node.js (v16以降)

  • ブラウザレンダリングが有効になっているCloudflareアカウント

  • タイプスクリプト

  • Wrangler CLI (ワーカーのデプロイ用)

インストール

  1. リポジトリをクローンします。

git clone https://github.com/yourusername/cloudflare-browser-rendering.git
cd cloudflare-browser-rendering

  1. 依存関係をインストールします:

npm install

Cloudflareワーカーのセットアップ

  1. Cloudflare Puppeteer パッケージをインストールします。

npm install @cloudflare/puppeteer

  1. Wrangler を設定します。

# wrangler.toml
name = "browser-rendering-api"
main = "puppeteer-worker.js"
compatibility_date = "2023-10-30"
compatibility_flags = ["nodejs_compat"]

[browser]
binding = "browser"

  1. ワーカーをデプロイする:

npx wrangler deploy

  1. ワーカーをテストする:

node test-puppeteer.js

実験の実行

基本的なREST API実験

この実験では、Cloudflare ブラウザ レンダリング REST API を使用して Web コンテンツを取得および処理する方法を示します。

npm run experiment:rest

Puppeteer バインディング API 実験

この実験では、より高度なブラウザ自動化を実現するために、Puppeteer で Cloudflare Browser Rendering Workers Binding API を使用する方法を示します。

npm run experiment:puppeteer

コンテンツ抽出実験

この実験では、LLM のコンテキストとして使用するために Web コンテンツを抽出して処理する方法を示します。

npm run experiment:content

MCPサーバー

MCP サーバーは、LLM のコンテキストとして使用するために、Cloudflare ブラウザ レンダリングを使用して Web コンテンツを取得および処理するためのツールを提供します。

MCPサーバーの構築

npm run build

MCPサーバーの実行

npm start

あるいは、開発の場合:

npm run dev

MCP サーバーツール

MCP サーバーは次のツールを提供します。

  1. fetch_page - LLMコンテキストのWebページを取得して処理します

  2. search_documentation - Cloudflareのドキュメントを検索し、関連するコンテンツを返します

  3. extract_structured_content - CSSセレクターを使用してWebページから構造化コンテンツを抽出します

  4. summarize_content - LLM コンテキストをより簡潔にするために Web コンテンツを要約します

構成

Cloudflare ブラウザ レンダリング エンドポイントを使用するには、 BROWSER_RENDERING_API環境変数を設定します。

export BROWSER_RENDERING_API=https://YOUR_WORKER_URL_HERE

YOUR_WORKER_URL_HERE 、デプロイしたCloudflare WorkerのURLに置き換えてください。このプレースホルダは、以下のファイルで置き換える必要があります。

  1. テストファイル内: test-puppeteer.js 、 examples/debugging-tools/debug-test.js 、 examples/testing/content-test.js

  2. MCP サーバー構成: cline_mcp_settings.json.example

  3. ブラウザクライアントの場合: src/browser-client.ts (環境変数が設定されていない場合のフォールバックとして)

Clineとの統合

MCP サーバーを Cline と統合するには、 cline_mcp_settings.json.exampleファイルを適切な場所にコピーします。

cp cline_mcp_settings.json.example ~/Library/Application\ Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json

または、既存のcline_mcp_settings.jsonファイルに構成を追加します。

主な学び

  1. Cloudflare ブラウザ レンダリングでは、ブラウザ バインディングと対話するために@cloudflare/puppeteerパッケージが必要です。

  2. ブラウザ バインディングを使用する正しいパターンは次のとおりです。

    import puppeteer from '@cloudflare/puppeteer';

// Then in your handler:
const browser = await puppeteer.launch(env.browser);
const page = await browser.newPage();

  3. ブラウザ レンダリング バインディングを使用するワーカーをデプロイする場合は、 nodejs_compat互換性フラグを有効にする必要があります。

  4. リソースの漏洩を防ぐため、使用後は必ずブラウザを閉じてください。

ライセンス

マサチューセッツ工科大学

Available Tools

4 tools
extract_structured_contentC

Extracts structured content from a web page using CSS selectors

ParametersJSON Schema
NameRequiredDescriptionDefault
selectorsYesCSS selectors to extract content
urlYesURL to extract content from

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states what the tool does but lacks critical behavioral details: it doesn't specify if it fetches the page internally, handles errors, requires internet access, has rate limits, or what the output format is. For a tool with no annotations, this leaves significant gaps in understanding its operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise and front-loaded: a single sentence that directly states the tool's function without any fluff. Every word earns its place by conveying essential information about extraction, content type, source, and method. It's efficiently structured for quick understanding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of web extraction (involving network calls, parsing, and structured data output), the description is incomplete. There's no output schema, and the description doesn't explain return values, error handling, or behavioral traits. With no annotations and only basic parameter coverage, it fails to provide enough context for effective use in real-world scenarios.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, meaning the input schema already documents both parameters ('url' and 'selectors') with descriptions. The tool description adds no additional meaning beyond what's in the schema—it doesn't explain parameter interactions, provide examples, or clarify semantics. With high schema coverage, the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Extracts structured content from a web page using CSS selectors'. It specifies the verb ('extracts'), resource ('structured content'), and method ('CSS selectors'), which is specific and actionable. However, it doesn't explicitly differentiate from sibling tools like 'fetch_page' or 'summarize_content', which would require a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention scenarios where extraction is preferred over fetching the whole page, searching documentation, summarizing, or taking a screenshot. Without any context or exclusions, users must infer usage from the purpose alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_pageC

Fetches and processes a web page for LLM context

ParametersJSON Schema
NameRequiredDescriptionDefault
includeScreenshotNoWhether to include a screenshot (base64 encoded)
maxContentLengthNoMaximum content length to return
urlYesURL to fetch

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'fetches and processes' but doesn't clarify aspects like rate limits, authentication needs, error handling, or what 'processes' entails (e.g., cleaning HTML, extracting text). This leaves significant gaps for a tool that interacts with external resources.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose without any wasted words. It's appropriately sized for the tool's complexity, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (fetching and processing web pages), lack of annotations, and no output schema, the description is incomplete. It doesn't explain return values, error cases, or behavioral traits, leaving the agent with insufficient information for reliable use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, so the baseline is 3. The description adds no additional meaning beyond the schema, which already details parameters like 'url', 'includeScreenshot', and 'maxContentLength'. No extra syntax, format, or usage context is provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with specific verbs ('fetches and processes') and resource ('a web page'), and specifies the outcome ('for LLM context'). It doesn't explicitly differentiate from sibling tools like 'extract_structured_content' or 'summarize_content', which prevents a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'extract_structured_content' or 'search_documentation'. It lacks context about prerequisites, exclusions, or comparative use cases, offering only a basic functional statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_documentationC

Searches Cloudflare documentation and returns relevant content

ParametersJSON Schema
NameRequiredDescriptionDefault
maxResultsNoMaximum number of results to return
queryYesSearch query

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool searches and returns content, but doesn't describe important behaviors like whether it performs web searches, accesses a local database, requires authentication, has rate limits, or what format the returned content takes (e.g., text snippets, links, full documents).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that gets straight to the point without unnecessary words. It's appropriately sized for a simple search tool, though it could potentially be more structured with additional context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a search tool with no annotations and no output schema, the description is incomplete. It doesn't explain what kind of content is returned (snippets, full pages, metadata), how results are ranked, whether authentication is needed, or any limitations. Given the lack of structured fields, the description should provide more operational context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, clearly documenting both parameters ('query' and 'maxResults'). The description doesn't add any meaningful parameter semantics beyond what the schema already provides, such as search syntax examples or result format details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('Searches') and resource ('Cloudflare documentation'), making it immediately understandable. However, it doesn't distinguish this tool from its sibling tools like 'fetch_page' or 'extract_structured_content', which might also retrieve documentation content in different ways.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, constraints, or compare it to sibling tools like 'fetch_page' (which might retrieve a specific page) or 'summarize_content' (which might process content).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

summarize_contentC

Summarizes web content for more concise LLM context

ParametersJSON Schema
NameRequiredDescriptionDefault
maxLengthNoMaximum length of the summary
urlYesURL to summarize

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'summarizes web content' but doesn't describe how it works (e.g., extraction method, processing time, error handling), what limitations exist (e.g., supported content types, rate limits), or what the output looks like. This leaves significant gaps in understanding the tool's behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and includes a clear goal, making it appropriately sized and well-structured for quick understanding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of summarizing web content, no annotations, and no output schema, the description is incomplete. It doesn't explain the return format, potential errors, or behavioral traits like content processing methods. For a tool with 2 parameters and significant operational implications, more context is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, clearly documenting both parameters ('url' and 'maxLength'). The description adds no additional parameter semantics beyond what the schema provides, such as format details for 'url' or typical values for 'maxLength'. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('summarizes') and resource ('web content'), and it provides the goal ('for more concise LLM context'). However, it doesn't explicitly differentiate from sibling tools like 'extract_structured_content' or 'fetch_page', which might have overlapping functionality.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'extract_structured_content' or 'fetch_page'. It doesn't mention prerequisites, exclusions, or specific contexts where this summarization tool is preferred over other content-handling siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv1.0.0
    • First observedextract_structured_content
    • First observedfetch_page
    • First observedsearch_documentation
    • First observedsummarize_content
Deploy Server
Give agents eyes on the webHosted screenshot MCP. Send a URL, get a pixel-perfect image. $0.002 per grab. Works with Claude, Cursor, and Codex.grabbit.liveAd

TDQS

B3.1/5.0

Scored across 4 tools

Disambiguation4/5

The tools are mostly distinct in purpose, with extract_structured_content, fetch_page, and summarize_content each handling different aspects of web content processing. However, fetch_page and summarize_content could be slightly confused as both relate to processing web content for LLM context, though their specific focuses differ.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern with snake_case, such as extract_structured_content and fetch_page. This uniformity makes the set predictable and easy to understand, with no deviations in naming conventions.

Tool Count3/5

With 4 tools, the count is on the lower side for a web content server, which might feel thin for covering a broad domain like web content processing. However, it is reasonable for a focused set, though it could benefit from additional tools for more comprehensive coverage.

Completeness2/5

There are significant gaps in the tool surface for web content processing. For example, there are no tools for updating or deleting content, handling dynamic content like JavaScript, or managing multiple pages. The inclusion of search_documentation is also inconsistent with the general web content focus, creating a notable gap in core operations.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

Appeared in Searches