Skip to main content
Glama
enesjakozh
amotivv

Web Content MCP Server

by amotivv
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Remote

Cloudflare 浏览器渲染实验和 MCP 服务器

该项目演示了如何使用 Cloudflare 浏览器渲染为 LLM 上下文提取 Web 内容。它包含 REST API 和 Workers Binding API 的实验，以及可用于为 LLM 提供 Web 上下文的 MCP 服务器实现。

项目结构

cloudflare-browser-rendering/
├── examples/                   # Example implementations and utilities
│   ├── basic-worker-example.js # Basic Worker with Browser Rendering
│   ├── minimal-worker-example.js # Minimal implementation
│   ├── debugging-tools/        # Tools for debugging
│   │   └── debug-test.js       # Debug test utility
│   └── testing/                # Testing utilities
│       └── content-test.js     # Content testing utility
├── experiments/                # Educational experiments
│   ├── basic-rest-api/         # REST API tests
│   ├── puppeteer-binding/      # Workers Binding API tests
│   └── content-extraction/     # Content processing tests
├── src/                        # MCP server source code
│   ├── index.ts                # Main entry point
│   ├── server.ts               # MCP server implementation
│   ├── browser-client.ts       # Browser Rendering client
│   └── content-processor.ts    # Content processing utilities
├── puppeteer-worker.js         # Cloudflare Worker with Browser Rendering binding
├── test-puppeteer.js           # Tests for the main implementation
├── wrangler.toml               # Wrangler configuration for the Worker
├── cline_mcp_settings.json.example # Example MCP settings for Cline
├── .gitignore                  # Git ignore file
└── LICENSE                     # MIT License

Related MCP server: MCP Web Tools Server

先决条件

  • Node.js（v16 或更高版本）

  • 已启用浏览器渲染的 Cloudflare 帐户

  • TypeScript

  • Wrangler CLI（用于部署 Worker）

安装

  1. 克隆存储库：

git clone https://github.com/yourusername/cloudflare-browser-rendering.git
cd cloudflare-browser-rendering

  1. 安装依赖项：

npm install

Cloudflare Worker 设置

  1. 安装 Cloudflare Puppeteer 包：

npm install @cloudflare/puppeteer

  1. 配置Wrangler：

# wrangler.toml
name = "browser-rendering-api"
main = "puppeteer-worker.js"
compatibility_date = "2023-10-30"
compatibility_flags = ["nodejs_compat"]

[browser]
binding = "browser"

  1. 部署 Worker：

npx wrangler deploy

  1. 测试 Worker：

node test-puppeteer.js

运行实验

基本 REST API 实验

此实验演示了如何使用 Cloudflare 浏览器渲染 REST API 来获取和处理 Web 内容：

npm run experiment:rest

Puppeteer 绑定 API 实验

此实验演示了如何将 Cloudflare 浏览器渲染工作器绑定 API 与 Puppeteer 结合使用，实现更高级的浏览器自动化：

npm run experiment:puppeteer

内容提取实验

此实验演示了如何提取和处理 Web 内容，专门用作 LLM 中的上下文：

npm run experiment:content

MCP 服务器

MCP 服务器提供使用 Cloudflare 浏览器渲染来获取和处理 Web 内容的工具，以用作 LLM 中的上下文。

构建 MCP 服务器

npm run build

运行 MCP 服务器

npm start

或者，对于开发来说：

npm run dev

MCP 服务器工具

MCP 服务器提供以下工具：

  1. fetch_page - 获取并处理 LLM 上下文的网页

  2. search_documentation - 搜索 Cloudflare 文档并返回相关内容

  3. extract_structured_content - 使用 CSS 选择器从网页中提取结构化内容

  4. summarize_content - 总结网络内容以获得更简洁的 LLM 上下文

配置

要使用 Cloudflare 浏览器渲染端点，请设置BROWSER_RENDERING_API环境变量：

export BROWSER_RENDERING_API=https://YOUR_WORKER_URL_HERE

将YOUR_WORKER_URL_HERE替换为您已部署的 Cloudflare Worker 的 URL。您需要在多个文件中替换此占位符：

  1. 在测试文件中： test-puppeteer.js 、 examples/debugging-tools/debug-test.js 、 examples/testing/content-test.js

  2. 在 MCP 服务器配置中： cline_mcp_settings.json.example

  3. 在浏览器客户端中： src/browser-client.ts （如果未设置环境变量，则作为后备）

与 Cline 集成

要将 MCP 服务器与 Cline 集成，请将cline_mcp_settings.json.example文件复制到适当的位置：

cp cline_mcp_settings.json.example ~/Library/Application\ Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json

或者将配置添加到您现有的cline_mcp_settings.json文件中。

关键学习

  1. Cloudflare 浏览器渲染需要@cloudflare/puppeteer包与浏览器绑定进行交互。

  2. 使用浏览器绑定的正确模式是：

    import puppeteer from '@cloudflare/puppeteer';

// Then in your handler:
const browser = await puppeteer.launch(env.browser);
const page = await browser.newPage();

  3. 部署使用浏览器渲染绑定的 Worker 时，需要启用nodejs_compat兼容性标志。

  4. 使用后请务必关闭浏览器，以避免资源泄漏。

执照

麻省理工学院

Available Tools

4 tools
extract_structured_contentC

Extracts structured content from a web page using CSS selectors

ParametersJSON Schema
NameRequiredDescriptionDefault
selectorsYesCSS selectors to extract content
urlYesURL to extract content from

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states what the tool does but lacks critical behavioral details: it doesn't specify if it fetches the page internally, handles errors, requires internet access, has rate limits, or what the output format is. For a tool with no annotations, this leaves significant gaps in understanding its operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise and front-loaded: a single sentence that directly states the tool's function without any fluff. Every word earns its place by conveying essential information about extraction, content type, source, and method. It's efficiently structured for quick understanding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of web extraction (involving network calls, parsing, and structured data output), the description is incomplete. There's no output schema, and the description doesn't explain return values, error handling, or behavioral traits. With no annotations and only basic parameter coverage, it fails to provide enough context for effective use in real-world scenarios.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, meaning the input schema already documents both parameters ('url' and 'selectors') with descriptions. The tool description adds no additional meaning beyond what's in the schema—it doesn't explain parameter interactions, provide examples, or clarify semantics. With high schema coverage, the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Extracts structured content from a web page using CSS selectors'. It specifies the verb ('extracts'), resource ('structured content'), and method ('CSS selectors'), which is specific and actionable. However, it doesn't explicitly differentiate from sibling tools like 'fetch_page' or 'summarize_content', which would require a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention scenarios where extraction is preferred over fetching the whole page, searching documentation, summarizing, or taking a screenshot. Without any context or exclusions, users must infer usage from the purpose alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_pageC

Fetches and processes a web page for LLM context

ParametersJSON Schema
NameRequiredDescriptionDefault
includeScreenshotNoWhether to include a screenshot (base64 encoded)
maxContentLengthNoMaximum content length to return
urlYesURL to fetch

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'fetches and processes' but doesn't clarify aspects like rate limits, authentication needs, error handling, or what 'processes' entails (e.g., cleaning HTML, extracting text). This leaves significant gaps for a tool that interacts with external resources.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose without any wasted words. It's appropriately sized for the tool's complexity, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (fetching and processing web pages), lack of annotations, and no output schema, the description is incomplete. It doesn't explain return values, error cases, or behavioral traits, leaving the agent with insufficient information for reliable use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, so the baseline is 3. The description adds no additional meaning beyond the schema, which already details parameters like 'url', 'includeScreenshot', and 'maxContentLength'. No extra syntax, format, or usage context is provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with specific verbs ('fetches and processes') and resource ('a web page'), and specifies the outcome ('for LLM context'). It doesn't explicitly differentiate from sibling tools like 'extract_structured_content' or 'summarize_content', which prevents a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'extract_structured_content' or 'search_documentation'. It lacks context about prerequisites, exclusions, or comparative use cases, offering only a basic functional statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_documentationC

Searches Cloudflare documentation and returns relevant content

ParametersJSON Schema
NameRequiredDescriptionDefault
maxResultsNoMaximum number of results to return
queryYesSearch query

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool searches and returns content, but doesn't describe important behaviors like whether it performs web searches, accesses a local database, requires authentication, has rate limits, or what format the returned content takes (e.g., text snippets, links, full documents).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that gets straight to the point without unnecessary words. It's appropriately sized for a simple search tool, though it could potentially be more structured with additional context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a search tool with no annotations and no output schema, the description is incomplete. It doesn't explain what kind of content is returned (snippets, full pages, metadata), how results are ranked, whether authentication is needed, or any limitations. Given the lack of structured fields, the description should provide more operational context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, clearly documenting both parameters ('query' and 'maxResults'). The description doesn't add any meaningful parameter semantics beyond what the schema already provides, such as search syntax examples or result format details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('Searches') and resource ('Cloudflare documentation'), making it immediately understandable. However, it doesn't distinguish this tool from its sibling tools like 'fetch_page' or 'extract_structured_content', which might also retrieve documentation content in different ways.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, constraints, or compare it to sibling tools like 'fetch_page' (which might retrieve a specific page) or 'summarize_content' (which might process content).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

summarize_contentC

Summarizes web content for more concise LLM context

ParametersJSON Schema
NameRequiredDescriptionDefault
maxLengthNoMaximum length of the summary
urlYesURL to summarize

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'summarizes web content' but doesn't describe how it works (e.g., extraction method, processing time, error handling), what limitations exist (e.g., supported content types, rate limits), or what the output looks like. This leaves significant gaps in understanding the tool's behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and includes a clear goal, making it appropriately sized and well-structured for quick understanding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of summarizing web content, no annotations, and no output schema, the description is incomplete. It doesn't explain the return format, potential errors, or behavioral traits like content processing methods. For a tool with 2 parameters and significant operational implications, more context is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, clearly documenting both parameters ('url' and 'maxLength'). The description adds no additional parameter semantics beyond what the schema provides, such as format details for 'url' or typical values for 'maxLength'. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('summarizes') and resource ('web content'), and it provides the goal ('for more concise LLM context'). However, it doesn't explicitly differentiate from sibling tools like 'extract_structured_content' or 'fetch_page', which might have overlapping functionality.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'extract_structured_content' or 'fetch_page'. It doesn't mention prerequisites, exclusions, or specific contexts where this summarization tool is preferred over other content-handling siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv1.0.0
    • First observedextract_structured_content
    • First observedfetch_page
    • First observedsearch_documentation
    • First observedsummarize_content
Deploy Server
Give agents eyes on the webHosted screenshot MCP. Send a URL, get a pixel-perfect image. $0.002 per grab. Works with Claude, Cursor, and Codex.grabbit.liveAd

TDQS

B3.1/5.0

Scored across 4 tools

Disambiguation4/5

The tools are mostly distinct in purpose, with extract_structured_content, fetch_page, and summarize_content each handling different aspects of web content processing. However, fetch_page and summarize_content could be slightly confused as both relate to processing web content for LLM context, though their specific focuses differ.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern with snake_case, such as extract_structured_content and fetch_page. This uniformity makes the set predictable and easy to understand, with no deviations in naming conventions.

Tool Count3/5

With 4 tools, the count is on the lower side for a web content server, which might feel thin for covering a broad domain like web content processing. However, it is reasonable for a focused set, though it could benefit from additional tools for more comprehensive coverage.

Completeness2/5

There are significant gaps in the tool surface for web content processing. For example, there are no tools for updating or deleting content, handling dynamic content like JavaScript, or managing multiple pages. The inclusion of search_documentation is also inconsistent with the general web content focus, creating a notable gap in core operations.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

Appeared in Searches