Skip to main content
Glama
enesjakozh
amotivv

Web Content MCP Server

by amotivv
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Remote

Experimentos de renderizado del navegador Cloudflare y servidor MCP

Este proyecto demuestra cómo usar la renderización del navegador de Cloudflare para extraer contenido web para el contexto LLM. Incluye experimentos con la API REST y la API de enlace de trabajadores, así como una implementación de servidor MCP que puede utilizarse para proporcionar contexto web a los LLM.

Estructura del proyecto

cloudflare-browser-rendering/
├── examples/                   # Example implementations and utilities
│   ├── basic-worker-example.js # Basic Worker with Browser Rendering
│   ├── minimal-worker-example.js # Minimal implementation
│   ├── debugging-tools/        # Tools for debugging
│   │   └── debug-test.js       # Debug test utility
│   └── testing/                # Testing utilities
│       └── content-test.js     # Content testing utility
├── experiments/                # Educational experiments
│   ├── basic-rest-api/         # REST API tests
│   ├── puppeteer-binding/      # Workers Binding API tests
│   └── content-extraction/     # Content processing tests
├── src/                        # MCP server source code
│   ├── index.ts                # Main entry point
│   ├── server.ts               # MCP server implementation
│   ├── browser-client.ts       # Browser Rendering client
│   └── content-processor.ts    # Content processing utilities
├── puppeteer-worker.js         # Cloudflare Worker with Browser Rendering binding
├── test-puppeteer.js           # Tests for the main implementation
├── wrangler.toml               # Wrangler configuration for the Worker
├── cline_mcp_settings.json.example # Example MCP settings for Cline
├── .gitignore                  # Git ignore file
└── LICENSE                     # MIT License

Related MCP server: MCP Web Tools Server

Prerrequisitos

  • Node.js (v16 o posterior)

  • Una cuenta de Cloudflare con la representación del navegador habilitada

  • Mecanografiado

  • CLI de Wrangler (para implementar el trabajador)

Instalación

  1. Clonar el repositorio:

git clone https://github.com/yourusername/cloudflare-browser-rendering.git
cd cloudflare-browser-rendering

  1. Instalar dependencias:

npm install

Configuración de Cloudflare Worker

  1. Instalar el paquete Puppeteer de Cloudflare:

npm install @cloudflare/puppeteer

  1. Configurar Wrangler:

# wrangler.toml
name = "browser-rendering-api"
main = "puppeteer-worker.js"
compatibility_date = "2023-10-30"
compatibility_flags = ["nodejs_compat"]

[browser]
binding = "browser"

  1. Implementar el trabajador:

npx wrangler deploy

  1. Pon a prueba al trabajador:

node test-puppeteer.js

Ejecución de los experimentos

Experimento básico de API REST

Este experimento demuestra cómo utilizar la API REST de representación del navegador de Cloudflare para obtener y procesar contenido web:

npm run experiment:rest

Experimento de API de enlace de Puppeteer

Este experimento demuestra cómo usar la API de enlace de trabajadores de representación del navegador de Cloudflare con Puppeteer para una automatización más avanzada del navegador:

npm run experiment:puppeteer

Experimento de extracción de contenido

Este experimento demuestra cómo extraer y procesar contenido web específicamente para su uso como contexto en LLM:

npm run experiment:content

Servidor MCP

El servidor MCP proporciona herramientas para obtener y procesar contenido web mediante Cloudflare Browser Rendering para su uso como contexto en LLM.

Construyendo el servidor MCP

npm run build

Ejecución del servidor MCP

npm start

O, para el desarrollo:

npm run dev

Herramientas del servidor MCP

El servidor MCP proporciona las siguientes herramientas:

  1. fetch_page : obtiene y procesa una página web para el contexto LLM

  2. search_documentation : busca la documentación de Cloudflare y devuelve contenido relevante.

  3. extract_structured_content : extrae contenido estructurado de una página web mediante selectores CSS

  4. summarize_content - Resume el contenido web para un contexto LLM más conciso

Configuración

Para utilizar el punto final de representación del navegador de Cloudflare, configure la variable de entorno BROWSER_RENDERING_API :

export BROWSER_RENDERING_API=https://YOUR_WORKER_URL_HERE

Reemplace YOUR_WORKER_URL_HERE con la URL de su Cloudflare Worker implementado. Deberá reemplazar este marcador de posición en varios archivos:

  1. En los archivos de prueba: test-puppeteer.js , examples/debugging-tools/debug-test.js , examples/testing/content-test.js

  2. En la configuración del servidor MCP: cline_mcp_settings.json.example

  3. En el cliente del navegador: src/browser-client.ts (como alternativa si la variable de entorno no está configurada)

Integración con Cline

Para integrar el servidor MCP con Cline, copie el archivo cline_mcp_settings.json.example en la ubicación adecuada:

cp cline_mcp_settings.json.example ~/Library/Application\ Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json

O agregue la configuración a su archivo cline_mcp_settings.json existente.

Aprendizajes clave

  1. La representación del navegador de Cloudflare requiere el paquete @cloudflare/puppeteer para interactuar con el enlace del navegador.

  2. El patrón correcto para utilizar el enlace del navegador es:

    import puppeteer from '@cloudflare/puppeteer';

// Then in your handler:
const browser = await puppeteer.launch(env.browser);
const page = await browser.newPage();

  3. Al implementar un Worker que utiliza el enlace de representación del navegador, debe habilitar el indicador de compatibilidad nodejs_compat .

  4. Cierre siempre el navegador después de usarlo para evitar fugas de recursos.

Licencia

Instituto Tecnológico de Massachusetts (MIT)

Available Tools

4 tools
extract_structured_contentC

Extracts structured content from a web page using CSS selectors

ParametersJSON Schema
NameRequiredDescriptionDefault
selectorsYesCSS selectors to extract content
urlYesURL to extract content from

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states what the tool does but lacks critical behavioral details: it doesn't specify if it fetches the page internally, handles errors, requires internet access, has rate limits, or what the output format is. For a tool with no annotations, this leaves significant gaps in understanding its operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise and front-loaded: a single sentence that directly states the tool's function without any fluff. Every word earns its place by conveying essential information about extraction, content type, source, and method. It's efficiently structured for quick understanding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of web extraction (involving network calls, parsing, and structured data output), the description is incomplete. There's no output schema, and the description doesn't explain return values, error handling, or behavioral traits. With no annotations and only basic parameter coverage, it fails to provide enough context for effective use in real-world scenarios.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, meaning the input schema already documents both parameters ('url' and 'selectors') with descriptions. The tool description adds no additional meaning beyond what's in the schema—it doesn't explain parameter interactions, provide examples, or clarify semantics. With high schema coverage, the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Extracts structured content from a web page using CSS selectors'. It specifies the verb ('extracts'), resource ('structured content'), and method ('CSS selectors'), which is specific and actionable. However, it doesn't explicitly differentiate from sibling tools like 'fetch_page' or 'summarize_content', which would require a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention scenarios where extraction is preferred over fetching the whole page, searching documentation, summarizing, or taking a screenshot. Without any context or exclusions, users must infer usage from the purpose alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_pageC

Fetches and processes a web page for LLM context

ParametersJSON Schema
NameRequiredDescriptionDefault
includeScreenshotNoWhether to include a screenshot (base64 encoded)
maxContentLengthNoMaximum content length to return
urlYesURL to fetch

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'fetches and processes' but doesn't clarify aspects like rate limits, authentication needs, error handling, or what 'processes' entails (e.g., cleaning HTML, extracting text). This leaves significant gaps for a tool that interacts with external resources.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose without any wasted words. It's appropriately sized for the tool's complexity, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (fetching and processing web pages), lack of annotations, and no output schema, the description is incomplete. It doesn't explain return values, error cases, or behavioral traits, leaving the agent with insufficient information for reliable use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, so the baseline is 3. The description adds no additional meaning beyond the schema, which already details parameters like 'url', 'includeScreenshot', and 'maxContentLength'. No extra syntax, format, or usage context is provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with specific verbs ('fetches and processes') and resource ('a web page'), and specifies the outcome ('for LLM context'). It doesn't explicitly differentiate from sibling tools like 'extract_structured_content' or 'summarize_content', which prevents a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'extract_structured_content' or 'search_documentation'. It lacks context about prerequisites, exclusions, or comparative use cases, offering only a basic functional statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_documentationC

Searches Cloudflare documentation and returns relevant content

ParametersJSON Schema
NameRequiredDescriptionDefault
maxResultsNoMaximum number of results to return
queryYesSearch query

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool searches and returns content, but doesn't describe important behaviors like whether it performs web searches, accesses a local database, requires authentication, has rate limits, or what format the returned content takes (e.g., text snippets, links, full documents).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that gets straight to the point without unnecessary words. It's appropriately sized for a simple search tool, though it could potentially be more structured with additional context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a search tool with no annotations and no output schema, the description is incomplete. It doesn't explain what kind of content is returned (snippets, full pages, metadata), how results are ranked, whether authentication is needed, or any limitations. Given the lack of structured fields, the description should provide more operational context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, clearly documenting both parameters ('query' and 'maxResults'). The description doesn't add any meaningful parameter semantics beyond what the schema already provides, such as search syntax examples or result format details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('Searches') and resource ('Cloudflare documentation'), making it immediately understandable. However, it doesn't distinguish this tool from its sibling tools like 'fetch_page' or 'extract_structured_content', which might also retrieve documentation content in different ways.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, constraints, or compare it to sibling tools like 'fetch_page' (which might retrieve a specific page) or 'summarize_content' (which might process content).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

summarize_contentC

Summarizes web content for more concise LLM context

ParametersJSON Schema
NameRequiredDescriptionDefault
maxLengthNoMaximum length of the summary
urlYesURL to summarize

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'summarizes web content' but doesn't describe how it works (e.g., extraction method, processing time, error handling), what limitations exist (e.g., supported content types, rate limits), or what the output looks like. This leaves significant gaps in understanding the tool's behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and includes a clear goal, making it appropriately sized and well-structured for quick understanding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of summarizing web content, no annotations, and no output schema, the description is incomplete. It doesn't explain the return format, potential errors, or behavioral traits like content processing methods. For a tool with 2 parameters and significant operational implications, more context is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, clearly documenting both parameters ('url' and 'maxLength'). The description adds no additional parameter semantics beyond what the schema provides, such as format details for 'url' or typical values for 'maxLength'. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('summarizes') and resource ('web content'), and it provides the goal ('for more concise LLM context'). However, it doesn't explicitly differentiate from sibling tools like 'extract_structured_content' or 'fetch_page', which might have overlapping functionality.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'extract_structured_content' or 'fetch_page'. It doesn't mention prerequisites, exclusions, or specific contexts where this summarization tool is preferred over other content-handling siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv1.0.0
    • First observedextract_structured_content
    • First observedfetch_page
    • First observedsearch_documentation
    • First observedsummarize_content
Deploy Server
Give agents eyes on the webHosted screenshot MCP. Send a URL, get a pixel-perfect image. $0.002 per grab. Works with Claude, Cursor, and Codex.grabbit.liveAd

TDQS

B3.1/5.0

Scored across 4 tools

Disambiguation4/5

The tools are mostly distinct in purpose, with extract_structured_content, fetch_page, and summarize_content each handling different aspects of web content processing. However, fetch_page and summarize_content could be slightly confused as both relate to processing web content for LLM context, though their specific focuses differ.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern with snake_case, such as extract_structured_content and fetch_page. This uniformity makes the set predictable and easy to understand, with no deviations in naming conventions.

Tool Count3/5

With 4 tools, the count is on the lower side for a web content server, which might feel thin for covering a broad domain like web content processing. However, it is reasonable for a focused set, though it could benefit from additional tools for more comprehensive coverage.

Completeness2/5

There are significant gaps in the tool surface for web content processing. For example, there are no tools for updating or deleting content, handling dynamic content like JavaScript, or managing multiple pages. The inclusion of search_documentation is also inconsistent with the general web content focus, creating a notable gap in core operations.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

Appeared in Searches