MCP URL Fetcher
MCP URL 格式转换器
模型上下文协议 (MCP) 服务器可从任何 URL 获取内容并将其转换为所需的输出格式。
概述
MCP URL 格式转换器提供从任何 Web URL 检索内容并将其转换为各种格式(HTML、JSON、Markdown 或纯文本)的工具,无论原始内容类型如何。它旨在与任何兼容 MCP 的客户端(包括 Claude for Desktop)配合使用,使 LLM 能够以一致的格式访问、转换和分析 Web 内容。
Related MCP server: MCP Web Tools Server
特征
🔄格式转换:将任何 Web 内容转换为 HTML、JSON、Markdown 或纯文本
🌐通用输入支持:处理网站、API、原始文件等
🔍自动内容检测:智能识别源格式
🧰强大的库支持:使用行业标准库:
Cheerio 用于 HTML 解析
标记为 Markdown 处理
用于 XML 处理的 Fast-XML-Parser
CSVtoJSON 用于 CSV 转换
SanitizeHTML 安全性
拒绝 HTML 到 Markdown 的转换
🔧高级格式处理:
使用元数据提取进行 HTML 解析
JSON 漂亮打印和结构保存
Markdown 渲染与样式
CSV 到表格的转换
XML 到 JSON 的转换
📜历史记录跟踪:维护最近获取的 URL 的日志
🛡️安全焦点:内容清理以防止 XSS 攻击
安装
先决条件
Node.js 16.x 或更高版本
npm 或 yarn
快速入门
克隆存储库:
git clone https://github.com/yourusername/mcp-url-converter.git cd mcp-url-converter安装依赖项:
npm install构建项目:
npm run build运行服务器:
npm start
与 Claude for Desktop 集成
打开您的 Claude for Desktop 配置文件:
macOS:
~/Library/Application Support/Claude/claude_desktop_config.jsonWindows:
%APPDATA%\Claude\claude_desktop_config.json
将 URL 转换器服务器添加到您的配置中:
{ "mcpServers": { "url-converter": { "command": "node", "args": ["/absolute/path/to/mcp-url-converter/build/index.js"] } } }重启 Claude 桌面版
可用工具
fetch
从任何 URL 获取内容并自动检测最佳输出格式。
参数:
url(字符串,必需):获取内容的 URLformat(字符串,可选):要转换为的格式(auto,html,json,markdown,text)。默认值:auto
例子:
Can you fetch https://example.com and choose the best format to display it?fetch-json
从任何 URL 获取内容并将其转换为 JSON 格式。
参数:
url(字符串,必需):获取内容的 URLprettyPrint(布尔值,可选):是否以美观的方式打印 JSON。默认值:true
例子:
Can you fetch https://example.com and convert it to JSON format?fetch-html
从任何 URL 获取内容并将其转换为 HTML 格式。
参数:
url(字符串,必需):获取内容的 URLextractText(布尔值,可选):是否仅提取文本内容。默认值:false
例子:
Can you fetch https://api.example.com/users and convert it to HTML?fetch-markdown
从任何 URL 获取内容并将其转换为 Markdown 格式。
参数:
url(字符串,必需):获取内容的 URL
例子:
Can you fetch https://example.com and convert it to Markdown?fetch-text
从任何 URL 获取内容并将其转换为纯文本格式。
参数:
url(字符串,必需):获取内容的 URL
例子:
Can you fetch https://example.com and convert it to plain text?web-search和deep-research
这些工具为 Perplexity 搜索功能提供接口(当 MCP 主机支持时)。
可用资源
recent-urls://list
返回最近获取的带有时间戳和输出格式的 URL 列表。
例子:
What URLs have I fetched recently?安全
该服务器实施了多项安全措施:
使用
sanitize-html进行 HTML 清理以防止 XSS 攻击处理之前的内容验证
错误处理和安全默认值
使用 Zod 进行输入参数验证
安全输出编码
测试
您可以使用 MCP 检查器测试服务器:
npm run test故障排除
常见问题
连接错误:验证 URL 是否可访问且格式正确
转换错误:某些复杂内容可能无法在格式之间顺利转换
跨域问题:某些网站可能会阻止来自未知来源的请求
调试模式
要获取更多调试信息,请设置DEBUG环境变量:
DEBUG=mcp:* npm start执照
该项目根据 MIT 许可证获得许可 - 有关详细信息,请参阅 LICENSE 文件。
致谢
使用模型上下文协议构建
使用现代的、积极维护的库,并注重安全
基于 OWASP 建议的清理方法
最后更新:2025年3月29日
Available Tools
5 toolsfetchC
Fetch content from a URL with automatic content type detection
| Name | Required | Description | Default |
|---|---|---|---|
| format | No | Format to convert to (default: auto) | |
| url | Yes | URL to fetch content from |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions automatic content type detection, which hints at behavior, but lacks critical details like error handling, timeouts, authentication needs, rate limits, or response structure. For a tool that interacts with external URLs, this is a significant gap in behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that states the core functionality without waste. It is front-loaded and appropriately sized for the tool's purpose, making it easy to parse and understand quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of fetching from URLs, lack of annotations, and no output schema, the description is incomplete. It does not cover potential issues like network errors, content parsing, or return values, which are crucial for an agent to use the tool effectively in varied contexts.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear descriptions for both parameters (url and format). The description adds minimal value beyond the schema, as it only implies content type detection relates to the format parameter. No additional semantics or usage examples are provided, so it meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool's purpose as fetching content from a URL with automatic content type detection, which is clear but vague about what 'content' entails. It does not distinguish from sibling tools like fetch-html, fetch-json, etc., which likely fetch specific formats, making it less specific. The verb 'fetch' is generic, and the description lacks detail on the resource or output type beyond detection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus its siblings (fetch-html, fetch-json, etc.), which are explicitly named alternatives. It mentions automatic content type detection but does not specify scenarios where this is preferred over format-specific tools, leaving the agent without clear usage context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
fetch-htmlB
Fetch content from any URL and convert to HTML format
| Name | Required | Description | Default |
|---|---|---|---|
| extractText | No | Whether to extract text content only (default: false) | |
| url | Yes | URL to fetch content from |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions fetching and converting to HTML, it doesn't address important behavioral aspects like error handling (e.g., what happens with invalid URLs), authentication requirements, rate limits, timeout behavior, or whether the tool performs any sanitization of the fetched HTML.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that communicates the core functionality without any wasted words. It's appropriately sized for a simple tool and front-loads the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a relatively simple tool with good schema coverage but no annotations and no output schema, the description provides the basic purpose but lacks important contextual information. It doesn't explain what the HTML output looks like, whether it includes metadata, or how it handles different content types. The absence of output schema means the description should ideally provide some information about return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with both parameters clearly documented. The description adds no additional parameter information beyond what's in the schema. According to scoring rules, when schema_description_coverage is high (>80%), the baseline is 3 even with no param info in the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('fetch') and resource ('content from any URL') with the specific output format ('HTML format'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like fetch-json or fetch-markdown, which presumably fetch the same content but convert to different formats.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus its siblings (fetch, fetch-json, fetch-markdown, fetch-text). It mentions converting to HTML format, but doesn't explain when HTML format is preferable over other formats or what distinguishes it from the generic 'fetch' tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
fetch-jsonC
Fetch content from any URL and convert to JSON format
| Name | Required | Description | Default |
|---|---|---|---|
| prettyPrint | No | Whether to pretty-print the JSON (default: true) | |
| url | Yes | URL to fetch content from |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions fetching and converting to JSON but fails to describe critical behaviors such as error handling (e.g., for non-JSON content), authentication needs, rate limits, or what happens if the URL is inaccessible. This leaves significant gaps for a tool that interacts with external resources.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core functionality ('fetch content from any URL') and adds the key detail ('convert to JSON format') without any wasted words. It is appropriately sized for the tool's scope.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (fetching from external URLs with potential for errors) and the absence of both annotations and an output schema, the description is insufficient. It doesn't explain return values, error cases, or behavioral nuances, making it incomplete for safe and effective use by an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, clearly documenting both parameters (url and prettyPrint). The description adds no additional semantic meaning beyond what the schema provides, such as examples or constraints. With high schema coverage, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('fetch') and resource ('content from any URL'), and specifies the output format ('convert to JSON format'). However, it doesn't explicitly differentiate from sibling tools like 'fetch-html' or 'fetch-markdown' beyond the JSON output mention, which is implied but not directly contrasted.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus its siblings (fetch, fetch-html, fetch-markdown, fetch-text). It lacks explicit context about alternatives, prerequisites, or exclusions, leaving the agent to infer usage based solely on the output format.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
fetch-markdownA
Fetch content from any URL and convert to Markdown format
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to fetch content from |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions fetching and converting to Markdown but does not cover critical aspects such as error handling (e.g., invalid URLs, network failures), authentication needs, rate limits, or what happens if conversion fails. This leaves significant gaps in understanding the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that is front-loaded with the core functionality. There is no wasted language, and it directly communicates the tool's purpose without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (fetching and converting web content) and lack of annotations and output schema, the description is incomplete. It covers the basic purpose but misses behavioral details and output expectations. However, it is adequate for a simple tool with high schema coverage, though it could benefit from more context on limitations or results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the 'url' parameter fully documented. The description adds no additional meaning beyond the schema, such as URL format constraints or examples. Since schema coverage is high, the baseline score of 3 is appropriate, as the description does not compensate but also does not detract.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('fetch content from any URL') and transformation ('convert to Markdown format'), distinguishing it from sibling tools like fetch-html, fetch-json, and fetch-text by specifying the output format. It uses precise verbs and identifies the resource (URL content).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when Markdown output is needed from a URL, but it does not explicitly state when to use this tool versus alternatives like fetch-html or fetch-text. There is no guidance on exclusions or prerequisites, leaving usage context to inference from the tool name and description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
fetch-textC
Fetch content from any URL and convert to plain text format
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to fetch content from |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions fetching and converting to plain text, but lacks details on error handling, rate limits, authentication needs, or what happens with non-text content (e.g., images, binary files). For a tool with no annotations, this leaves significant gaps in understanding its behavior and constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the key information: fetching content and converting to plain text. There is no wasted verbiage, and it directly communicates the tool's core functionality without unnecessary details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (fetching from URLs, which can involve network issues, varied content types) and the absence of annotations and output schema, the description is incomplete. It doesn't address potential errors, output format details (e.g., how plain text is extracted), or behavioral traits like timeouts or size limits. This leaves the agent with insufficient context for reliable use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the 'url' parameter clearly documented. The description adds no additional semantic information beyond what the schema provides (e.g., it doesn't specify URL formats, protocols supported, or content types). With high schema coverage, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('fetch content') and resource ('from any URL'), and specifies the output format ('plain text'). It distinguishes from siblings by mentioning the conversion to plain text, which suggests differentiation from fetch-html, fetch-json, and fetch-markdown. However, it doesn't explicitly name the siblings or detail how it differs from the generic 'fetch' tool, keeping it from a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus its siblings (fetch, fetch-html, fetch-json, fetch-markdown). It implies usage for converting content to plain text, but doesn't specify scenarios, exclusions, or alternatives. This lack of explicit context leaves the agent to infer usage, which is insufficient for effective tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
5 tool updates
v1.0.0- First observed
fetch - First observed
fetch-html - First observed
fetch-json - First observed
fetch-markdown - First observed
fetch-text
TDQS
Scored across 5 tools
The tools have significant overlap in purpose, as all fetch content from URLs, differing only in output format. An agent could easily misselect between fetch-html, fetch-json, fetch-markdown, and fetch-text, since the descriptions don't clarify when to use one format over another. The generic 'fetch' tool with automatic detection further confuses boundaries, as it might duplicate or conflict with the format-specific tools.
Tool names follow a highly consistent verb-noun pattern throughout, with all tools using 'fetch' as the verb followed by a hyphen and format descriptor (e.g., fetch-html, fetch-json). There are no deviations in style or convention, making the naming predictable and easy to parse for an agent.
With 5 tools, the count is reasonable for a URL fetching server, but it feels borderline due to redundancy. The tools could potentially be consolidated into fewer, more flexible tools (e.g., a single fetch tool with a format parameter), making the current set feel slightly over-specified for the simple domain of fetching URLs.
For the domain of fetching URL content, the tool set is nearly complete, covering automatic detection and common output formats (HTML, JSON, Markdown, plain text). A minor gap exists in handling errors or advanced configurations (e.g., headers, timeouts), but agents can likely work around this with the provided tools for basic fetching tasks.
Maintenance
Related MCP Connectors
A comprehensive Model Context Protocol (MCP) server that enables AI assistants to interact with yo…
A Model Context Protocol server for Wix AI tools
MCP server for Firecrawl — web search, scraping, and biomedical/arXiv paper search.
Jina AI Reader/Search MCP — turn any URL into clean LLM-ready markdown, plus web search.
Related MCP Servers
- AlicenseAqualityFmaintenanceA production-ready Model Context Protocol server that enables language models to leverage AI-powered web scraping capabilities, offering tools for transforming webpages to markdown, extracting structured data, and executing AI-powered web searches.8122 PyPI115MIT
- AlicenseNot gradedqualityDmaintenanceA Model Context Protocol server that allows LLMs to interact with web content through standardized tools, currently supporting web scraping functionality.1MIT
- AlicenseBqualityDmaintenanceA Model Context Protocol server that intelligently fetches and processes web content, transforming websites and documentation into clean, structured markdown with nested URL crawling capabilities.210 npm9MIT
- AlicenseBqualityDmaintenanceA Model Context Protocol server that enables AI assistants to securely fetch and extract readable text content from web pages through a standardized interface.1MIT