MCP-сервер для загрузки содержимого веб-страниц с помощью браузера Playwright headless.
Поддержка JavaScript : в отличие от традиционных веб-скрейперов, Fetcher MCP использует Playwright для выполнения JavaScript, что позволяет обрабатывать динамический веб-контент и современные веб-приложения.
Интеллектуальное извлечение контента : встроенный алгоритм читабельности автоматически извлекает основной контент с веб-страниц, удаляя рекламу, навигацию и другие несущественные элементы.
Гибкий формат вывода : поддерживает форматы вывода HTML и Markdown, что упрощает интеграцию с различными нисходящими приложениями.
Параллельная обработка : инструмент fetch_urls обеспечивает одновременную загрузку нескольких URL-адресов, что значительно повышает эффективность пакетных операций.
Оптимизация ресурсов : автоматически блокирует ненужные ресурсы (изображения, таблицы стилей, шрифты, мультимедиа) для снижения использования полосы пропускания и повышения производительности.
Надежная обработка ошибок : комплексная обработка ошибок и ведение журнала обеспечивают надежную работу даже при работе с проблемными веб-страницами.
Настраиваемые параметры : детальный контроль тайм-аутов, извлечения контента и форматирования вывода для различных вариантов использования.
Related MCP server: mcp-playwright-scraper
Запустить напрямую с помощью npx:
npx -y fetcher-mcpПервоначальная настройка — установите нужный браузер, выполнив следующую команду в терминале:
npx playwright install chromiumИспользуйте параметр --transport=http для одновременного запуска служб конечной точки потокового HTTP и конечной точки SSE:
npx -y fetcher-mcp --log --transport=http --host=0.0.0.0 --port=3000После запуска сервер предоставляет следующие конечные точки:
/mcp — потоковая конечная точка HTTP (современный протокол MCP)
/sse — конечная точка SSE (устаревший протокол MCP)
Клиенты могут выбрать способ подключения в зависимости от своих потребностей.
Запустите с параметром --debug , чтобы отобразить окно браузера для отладки:
npx -y fetcher-mcp --debugНастройте этот MCP-сервер в Claude Desktop:
На MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json
В Windows: %APPDATA%/Claude/claude_desktop_config.json
{
"mcpServers": {
"fetcher": {
"command": "npx",
"args": ["-y", "fetcher-mcp"]
}
}
}docker run -p 3000:3000 ghcr.io/jae-jae/fetcher-mcp:latestСоздайте файл docker-compose.yml :
version: "3.8"
services:
fetcher-mcp:
image: ghcr.io/jae-jae/fetcher-mcp:latest
container_name: fetcher-mcp
restart: unless-stopped
ports:
- "3000:3000"
environment:
- NODE_ENV=production
# Using host network mode on Linux hosts can improve browser access efficiency
# network_mode: "host"
volumes:
# For Playwright, may need to share certain system paths
- /tmp:/tmp
# Health check
healthcheck:
test: ["CMD", "wget", "--spider", "-q", "http://localhost:3000"]
interval: 30s
timeout: 10s
retries: 3Затем выполните:
docker-compose up -dfetch_url — Извлечение содержимого веб-страницы с указанного URL-адреса
Использует браузер Playwright headless для анализа JavaScript
Поддерживает интеллектуальное извлечение основного контента и преобразование в Markdown
Поддерживает следующие параметры:
url : URL-адрес веб-страницы для загрузки (обязательный параметр)
timeout : время ожидания загрузки страницы в миллисекундах, по умолчанию 30000 (30 секунд)
waitUntil : указывает, когда навигация считается завершенной, параметры: «load», «domcontentloaded», «networkidle», «commit», по умолчанию «load»
extractContent : следует ли интеллектуально извлекать основное содержимое, по умолчанию true
maxLength : максимальная длина возвращаемого содержимого (в символах), по умолчанию ограничений нет.
returnHtml : Возвращать ли HTML-контент вместо Markdown, по умолчанию false
waitForNavigation : следует ли ждать дополнительной навигации после первоначальной загрузки страницы (полезно для сайтов с проверкой на наличие ботов), по умолчанию — false
navigationTimeout : максимальное время ожидания дополнительной навигации в миллисекундах, по умолчанию 10000 (10 секунд)
disableMedia : Отключать ли медиаресурсы (изображения, таблицы стилей, шрифты, медиа), по умолчанию true
debug : Включать ли режим отладки (отображение окна браузера), переопределяет флаг командной строки --debug, если он указан
fetch_urls — пакетное извлечение содержимого веб-страницы из нескольких URL-адресов параллельно
Использует параллельную выборку из нескольких вкладок для повышения производительности
Возвращает объединенные результаты с четким разделением веб-страниц
Поддерживает следующие параметры:
urls : Массив URL-адресов для извлечения (обязательный параметр)
Остальные параметры такие же, как у fetch_url
Дождитесь полной загрузки : для веб-сайтов, использующих CAPTCHA, перенаправления или другие механизмы проверки, включите в запрос:
Please wait for the page to fully loadПри этом будет использоваться параметр waitForNavigation: true .
Увеличьте длительность тайм-аута : для веб-сайтов, которые загружаются медленно:
Please set the page loading timeout to 60 secondsЭто соответствующим образом настраивает параметры timeout и navigationTimeout .
Сохранение исходной структуры HTML : когда извлечение контента может завершиться неудачей:
Please preserve the original HTML contentУстанавливает extractContent: false и returnHtml: true .
Извлечь полное содержимое страницы : если извлеченное содержимое слишком ограничено:
Please fetch the complete webpage content instead of just the main contentУстанавливает extractContent: false .
Вернуть содержимое как HTML : когда требуется формат HTML вместо Markdown по умолчанию:
Please return the content in HTML formatУстанавливает returnHtml: true .
Активация динамической отладки : для отображения окна браузера во время определенной операции выборки:
Please enable debug mode for this fetch operationЭто устанавливает debug: true даже если сервер был запущен без флага --debug .
Ручной вход : Чтобы войти, используя свои учетные данные:
Please run in debug mode so I can manually log in to the websiteУстанавливает debug: true или использует флаг --debug , оставляя окно браузера открытым для ручного входа.
Взаимодействие с отладочным браузером : Когда включен режим отладки:
Окно браузера остается открытым
Вы можете вручную войти на сайт, используя свои учетные данные.
После завершения входа в систему содержимое будет извлечено с помощью вашего аутентифицированного сеанса.
Включить отладку для определенных запросов : даже если сервер уже запущен, вы можете включить режим отладки для определенного запроса:
Please enable debug mode for this authentication stepУстанавливает debug: true только для этого конкретного запроса, открывая окно браузера для ручного входа.
npm installУстановите браузеры, необходимые для Playwright:
npm run install-browsernpm run buildИспользуйте MCP Inspector для отладки:
npm run inspectorВы также можете включить видимый режим браузера для отладки:
node build/index.js --debugg-search-mcp : Мощный сервер MCP для поиска Google, который позволяет выполнять параллельный поиск по нескольким ключевым словам одновременно. Идеально подходит для пакетных операций поиска и сбора данных.
Лицензировано в соответствии с лицензией MIT
Install Playwright Chromium browser binary. Call this if you get an error about the browser not being installed.
| Name | Required | Description | Default |
|---|---|---|---|
| withDeps | No | Install system dependencies required by Chromium browser. Default is false | |
| force | No | Force installation even if Chromium is already installed. Default is false |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the tool's purpose and trigger condition but doesn't describe what happens during installation (e.g., download size, time, network requirements), potential side effects, or what constitutes successful completion. It provides basic context but lacks richer behavioral details needed for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly concise with two sentences that each serve a distinct purpose: the first states what the tool does, and the second provides usage guidance. There's zero wasted language, and the information is front-loaded with the core functionality stated immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given this is a mutation tool (installation) with no annotations and no output schema, the description provides good context about when to use it and what it does. However, it doesn't explain what happens after installation completes or what the agent should expect, leaving some gaps in completeness for a tool that modifies system state.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both parameters (withDeps, force) well-documented in the schema. The description doesn't add any parameter-specific information beyond what's already in the schema, so it meets the baseline of 3 where the schema does the heavy lifting without additional value from the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Install Playwright Chromium browser binary') and resource ('Chromium browser binary'), distinguishing it from sibling tools like fetch_url and fetch_urls which perform different operations. It provides a concrete use case ('if you get an error about the browser not being installed') that makes the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool ('if you get an error about the browser not being installed'), providing clear contextual guidance. While it doesn't mention alternatives or exclusions, the specific error-based trigger makes usage guidelines comprehensive and actionable for the stated purpose.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Retrieve web page content from a specified URL
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to fetch. Make sure to include the schema (http:// or https:// if not defined, preferring https for most cases) | |
| timeout | No | Page loading timeout in milliseconds, default is 30000 (30 seconds) | |
| waitUntil | No | Specifies when navigation is considered complete, options: 'load', 'domcontentloaded', 'networkidle', 'commit', default is 'load' | |
| extractContent | No | Whether to intelligently extract the main content, default is true | |
| maxLength | No | Maximum length of returned content (in characters), default is no limit | |
| returnHtml | No | Whether to return HTML content instead of Markdown, default is false | |
| waitForNavigation | No | Whether to wait for additional navigation after initial page load (useful for sites with anti-bot verification), default is false | |
| navigationTimeout | No | Maximum time to wait for additional navigation in milliseconds, default is 10000 (10 seconds) | |
| disableMedia | No | Whether to disable media resources (images, stylesheets, fonts, media), default is true | |
| debug | No | Whether to enable debug mode (showing browser window), overrides the --debug command line flag if specified |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure but only states the basic action ('Retrieve web page content'). It doesn't mention potential side effects (e.g., network requests, rate limits), authentication needs, error handling, or what the return content looks like (structure or format). This is inadequate for a tool with 10 parameters and no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. It earns its place by clearly stating what the tool does, making it easy to scan and understand.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with 10 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain behavioral traits, return values, or usage context, leaving significant gaps that could hinder correct tool selection and invocation by an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, providing detailed documentation for all 10 parameters. The description adds no additional parameter semantics beyond implying a 'URL' input, which is already covered in the schema. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Retrieve') and resource ('web page content from a specified URL'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'fetch_urls' (plural) or 'browser_install', but the singular 'URL' suggests this is for single-page retrieval versus batch operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'fetch_urls' or 'browser_install'. The description lacks context about prerequisites, typical use cases, or exclusions, leaving the agent to infer usage based on tool names alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Retrieve web page content from multiple specified URLs
| Name | Required | Description | Default |
|---|---|---|---|
| urls | Yes | Array of URLs to fetch | |
| timeout | No | Page loading timeout in milliseconds, default is 30000 (30 seconds) | |
| waitUntil | No | Specifies when navigation is considered complete, options: 'load', 'domcontentloaded', 'networkidle', 'commit', default is 'load' | |
| extractContent | No | Whether to intelligently extract the main content, default is true | |
| maxLength | No | Maximum length of returned content (in characters), default is no limit | |
| returnHtml | No | Whether to return HTML content instead of Markdown, default is false | |
| waitForNavigation | No | Whether to wait for additional navigation after initial page load (useful for sites with anti-bot verification), default is false | |
| navigationTimeout | No | Maximum time to wait for additional navigation in milliseconds, default is 10000 (10 seconds) | |
| disableMedia | No | Whether to disable media resources (images, stylesheets, fonts, media), default is true | |
| debug | No | Whether to enable debug mode (showing browser window), overrides the --debug command line flag if specified |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves content but lacks details on critical behaviors: it doesn't mention authentication needs, rate limits, error handling, or what the output looks like (e.g., format, structure). For a tool with 10 parameters and no output schema, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence: 'Retrieve web page content from multiple specified URLs.' It's front-loaded with the core purpose, has zero wasted words, and is appropriately sized for the tool's complexity. Every part of the sentence earns its place by clearly stating the action and scope.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (10 parameters, no annotations, no output schema), the description is incomplete. It doesn't address behavioral aspects like how content is returned, error cases, or performance constraints. While the schema covers parameters well, the description fails to provide necessary context for effective use, especially without annotations or output schema to fill gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, meaning all parameters are well-documented in the input schema itself. The description doesn't add any semantic details beyond what's in the schema (e.g., it doesn't explain how 'urls' are processed or interactions between parameters). With high schema coverage, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Retrieve web page content from multiple specified URLs.' It specifies the verb ('Retrieve'), resource ('web page content'), and scope ('multiple specified URLs'), which is specific and actionable. However, it doesn't explicitly distinguish this tool from its sibling 'fetch_url' (which presumably handles single URLs), missing full differentiation for a top score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention the sibling tool 'fetch_url' or explain scenarios where fetching multiple URLs is preferred over single ones. There's no context about prerequisites, limitations, or best practices, leaving the agent without usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v1.0.0browser_installfetch_urlfetch_urlsScored across 3 tools
The tools are mostly distinct, with 'fetch_url' and 'fetch_urls' clearly differentiated by single vs. multiple URLs. However, an agent might initially confuse 'fetch_url' and 'fetch_urls' as similar operations, though the descriptions clarify the distinction. 'browser_install' is completely separate in purpose.
All tool names follow a consistent snake_case pattern with clear verb_noun structure: 'browser_install', 'fetch_url', and 'fetch_urls'. The naming is predictable and readable throughout the set.
With only 3 tools, the server feels thin for a web fetching domain, lacking operations like navigation, interaction, or content parsing. While the core fetching is covered, the scope seems limited compared to typical web automation needs.
For a web fetching server, there are significant gaps: no tools for interacting with pages (e.g., clicking, filling forms), extracting specific content, handling JavaScript, or managing sessions. The surface is severely incomplete for common web automation tasks beyond basic URL retrieval.
MCP server (stdio): fetch web pages as clean readable markdown via the AgentForge API
MCP server for Firecrawl — web search, scraping, and biomedical/arXiv paper search.
Turn any public website into an MCP server for agents to search, read and navigate.
MCP server for web extraction and rendering via AceDataCloud WebExtrator