Skip to main content
Glama
jae-jae
by jae-jae

中文 | Deutsch | Español | français | 日本語 | 한국어 | Português | Русский

Fetcher MCP

MCP server for fetch web page content using Playwright headless browser.

🌟 Recommended: OllaMan - Powerful Ollama AI Model Manager.

Advantages

  • JavaScript Support: Unlike traditional web scrapers, Fetcher MCP uses Playwright to execute JavaScript, making it capable of handling dynamic web content and modern web applications.

  • Intelligent Content Extraction: Built-in Readability algorithm automatically extracts the main content from web pages, removing ads, navigation, and other non-essential elements.

  • Flexible Output Format: Supports both HTML and Markdown output formats, making it easy to integrate with various downstream applications.

  • Parallel Processing: The fetch_urls tool enables concurrent fetching of multiple URLs, significantly improving efficiency for batch operations.

  • Resource Optimization: Automatically blocks unnecessary resources (images, stylesheets, fonts, media) to reduce bandwidth usage and improve performance.

  • Robust Error Handling: Comprehensive error handling and logging ensure reliable operation even when dealing with problematic web pages.

  • Configurable Parameters: Fine-grained control over timeouts, content extraction, and output formatting to suit different use cases.

Related MCP server: mcp-playwright-scraper

Quick Start

Run directly with npx:

npx -y fetcher-mcp

First time setup - install the required browser by running the following command in your terminal:

npx playwright install chromium

HTTP and SSE Transport

Use the --transport=http parameter to start both Streamable HTTP endpoint and SSE endpoint services simultaneously:

npx -y fetcher-mcp --log --transport=http --host=0.0.0.0 --port=3000

After startup, the server provides the following endpoints:

  • /mcp - Streamable HTTP endpoint (modern MCP protocol)

  • /sse - SSE endpoint (legacy MCP protocol)

Clients can choose which method to connect based on their needs.

Debug Mode

Run with the --debug option to show the browser window for debugging:

npx -y fetcher-mcp --debug

Configuration MCP

Configure this MCP server in Claude Desktop:

On MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json

On Windows: %APPDATA%/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "fetcher": {
      "command": "npx",
      "args": ["-y", "fetcher-mcp"]
    }
  }
}

Docker Deployment

Running with Docker

docker run -p 3000:3000 ghcr.io/jae-jae/fetcher-mcp:latest

Deploying with Docker Compose

Create a docker-compose.yml file:

version: "3.8"

services:
  fetcher-mcp:
    image: ghcr.io/jae-jae/fetcher-mcp:latest
    container_name: fetcher-mcp
    restart: unless-stopped
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
    # Using host network mode on Linux hosts can improve browser access efficiency
    # network_mode: "host"
    volumes:
      # For Playwright, may need to share certain system paths
      - /tmp:/tmp
    # Health check
    healthcheck:
      test: ["CMD", "wget", "--spider", "-q", "http://localhost:3000"]
      interval: 30s
      timeout: 10s
      retries: 3

Then run:

docker-compose up -d

Features

  • fetch_url - Retrieve web page content from a specified URL

    • Uses Playwright headless browser to parse JavaScript

    • Supports intelligent extraction of main content and conversion to Markdown

    • Supports the following parameters:

      • url: The URL of the web page to fetch (required parameter)

      • timeout: Page loading timeout in milliseconds, default is 30000 (30 seconds)

      • waitUntil: Specifies when navigation is considered complete, options: 'load', 'domcontentloaded', 'networkidle', 'commit', default is 'load'

      • extractContent: Whether to intelligently extract the main content, default is true

      • maxLength: Maximum length of returned content (in characters), default is no limit

      • returnHtml: Whether to return HTML content instead of Markdown, default is false

      • waitForNavigation: Whether to wait for additional navigation after initial page load (useful for sites with anti-bot verification), default is false

      • navigationTimeout: Maximum time to wait for additional navigation in milliseconds, default is 10000 (10 seconds)

      • disableMedia: Whether to disable media resources (images, stylesheets, fonts, media), default is true

      • debug: Whether to enable debug mode (showing browser window), overrides the --debug command line flag if specified

  • fetch_urls - Batch retrieve web page content from multiple URLs in parallel

    • Uses multi-tab parallel fetching for improved performance

    • Returns combined results with clear separation between webpages

    • Supports the following parameters:

      • urls: Array of URLs to fetch (required parameter)

      • Other parameters are the same as fetch_url

  • browser_install - Install Playwright Chromium browser binary automatically

    • Installs required Chromium browser binary when not available

    • Automatically suggested when browser installation errors occur

    • Supports the following parameters:

      • withDeps: Install system dependencies required by Chromium browser, default is false

      • force: Force installation even if Chromium is already installed, default is false

Tips

Handling Special Website Scenarios

Dealing with Anti-Crawler Mechanisms

  • Wait for Complete Loading: For websites using CAPTCHA, redirects, or other verification mechanisms, include in your prompt:

    Please wait for the page to fully load

    This will use the waitForNavigation: true parameter.

  • Increase Timeout Duration: For websites that load slowly:

    Please set the page loading timeout to 60 seconds

    This adjusts both timeout and navigationTimeout parameters accordingly.

Content Retrieval Adjustments

  • Preserve Original HTML Structure: When content extraction might fail:

    Please preserve the original HTML content

    Sets extractContent: false and returnHtml: true.

  • Fetch Complete Page Content: When extracted content is too limited:

    Please fetch the complete webpage content instead of just the main content

    Sets extractContent: false.

  • Return Content as HTML: When HTML format is needed instead of default Markdown:

    Please return the content in HTML format

    Sets returnHtml: true.

Debugging and Authentication

Enabling Debug Mode

  • Dynamic Debug Activation: To display the browser window during a specific fetch operation:

    Please enable debug mode for this fetch operation

    This sets debug: true even if the server was started without the --debug flag.

Using Custom Cookies for Authentication

  • Manual Login: To login using your own credentials:

    Please run in debug mode so I can manually log in to the website

    Sets debug: true or uses the --debug flag, keeping the browser window open for manual login.

  • Interacting with Debug Browser: When debug mode is enabled:

    1. The browser window remains open

    2. You can manually log into the website using your credentials

    3. After login is complete, content will be fetched with your authenticated session

  • Enable Debug for Specific Requests: Even if the server is already running, you can enable debug mode for a specific request:

    Please enable debug mode for this authentication step

    Sets debug: true for this specific request only, opening the browser window for manual login.

Development

Install Dependencies

npm install

Install Playwright Browser

Install the browsers needed for Playwright:

npm run install-browser

Build the Server

npm run build

Debugging

Use MCP Inspector for debugging:

npm run inspector

You can also enable visible browser mode for debugging:

node build/index.js --debug
  • g-search-mcp: A powerful MCP server for Google search that enables parallel searching with multiple keywords simultaneously. Perfect for batch search operations and data collection.

License

Licensed under the MIT License

Powered by DartNode

Available Tools

3 tools
browser_installA

Install Playwright Chromium browser binary. Call this if you get an error about the browser not being installed.

ParametersJSON Schema
NameRequiredDescriptionDefault
withDepsNoInstall system dependencies required by Chromium browser. Default is false
forceNoForce installation even if Chromium is already installed. Default is false

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the tool's purpose and trigger condition but doesn't describe what happens during installation (e.g., download size, time, network requirements), potential side effects, or what constitutes successful completion. It provides basic context but lacks richer behavioral details needed for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is perfectly concise with two sentences that each serve a distinct purpose: the first states what the tool does, and the second provides usage guidance. There's zero wasted language, and the information is front-loaded with the core functionality stated immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given this is a mutation tool (installation) with no annotations and no output schema, the description provides good context about when to use it and what it does. However, it doesn't explain what happens after installation completes or what the agent should expect, leaving some gaps in completeness for a tool that modifies system state.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both parameters (withDeps, force) well-documented in the schema. The description doesn't add any parameter-specific information beyond what's already in the schema, so it meets the baseline of 3 where the schema does the heavy lifting without additional value from the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Install Playwright Chromium browser binary') and resource ('Chromium browser binary'), distinguishing it from sibling tools like fetch_url and fetch_urls which perform different operations. It provides a concrete use case ('if you get an error about the browser not being installed') that makes the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool ('if you get an error about the browser not being installed'), providing clear contextual guidance. While it doesn't mention alternatives or exclusions, the specific error-based trigger makes usage guidelines comprehensive and actionable for the stated purpose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_urlC

Retrieve web page content from a specified URL

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesURL to fetch. Make sure to include the schema (http:// or https:// if not defined, preferring https for most cases)
timeoutNoPage loading timeout in milliseconds, default is 30000 (30 seconds)
waitUntilNoSpecifies when navigation is considered complete, options: 'load', 'domcontentloaded', 'networkidle', 'commit', default is 'load'
extractContentNoWhether to intelligently extract the main content, default is true
maxLengthNoMaximum length of returned content (in characters), default is no limit
returnHtmlNoWhether to return HTML content instead of Markdown, default is false
waitForNavigationNoWhether to wait for additional navigation after initial page load (useful for sites with anti-bot verification), default is false
navigationTimeoutNoMaximum time to wait for additional navigation in milliseconds, default is 10000 (10 seconds)
disableMediaNoWhether to disable media resources (images, stylesheets, fonts, media), default is true
debugNoWhether to enable debug mode (showing browser window), overrides the --debug command line flag if specified

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure but only states the basic action ('Retrieve web page content'). It doesn't mention potential side effects (e.g., network requests, rate limits), authentication needs, error handling, or what the return content looks like (structure or format). This is inadequate for a tool with 10 parameters and no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. It earns its place by clearly stating what the tool does, making it easy to scan and understand.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 10 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain behavioral traits, return values, or usage context, leaving significant gaps that could hinder correct tool selection and invocation by an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, providing detailed documentation for all 10 parameters. The description adds no additional parameter semantics beyond implying a 'URL' input, which is already covered in the schema. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Retrieve') and resource ('web page content from a specified URL'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'fetch_urls' (plural) or 'browser_install', but the singular 'URL' suggests this is for single-page retrieval versus batch operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'fetch_urls' or 'browser_install'. The description lacks context about prerequisites, typical use cases, or exclusions, leaving the agent to infer usage based on tool names alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_urlsC

Retrieve web page content from multiple specified URLs

ParametersJSON Schema
NameRequiredDescriptionDefault
urlsYesArray of URLs to fetch
timeoutNoPage loading timeout in milliseconds, default is 30000 (30 seconds)
waitUntilNoSpecifies when navigation is considered complete, options: 'load', 'domcontentloaded', 'networkidle', 'commit', default is 'load'
extractContentNoWhether to intelligently extract the main content, default is true
maxLengthNoMaximum length of returned content (in characters), default is no limit
returnHtmlNoWhether to return HTML content instead of Markdown, default is false
waitForNavigationNoWhether to wait for additional navigation after initial page load (useful for sites with anti-bot verification), default is false
navigationTimeoutNoMaximum time to wait for additional navigation in milliseconds, default is 10000 (10 seconds)
disableMediaNoWhether to disable media resources (images, stylesheets, fonts, media), default is true
debugNoWhether to enable debug mode (showing browser window), overrides the --debug command line flag if specified

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. While 'Retrieve web page content' implies a read operation, it doesn't mention potential side effects (e.g., network requests, rate limits), authentication needs, error handling, or what format/content is returned. The description is minimal and lacks crucial operational context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that communicates the core purpose without any fluff. It's appropriately sized for a tool with well-documented parameters in the schema, though it could benefit from additional context about when to use it.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 10 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what 'web page content' means in practice (HTML? text? metadata?), how results are structured for multiple URLs, error conditions, or performance characteristics. The agent lacks crucial information to use this tool effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description mentions 'multiple specified URLs' which aligns with the 'urls' parameter, but adds no additional semantic context beyond what the comprehensive schema already provides (100% coverage). With excellent schema documentation, the baseline is 3 even though the description offers minimal parameter insight.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Retrieve web page content') and target ('from multiple specified URLs'), making the purpose immediately understandable. It distinguishes from the sibling 'fetch_url' by specifying 'multiple' URLs, though it doesn't explain the functional difference between the two tools beyond that.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus the sibling 'fetch_url' or 'browser_install'. It doesn't mention any prerequisites, limitations, or alternative scenarios. The agent must infer usage from the tool name and parameters alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

B3.2/5.0
Disambiguation4/5

The tools are mostly distinct, with 'fetch_url' and 'fetch_urls' clearly differentiated by single vs. multiple URLs. However, an agent might initially confuse 'fetch_url' and 'fetch_urls' as similar operations, though the descriptions clarify the distinction. 'browser_install' is completely separate in purpose.

Naming Consistency5/5

All tool names follow a consistent snake_case pattern with clear verb_noun structure: 'browser_install', 'fetch_url', and 'fetch_urls'. The naming is predictable and readable throughout the set.

Tool Count3/5

With only 3 tools, the server feels thin for a web fetching domain, lacking operations like navigation, interaction, or content parsing. While the core fetching is covered, the scope seems limited compared to typical web automation needs.

Completeness2/5

For a web fetching server, there are significant gaps: no tools for interacting with pages (e.g., clicking, filling forms), extracting specific content, handling JavaScript, or managing sessions. The surface is severely incomplete for common web automation tasks beyond basic URL retrieval.

Maintenance

ActivityInactive
ResponsivenessUnresponsive

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • F
    license
    Not graded
    quality
    Not graded
    maintenance
    An MCP server for web content extraction that converts HTML pages into clean, LLM-optimized Markdown using Mozilla's Readability. It supports batch processing, intelligent multi-page crawling, and configurable caching while respecting robots.txt standards.
    43
  • A
    license
    Not graded
    quality
    D
    maintenance
    An MCP server that scrapes content from web pages, including JavaScript-heavy sites, and converts it into high-quality Markdown. It leverages Playwright for headless browser automation and Pypandoc for clean content conversion.
    Apache 2.0
  • A
    license
    B
    quality
    B
    maintenance
    An MCP server for deep research that extracts clean text from web pages using hybrid extraction strategies and compiles findings into structured markdown reports. It provides tools for single and batch URL content extraction with optional Playwright support for JavaScript-heavy sites.
    4
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/jae-jae/fetcher-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server