中文

mcp-server-requests

An MCP server that provides HTTP request capabilities, enabling LLMs to fetch and process web content.

Features

• fetch: Web content fetching tool

  • Supports converting web content to Markdown format

  • Supports filtering web content (non-visual elements like script, style, meta)

  • Feeds into LLM context, reducing token consumption through Markdown conversion or filtering non-visual elements

• fetch_to_file: Web content saving tool

  • Web content is not fed into LLM context, avoiding excessive token consumption

• http_request: Generic HTTP request tool

  • Full HTTP method support (GET, POST, PUT, PATCH, DELETE)

  • Supports custom request headers

  • Supports text and JSON request bodies

  • Returns complete HTTP response information (status, headers, body) for LLMs

• Supports custom User-Agent

• Supports random User-Agent generation

• Supports using MCP Root capability, controlled by --use-root, usable with MCP Clients that have root capability

Related MCP server: cURL MCP Server

Installation

git clone https://github.com/coucya/mcp-server-requests.git
cd mcp-server-requests
pip install .

Usage

MCP Server Configuration

{
    "mcpServers": {
        "mcp-server-requests": {
            "command": "python",
            "args": [
                "-m",
                "mcp_server_requests"
            ]
        }
    }
}

Command Line

0. Start MCP Server

Start the MCP server directly:

python -m mcp_server_requests

Options

• --user-agent TEXT: Specify custom User-Agent string

• --random-user-agent [browser=xxx;os=xxx]: Use randomly generated User-Agent

• --force-user-agent: Force using command line specified User-Agent, ignoring LLM provided UA

• --list-os-and-browser: List available browsers and OS for random User-Agent generation

• --use-root: Enable workspace root support for file operations (requires MCP Client with root capability support)

• --allow-external-file-access: Allow file operations outside workspace (only with --use-root)

Option Details

• --user-agent and --random-user-agent are mutually exclusive and cannot be used together

• User-Agent setup methods:

  • Custom string: --user-agent "Mozilla/5.0 (...)"

  • Fully random: --random-user-agent

  • Conditional random generation:

    • Specify browser type: --random-user-agent browser=chrome

    • Specify OS: --random-user-agent os=windows

    • Both browser and OS: --random-user-agent browser=chrome;os=windows

    • Note: Browser and OS parameters are case insensitive

• Use --list-os-and-browser to view available browsers and OS for --random-user-agent.

• --force-user-agent option controls User-Agent priority:

  • When --force-user-agent is enabled: Force using command-line specified User-Agent (via --user-agent or --random-user-agent)

  • When --force-user-agent is disabled:

    • If LLM provides User-Agent in headers, use the LLM-provided one

    • Otherwise use command-line specified User-Agent

  • If none specified, use default User-Agent: Mozilla/5.0 (compatible; mcp-server-requests/{version})

• --use-root enables file operations relative to workspace root:

  • Prerequisite: Requires MCP Client with root capability support

  • When enabled: fetch_to_file allows absolute paths or relative paths, relative paths are relative to workspace root

  • When disabled: fetch_to_file must use absolute paths, files are saved to specified absolute path locations

• --allow-external-file-access: Controls whether file operations outside workspace are allowed (only usable with --use-root)

  • Prerequisite: Requires --use-root to be enabled, this option has no effect if --use-root is not enabled

  • When enabled: fetch_to_file can operate on files outside the workspace directory

  • When disabled: fetch_to_file can only operate on files within the workspace directory, even if absolute paths are provided, files outside the workspace cannot be operated on

1. fetch - Fetch Web Content

The fetch subcommand is equivalent to the MCP fetch tool functionality, used to demonstrate fetch capabilities.

python -m mcp_server_requests fetch <URL> [--return-content {raw,basic_clean,strict_clean,markdown}]

Options:

• --return-content: Return content type (default: markdown)

  • raw: Return raw unprocessed HTML content (includes response headers)

  • basic_clean: Basic cleanup, removing non-display tags (script, style, meta, etc.) while preserving structure

  • strict_clean: Strict cleanup, removing non-display tags and most HTML attributes, keeping only essential structure

  • markdown: Convert HTML to clean Markdown format

Functionality

MCP Tool Details

1. fetch - Fetch Web Content

Fetch content from the specified URL and return it in the specified format.

Parameters:

• url (required, string): Target URL, supports HTTP/HTTPS protocols

• return_content (optional, enum): Content processing format, defaults to "markdown"

  • "raw": Return raw HTML content with complete HTTP response headers

  • "basic_clean": Basic cleanup, removing non-display tags (script, style, meta, etc.) while preserving HTML structure

  • "strict_clean": Strict cleanup, removing non-display tags and most HTML attributes, keeping only essential structure

  • "markdown": Convert HTML content to clean Markdown format

Returns: Content processed according to the specified format

2. fetch_to_file - Fetch Web Content and Save to File

Fetch content from the specified URL and save it to a file.

Parameters:

• url (required, string): Target URL

• file_path (required, string): File save path

  • When --use-root is enabled: Must be a relative path (relative to workspace root)

  • When --use-root is disabled: Must be an absolute path

• return_content (optional, enum): Content processing format, defaults to "markdown", same options as fetch tool

Features:

• Automatically creates parent directories for nested paths

• All files saved with UTF-8 encoding

• When --use-root is enabled, can work with --allow-external-file-access to control file access scope

Returns: Operation result or error message

3. http_request - Generic HTTP Request Tool

Send HTTP requests using any method and get complete responses.

Parameters:

• url (required, string): Target URL

• method (optional, enum): HTTP method, defaults to "GET"

  • "GET", "POST", "PUT", "PATCH", "DELETE"

• query (optional, object): URL query parameters as key-value pairs, automatically URL-encoded

• headers (optional, object): Custom HTTP request headers

• data (optional, string): Text format request body data, mutually exclusive with json parameter

• json (optional, any type): JSON format request body data, mutually exclusive with data parameter

Parameter Constraints:

• data and json parameters cannot be used together

• When using json parameter, Content-Type: application/json is automatically set

• When using data parameter, it's recommended to manually set appropriate Content-Type

Returns: Complete HTTP response (status line, response headers, and response body)

Note: Individual HTTP method tools (http_get, http_post, http_put, http_patch, http_delete) have been removed and replaced by the unified http_request tool.

License

MIT