[中文](README-zh.md)

-----

# mcp-server-requests

An MCP server that provides HTTP request capabilities, enabling LLMs to fetch and process web content.

## Features

- **fetch**: Web content fetching tool
  - Supports converting web content to Markdown format
  - Supports filtering web content (non-visual elements like script, style, meta)
  - Feeds into LLM context, reducing token consumption through Markdown conversion or filtering non-visual elements
- **fetch_to_file**: Web content saving tool
  - Web content is not fed into LLM context, avoiding excessive token consumption
- **http_request**: Generic HTTP request tool
  - Full HTTP method support (GET, POST, PUT, PATCH, DELETE)
  - Supports custom request headers
  - Supports text and JSON request bodies
  - Returns complete HTTP response information (status, headers, body) for LLMs
- Supports custom User-Agent
- Supports random User-Agent generation
- Supports using MCP Root capability, controlled by `--use-root`, usable with MCP Clients that have `root` capability

## Installation

```bash
git clone https://github.com/coucya/mcp-server-requests.git
cd mcp-server-requests
pip install .
```

## Usage

### MCP Server Configuration

```json
{
    "mcpServers": {
        "mcp-server-requests": {
            "command": "python",
            "args": [
                "-m",
                "mcp_server_requests"
            ]
        }
    }
}
```

### Command Line

### 0. **Start MCP Server**

Start the MCP server directly:

```bash
python -m mcp_server_requests
```

#### Options
- `--user-agent TEXT`: Specify custom User-Agent string
- `--random-user-agent [browser=xxx;os=xxx]`: Use randomly generated User-Agent
- `--force-user-agent`: Force using command line specified User-Agent, ignoring LLM provided UA
- `--list-os-and-browser`: List available browsers and OS for random User-Agent generation
- `--use-root`: Enable workspace root support for file operations (requires MCP Client with `root` capability support)
- `--allow-external-file-access`: Allow file operations outside workspace (only with --use-root)

#### Option Details
- `--user-agent` and `--random-user-agent` are mutually exclusive and cannot be used together
- User-Agent setup methods:
  - Custom string: `--user-agent "Mozilla/5.0 (...)"`
  - Fully random: `--random-user-agent`
  - Conditional random generation:
    - Specify browser type: `--random-user-agent browser=chrome`
    - Specify OS: `--random-user-agent os=windows`
    - Both browser and OS: `--random-user-agent browser=chrome;os=windows`
    - Note: Browser and OS parameters are case insensitive

- Use `--list-os-and-browser` to view available browsers and OS for `--random-user-agent`.

- `--force-user-agent` option controls User-Agent priority:
  - When `--force-user-agent` is enabled: Force using command-line specified User-Agent (via `--user-agent` or `--random-user-agent`)
  - When `--force-user-agent` is disabled:
    - If LLM provides User-Agent in `headers`, use the LLM-provided one
    - Otherwise use command-line specified User-Agent
  - If none specified, use default User-Agent: `Mozilla/5.0 (compatible; mcp-server-requests/{version})`

- `--use-root` enables file operations relative to workspace root:
  - **Prerequisite**: Requires MCP Client with `root` capability support
  - When enabled: `fetch_to_file` allows absolute paths or relative paths, relative paths are relative to workspace root
  - When disabled: `fetch_to_file` must use absolute paths, files are saved to specified absolute path locations
- `--allow-external-file-access`: Controls whether file operations outside workspace are allowed (only usable with --use-root)
  - **Prerequisite**: Requires `--use-root` to be enabled, this option has no effect if `--use-root` is not enabled
  - When enabled: `fetch_to_file` can operate on files outside the workspace directory
  - When disabled: `fetch_to_file` can only operate on files within the workspace directory, even if absolute paths are provided, files outside the workspace cannot be operated on

---

### 1. **fetch - Fetch Web Content**

The fetch subcommand is equivalent to the MCP `fetch` tool functionality, used to demonstrate fetch capabilities.

```bash
python -m mcp_server_requests fetch <URL> [--return-content {raw,basic_clean,strict_clean,markdown}]
```

**Options:**
- `--return-content`: Return content type (default: markdown)
  - **raw**: Return raw unprocessed HTML content (includes response headers)
  - **basic_clean**: Basic cleanup, removing non-display tags (script, style, meta, etc.) while preserving structure
  - **strict_clean**: Strict cleanup, removing non-display tags and most HTML attributes, keeping only essential structure
  - **markdown**: Convert HTML to clean Markdown format

---

## Functionality

### MCP Tool Details

#### 1. **fetch** - Fetch Web Content
Fetch content from the specified URL and return it in the specified format.

**Parameters:**
- **url** (required, string): Target URL, supports HTTP/HTTPS protocols
- **return_content** (optional, enum): Content processing format, defaults to "markdown"
  - `"raw"`: Return raw HTML content with complete HTTP response headers
  - `"basic_clean"`: Basic cleanup, removing non-display tags (script, style, meta, etc.) while preserving HTML structure
  - `"strict_clean"`: Strict cleanup, removing non-display tags and most HTML attributes, keeping only essential structure
  - `"markdown"`: Convert HTML content to clean Markdown format

**Returns:** Content processed according to the specified format

---

#### 2. **fetch_to_file** - Fetch Web Content and Save to File
Fetch content from the specified URL and save it to a file.

**Parameters:**
- **url** (required, string): Target URL
- **file_path** (required, string): File save path
  - When `--use-root` is enabled: Must be a **relative path** (relative to workspace root)
  - When `--use-root` is disabled: Must be an **absolute path**
- **return_content** (optional, enum): Content processing format, defaults to "markdown", same options as fetch tool

**Features:**
- Automatically creates parent directories for nested paths
- All files saved with UTF-8 encoding
- When `--use-root` is enabled, can work with `--allow-external-file-access` to control file access scope

**Returns:** Operation result or error message

---

#### 3. **http_request** - Generic HTTP Request Tool
Send HTTP requests using any method and get complete responses.

**Parameters:**
- **url** (required, string): Target URL
- **method** (optional, enum): HTTP method, defaults to "GET"
  - `"GET"`, `"POST"`, `"PUT"`, `"PATCH"`, `"DELETE"`
- **query** (optional, object): URL query parameters as key-value pairs, automatically URL-encoded
- **headers** (optional, object): Custom HTTP request headers
- **data** (optional, string): Text format request body data, mutually exclusive with json parameter
- **json** (optional, any type): JSON format request body data, mutually exclusive with data parameter

**Parameter Constraints:**
- `data` and `json` parameters cannot be used together
- When using `json` parameter, `Content-Type: application/json` is automatically set
- When using `data` parameter, it's recommended to manually set appropriate `Content-Type`

**Returns:** Complete HTTP response (status line, response headers, and response body)

---

**Note**: Individual HTTP method tools (http_get, http_post, http_put, http_patch, http_delete) have been removed and replaced by the unified **http_request** tool.

## License
MIT