A
securityA
licenseA
qualityThis server enables LLMs to retrieve and process content from web pages, converting HTML to markdown for easier consumption.
Last updated -
1
70,826
MIT License
Enables conversion of web pages to Markdown format, allowing for cleaner and more readable text representation of online content.
mcp-server-requests
An MCP server that provides HTTP request capabilities, enabling LLMs to fetch and process web content.
Features
Supports converting web content to Markdown format
Supports filtering out content useless for LLMs
Supports custom User-Agent headers
Supports random User-Agent headers
Supports custom request headers in HTTP requests
Supports full HTTP methods (GET, POST, PUT, DELETE, PATCH)
LLMs can access complete HTTP response header information
Installation
git clone https://github.com/coucya/mcp-server-requests.git
cd mcp-server-requests
pip install .
Usage
MCP Server Configuration
{
"mcpServers": {
"mcp-server-requests": {
"command": "python",
"args": [
"-m",
"mcp_server_requests"
]
}
}
}
Command Line
0. Start MCP Server
Start the MCP server directly:
python -m mcp_server_requests
Options
--user-agent TEXT: Specify custom User-Agent string--random-user-agent [browser=xxx;os=xxx]: Use randomly generated User-Agent--force-user-agent: Force using command line specified User-Agent, ignoring LLM provided UA--list-os-and-browser: List available browsers and OS for random User-Agent generation
Option Details
--user-agentand--random-user-agentare mutually exclusive and cannot be used togetherUser-Agent setup methods:
Custom string:
--user-agent "Mozilla/5.0 (...)"Fully random:
--random-user-agentConditional random generation:
Specify browser type:
--random-user-agent browser=chromeSpecify OS:
--random-user-agent os=windowsBoth browser and OS:
--random-user-agent browser=chrome;os=windowsNote: Browser and OS parameters are case insensitive
Use
--list-os-and-browserto view available browsers and OS for--random-user-agent.--force-user-agentcontrols User-Agent priority:When enabled: Prioritize command line specified User-Agent (via
--user-agentor--random-user-agent), ignoring LLM provided UAWhen disabled:
If LLM provides User-Agent, use that
Otherwise use command line specified User-Agent
1. fetch - Fetch Web Content
The fetch subcommand is equivalent to the fetch tool functionality, demonstrating fetch capabilities.
python -m mcp_server_requests fetch <URL> [--return-content {raw,basic_clean,strict_clean,markdown}]
Options:
--return-content: Return content type (default: markdown)raw: Return raw unprocessed HTML content
basic_clean: Basic cleanup, removing non-display tags like script, style
strict_clean: Strict cleanup, removing non-display tags and most HTML attributes
markdown: Convert HTML to clean Markdown format
Example:
python -m mcp_server_requests fetch https://example.com
2. get - Execute HTTP GET Request
The get subcommand is equivalent to the http_get tool functionality, demonstrating http_get capabilities.
python -m mcp_server_requests get <URL> [--headers HEADERS]
Options:
--headers: Custom request headers (format: "key1=value1;key2=value2")
3. post - Execute HTTP POST Request
The post subcommand is equivalent to the http_post tool functionality, demonstrating http_post capabilities.
python -m mcp_server_requests post <URL> [--headers HEADERS] [--data TEXT]
Options:
--headers: Custom request headers--data: Request body data
4. put - Execute HTTP PUT Request
The put subcommand is equivalent to the http_put tool functionality, demonstrating http_put capabilities.
python -m mcp_server_requests put <URL> [--headers HEADERS] [--data TEXT]
Options: Same as POST method
5. delete - Execute HTTP DELETE Request
The delete subcommand is equivalent to the http_delete tool functionality, demonstrating http_delete capabilities.
python -m mcp_server_requests delete <URL> [--headers HEADERS] [--data TEXT]
Options: Same as POST method
Functionality
Available Tools
fetch - Fetch web content
Parameters:
url (required): Target URL
return_content (optional): Return content type ('raw', 'basic_clean', 'strict_clean', 'markdown')
raw: Return raw HTML content
basic_clean: Return filtered HTML content, removing non-display tags like script, style
strict_clean: Return filtered HTML content, removing non-display tags and most useless HTML attributes
markdown: Return HTML converted to Markdown
http_get - Execute HTTP GET request
Parameters:
url (required): Target URL
query (optional): Query parameter key-value pairs
headers (optional): Custom request headers
LLM may specify User-Agent in headers, whether to use it is controlled by
--force-user-agent(same applies to other tools)
http_post - Execute HTTP POST request
Parameters:
url (required): Target URL
query (optional): Query parameter key-value pairs
headers (optional): Custom request headers
data (optional): Request body data (text)
json (optional): Request body data (JSON)
data and json cannot be used together
http_put - Execute HTTP PUT request
Parameters: Same as http_post
http_patch - Execute HTTP PATCH request
Parameters: Same as http_post
http_delete - Execute HTTP DELETE request
Parameters: Same as http_post
License
MIT
remote-capable server
The server can be hosted and run remotely because it primarily relies on remote services or has no dependency on the local environment.
Web Content Retrieval (full webpage, filtered content, or Markdown-converted), Custom User-Agent, Multi-HTTP Method Support (GET/POST/PUT/DELETE/PATCH), LLM-Controlled Request Headers, LLM-Accessible Response Headers, and more.
Related MCP Servers
- AsecurityAlicenseAqualityProvides functionality to fetch web content in various formats, including HTML, JSON, plain text, and Markdown.Last updated -4376610MIT License
- AsecurityFlicenseAqualityProvides functionality to fetch web content in various formats, including HTML, JSON, plain text, and Markdown.Last updated -4104,1622
- AsecurityFlicenseAqualityProvides functionality to fetch web content in various formats, including HTML, JSON, plain text, and Markdown with support for custom headers.Last updated -4104,1623