MCP Web Fetch Server

An MCP (Model Context Protocol) server that provides web fetching capabilities over HTTP/SSE. This server is designed to work with ChatGPT and other AI assistants that support MCP over HTTP.

Features

• Fetch content from any URL

• Support for different HTTP methods (GET, POST, PUT, DELETE, PATCH)

• Custom headers support

• Adaptive chunking for large responses to avoid MCP payload limits

• Request body support for POST/PUT/PATCH

• Dockerized for easy deployment

• External access via port 8080

Related MCP server: Fetch MCP Server

Quick Start

Using Docker Compose (Recommended)

docker-compose up -d

Using Docker

# Build the image
docker build -t mcp-web-fetch .

# Run the container
docker run -d -p 8080:8080 --name mcp-web-fetch mcp-web-fetch

Local Development

# Install dependencies
npm install

# Start the server
npm start

Endpoints

ChatGPT Actions Endpoints

• GET /openapi.json - OpenAPI 3.1 schema for ChatGPT Actions

• POST /fetch - Fetch content from a URL (simplified endpoint for Actions)

MCP Protocol Endpoints

• GET /.well-known/mcp.json - MCP manifest

• GET /tools/list - List available tools

• POST /tools/call - Execute a tool

Legacy SSE Endpoints

• GET /sse - SSE endpoint for MCP communication

• POST /messages - Message endpoint for MCP

Utility Endpoints

• GET / - Server information

• GET /health - Health check endpoint

Using with ChatGPT

Option 1: ChatGPT Actions (Recommended)

To use this server with ChatGPT Actions:

1. Deploy the server and ensure it's accessible externally on port 8080

2. In ChatGPT, go to your GPT settings and create a new Action

3. Import the OpenAPI schema:

   • Click "Import from URL"

   • Enter: https://your-host:8080/openapi.json

4. The fetchUrl action will now be available to ChatGPT

Option 2: MCP Protocol

To use this server with MCP-compatible clients:

1. Deploy the server and ensure it's accessible externally on port 8080

2. Configure your MCP client:

   • Server URL: https://your-host:8080

   • The server will be auto-discovered via the /.well-known/mcp.json manifest

Available Tools

fetch_url

Fetches content from a URL and returns the response.

Parameters:

• url (required): The URL to fetch

• method (optional): HTTP method (GET, POST, PUT, DELETE, PATCH). Default: GET

• headers (optional): Object containing HTTP headers

• body (optional): Request body for POST/PUT/PATCH requests

Example:

{
  "url": "https://api.example.com/data",
  "method": "GET",
  "headers": {
    "Authorization": "Bearer token123"
  }
}

When the fetched body exceeds the configured chunk size, MCP clients receive metadata plus additional chunk N/M text messages containing the body segments. The /fetch HTTP endpoint mirrors this behavior by returning bodyChunks, bodyChunkSize, and bodyChunkCount alongside the aggregate bodyLength.

Configuration

The server can be configured using environment variables:

• PORT: Port to run the server on (default: 8080)

• SERVER_URL: Public HTTPS URL for your server (used in OpenAPI schema). Required for ChatGPT Actions. Example: https://mcp.36technology.com

• MCP_BODY_CHUNK_SIZE: Maximum number of characters per body chunk when returning large payloads (default: 60000)

Setting Environment Variables

For Docker deployment, create a .env file:

cp .env.example .env
# Edit .env and set SERVER_URL to your public HTTPS URL

Or set it directly in docker-compose:

SERVER_URL=https://your-domain.com docker-compose up -d

Health Check

Check if the server is running:

curl http://localhost:8080/health

Testing

Test ChatGPT Actions Endpoints

Get the OpenAPI schema:

curl http://localhost:8080/openapi.json

Test the fetch endpoint:

curl -X POST http://localhost:8080/fetch \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://api.github.com/zen"
  }'

Test MCP Protocol Endpoints

Test the MCP manifest:

curl http://localhost:8080/.well-known/mcp.json

List available tools:

curl http://localhost:8080/tools/list

Call the fetch_url tool:

curl -X POST http://localhost:8080/tools/call \
  -H "Content-Type: application/json" \
  -d '{
    "name": "fetch_url",
    "arguments": {
      "url": "https://api.github.com/zen"
    }
  }'

Security Considerations

• This server can fetch any URL, so use appropriate network restrictions

• Consider adding authentication if exposing publicly

• Be mindful of rate limiting on external services

• Use HTTPS in production environments

License

MIT