fetch-url-raw

Short description: a stateless MCP server that exposes one tool, fetch_url_raw — a lightweight programmable HTTP client for LLM agents. Send arbitrary methods, headers, and bodies; control timeouts, redirects, response size, TLS verification, and DNS overrides. No cookies or session state between calls.

Requirements

• Python 3.12+

• Network access from the host that runs the server

Related MCP server: production-grade-mcp-agentic-system

Install

python3 -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e .

For tests:

pip install -e '.[dev]'

Deployment

The server speaks MCP over stdio. Deploy it as a local process that your MCP client launches; it does not open a public HTTP port by default.

1. Install into a venv (recommended)

cd /path/to/fetch-url-raw
python3 -m venv .venv
source .venv/bin/activate
pip install -e .

Confirm the entry point (stdio by default; waits on stdin for MCP):

fetch-url-raw
# Stop with Ctrl+C.

Or:

python -m fetch_url_raw

HTTP mode is off by default. To enable it, see section 3 below.

2. Wire into an MCP client

Point the client at the venv interpreter (or the fetch-url-raw script) so dependencies resolve correctly.

Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "fetch-url-raw": {
      "command": "/path/to/fetch-url-raw/.venv/bin/python",
      "args": ["-m", "fetch_url_raw"],
      "cwd": "/path/to/fetch-url-raw"
    }
  }
}

Generic MCP host / Cursor-style config:

{
  "mcpServers": {
    "fetch-url-raw": {
      "command": "/path/to/fetch-url-raw/.venv/bin/fetch-url-raw"
    }
  }
}

Codex / other stdio MCP runners: use the same command + args pattern; no extra env is required for basic use.

3. Optional HTTP server mode (not enabled by default)

Default transport is stdio. To expose MCP over HTTP instead, pass --transport explicitly and set listen address/port:

# Streamable HTTP (recommended HTTP transport)
fetch-url-raw --transport streamable-http --host 127.0.0.1 --port 8000

# SSE transport
fetch-url-raw --transport sse --host 127.0.0.1 --port 8000

# Bind all interfaces (trusted networks only)
fetch-url-raw --transport streamable-http --host 0.0.0.0 --port 9000 --allow-remote

Endpoints:

• streamable-http: http://<host>:<port>/mcp (or custom --path)

• sse: http://<host>:<port>/sse (messages under the default FastMCP message path)

Example MCP client config against a local HTTP server (client-specific; streamable HTTP):

{
  "mcpServers": {
    "fetch-url-raw": {
      "url": "http://127.0.0.1:8000/mcp"
    }
  }
}

HTTP mode is optional. Prefer stdio for desktop/local agent integrations. Only bind 0.0.0.0 or enable --allow-remote on trusted networks; the tool can initiate arbitrary outbound HTTP.

Private / local destination blocking

By default the server blocks connections by resolved destination IP (not by DNS name):

• DNS lookup is always allowed

• After resolve (or dns_override / literal IP URL), the TCP destination must not fall in private/local ranges

• Blocked by default: RFC1918 (10/8, 172.16/12, 192.168/16), loopback, link-local (169.254/16, fe80::/10), CGNAT (100.64/10), IPv6 ULA (fc00::/7)

• Always blocked: multicast / unspecified special-use ranges

• IPv4-mapped IPv6 addresses are checked as their IPv4 form

Opt in when you intentionally need LAN/metadata/loopback targets:

fetch-url-raw --allow-private-network
# with HTTP mode:
fetch-url-raw --transport streamable-http --host 127.0.0.1 --port 8000 --allow-private-network

Blocked attempts return:

{
  "success": false,
  "error": {
    "type": "DESTINATION_BLOCKED",
    "message": "destination IP 192.168.1.1 is private/local (...)"
  }
}

4. Operational notes

5. Optional: install from wheel

pip install dist/fetch_url_raw-0.1.0-py3-none-any.whl
fetch-url-raw

Usage

Once the MCP server is connected, call the fetch_url_raw tool from the client.

Parameters

Example tool calls

GET

{
  "url": "https://example.com/"
}

POST JSON (LLM-friendly: pass a JSON object directly; string body still works)

{
  "url": "https://httpbin.org/post",
  "method": "POST",
  "headers": {
    "Authorization": "Bearer token"
  },
  "body": {"hello": "world"},
  "timeout": 15
}

Raw string body (no auto Content-Type):

{
  "url": "https://httpbin.org/post",
  "method": "POST",
  "headers": {
    "Content-Type": "application/x-www-form-urlencoded"
  },
  "body": "hello=world&a=1"
}

No redirects + small body cap

{
  "url": "https://example.com/redirect",
  "follow_redirect": false,
  "max_response_bytes": 4096
}

DNS override (connect to 1.2.3.4 while keeping Host/SNI as api.example.com)

{
  "url": "https://api.example.com/health",
  "dns_override": {
    "api.example.com": "1.2.3.4"
  },
  "verify_tls": true
}

Inspect TLS cert only (no HTTP request)

{
  "url": "https://example.com/",
  "include_tls": true,
  "tls_only": true
}

Fetch and include TLS info

{
  "url": "https://example.com/",
  "include_tls": true,
  "max_response_bytes": 4096
}

Success response

{
  "success": true,
  "status": 200,
  "reason": "OK",
  "http_version": "HTTP/1.1",
  "headers": { "...": "..." },
  "body": "...",
  "body_json": null,
  "body_base64": null,
  "content_type": "text/html",
  "encoding": "utf-8",
  "elapsed_ms": 123,
  "redirected": false,
  "final_url": "https://example.com",
  "truncated": false,
  "received_bytes": 12345,
  "content_length": 12345
}

• Text-like content types (text/*, application/json, etc.) fill body (string still kept).

• body_json is set only when the body is valid JSON (object/array/etc.); otherwise null.

• Other non-text types set body to null and put Base64 data in body_base64.

• Internally the client may buffer up to 16 MiB so large text (e.g. JS bundles) can be decoded as text even when max_response_bytes is smaller. The tool result is then truncated to max_response_bytes.

• If the returned body is truncated, truncated is true and only the first N bytes (or text whose UTF-8 size is N) are returned.

• received_bytes is how many body bytes were actually returned to the LLM (after truncation).

• Optional tls object (when include_tls or on many TLS_ERRORs): version, cipher, ALPN, SNI, peer IP, leaf cert PEM/fingerprint/SAN/dates, and chain.

• content_length is the full response body size when known: from the Content-Length header if present, otherwise the full size if the body fit in the 16 MiB prefetch buffer, otherwise null if the stream was cut at the prefetch ceiling without a header. Use this so agents know the real size (e.g. 1 MiB JS) while reading only the first 64 KiB of text.

Error response

Failures return a structured object instead of raising:

{
  "success": false,
  "error": {
    "type": "TIMEOUT",
    "message": "Operation timed out"
  }
}

Example message values:

• TIMEOUT: Connection timed out while establishing TCP/TLS, Timed out while reading the response

• CONNECT_ERROR: Connection refused (no service listening or port closed), Connection reset by peer (TCP RST during connect or request), Network is unreachable (no route to destination network), No route to host (destination host unreachable)

• TLS_ERROR: TLS certificate has expired, TLS certificate is self-signed and not trusted (...), TLS certificate hostname mismatch: ... (wrong CNAME/SAN), TLS certificate chain incomplete or untrusted: unable to get local issuer certificate (...), TLS protocol version mismatch or unsupported/outdated TLS (...), TLS handshake failed: malformed or unexpected TLS data (...)

Features

• Methods: GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS, TRACE

• Custom headers and body (raw string or JSON value for LLM-friendly POSTs)

• Timeout, redirect control, response size limit

• DNS override (SNI and Host header preserved)

• TLS verification toggle

• Stateless: no cookies, session cache, or filesystem writes

• Default block of private/local destination IPs (post-resolve); opt-in with --allow-private-network

• Structured errors suitable for LLM tool loops

Development

pip install -e '.[dev]'
pytest
FETCH_URL_RAW_LIVE=1 pytest tests/test_live.py   # optional live network tests

Design

See design.md for architecture, DNS override details, body decoding rules, and security considerations.

License

MIT