# mcp-fetch-server

Standalone MCP network server exposing a single `fetch_url` tool. It fetches a URL
and returns the page's main content as stripped plain text — the same behavior as
Open WebUI's built-in `fetch_url`, as a self-hosted service any MCP client can use
over the network (streamable HTTP).

## Features

- One tool: `fetch_url(url)` — HTML is reduced to main text (trafilatura, with a
  BeautifulSoup fallback); `text/*` responses are returned as-is.
- Proxy support: all fetch traffic can be routed through an HTTP or SOCKS5 proxy.
- Configurable request timeout and content-length truncation.
- Environment-variable-only configuration (no config files).
- SSRF guard: blocks private, loopback, link-local, and reserved target addresses
  by default, including on redirects.
- Docker deployment, non-root.

## Configuration

All configuration is read from environment variables at startup. Invalid values
fail the server at startup with an error naming the variable.

| Variable | Meaning | Default |
|---|---|---|
| `HOST` | Address to bind | `0.0.0.0` |
| `PORT` | Port to bind | `8080` |
| `FETCH_PROXY` | Proxy URL: `http://host:port`, `socks5://host:port`, or `socks5h://host:port` (scheme selects the proxy type) | unset = direct connection |
| `FETCH_TIMEOUT` | Per-request timeout in seconds | `30` |
| `FETCH_MAX_CONTENT_LENGTH` | Extracted text is truncated beyond this many characters (a truncation marker is appended) | `15000` |
| `FETCH_ALLOW_PRIVATE` | `1` disables the SSRF guard; `0` keeps it enabled | `0` |
| `FETCH_USER_AGENT` | `User-Agent` header sent to fetched sites | `mcp-fetch-server/0.1.0` |

## Docker

```sh
docker build -t mcp-fetch-server .
```

Direct connection:

```sh
docker run -d --name mcp-fetch-server -p 8080:8080 mcp-fetch-server
```

Via an HTTP proxy:

```sh
docker run -d --name mcp-fetch-server -p 8080:8080 \
  -e FETCH_PROXY='http://user:pass@proxy.example:8888' \
  mcp-fetch-server
```

Via a SOCKS5 proxy (use `socks5h://` to resolve DNS through the proxy):

```sh
docker run -d --name mcp-fetch-server -p 8080:8080 \
  -e FETCH_PROXY='socks5h://proxy.example:1080' \
  mcp-fetch-server
```

The container runs as the non-root `nonroot` user, exposes port 8080, and carries
a `HEALTHCHECK` that verifies the MCP endpoint responds. The MCP endpoint path is
`/mcp`, e.g. `http://<host>:8080/mcp`.

Point any MCP client at the endpoint, e.g. (Open WebUI / generic streamable HTTP):

```json
{ "url": "http://<host>:8080/mcp" }
```

## Running without Docker

```sh
python3 -m venv .venv
.venv/bin/pip install -e .
.venv/bin/mcp-fetch-server
```

## SSRF guard

The server is network-exposed and the LLM chooses the URLs, so by default it
refuses to fetch addresses that are loopback, private (RFC 1918, `fc00::/7`),
link-local (including the cloud metadata address `169.254.169.254`), or otherwise
reserved. The check applies to the resolved IP and is re-run on every redirect hop,
and direct connections are pinned to the resolved address to close the
DNS-rebinding gap.

**Warning:** set `FETCH_ALLOW_PRIVATE=1` only when the deployment is behind a
proxy or network boundary that already isolates internal addresses. With the
guard disabled and a permissive proxy, the tool can be used to reach internal
services.

## Development

```sh
.venv/bin/pip install -e ".[dev]"
.venv/bin/python -m pytest
.venv/bin/ruff check .
.venv/bin/mypy mcp_fetch_server
```