local-web-mcp

Version 0.3.0 · AGPL-3.0

This tool is developed with support from AI, but it has been evaluated by a human before upload.

An MCP server that gives Claude a fallback web fetcher running on your own machine. When the built-in fetcher is blocked, this one tries from your IP, your connection, and optionally your logged-in session.

One tool: fetch_url_locally.

The problem it solves

Hosted fetchers run from datacentre IP ranges, which a lot of sites refuse outright. A local stdio MCP server does not: the client spawns the process on your machine, so requests originate from your ordinary connection.

Verified working on sites the hosted fetcher could not read, including CNBC, Politiken and Bloomberg.

Related MCP server: auth-fetch-mcp

What it does not change

Worth stating plainly, because the framing invites overclaiming.

• Model inference still runs on Anthropic's servers.

• Fetched page text is still sent to the model as tool output. Anything read through an authenticated session goes with it.

• It changes who talks to the website, not where the model runs.

The hard boundary

Can the page be read without executing JavaScript?

If yes, this handles it. If no, nothing at the header, TLS or cookie layer will get there, and no amount of User-Agent tuning helps. Sites using DataDome, Cloudflare Turnstile, PerimeterX and similar serve a challenge that must be executed to pass. For those, use a browser-based tool or open the page yourself.

This is a boundary to know, not a bug to fix.

Verdicts

Every response is labelled. The dangerous case is not an obvious 403, it is a 200 that is not the page: a bot interstitial, a consent wall, an empty client-rendered shell. Those get summarised as though they were the article.

Anything other than ok attaches an explicit warning telling the model not to treat the body as content.

Body evidence outranks the status code. A publisher that serves a paywalled article as HTTP 403 is reported as login_required, not blocked, because the useful advice there is to export cookies rather than to reach for a browser.

Detection matches on vendor infrastructure rather than wording: challenge pages are localised, so captcha-delivery.com is a reliable signal where "verifying your device" is not. Vendor cookies in response headers count as evidence when paired with a refusal status.

The tool

fetch_url_locally(url, max_chars=20000, use_session=true)

The reply is a short header followed by the extracted text:

URL: https://example.com/article
Status: 200
Verdict: ok
Title: The headline
Session: authenticated (the user's cookies were sent)

The article text...

Session: appears only when cookies were actually loaded and scoped to that host. A verdict other than ok adds a WARNING: line telling the model not to treat the body as content.

Failures come back as an error naming the cause, for example [blocked_host] Host '192.168.1.5' resolves to a private or internal address. Codes: invalid_url, blocked_scheme, blocked_host, dns_failure, timeout, too_many_redirects, upstream_error, invalid_argument.

Requirements

• Python 3.11+ (uses asyncio.timeout). With uv you do not need to install it yourself; uv fetches a suitable interpreter

• Claude Desktop or Claude Code. Stdio servers do not work in the browser or the mobile app, which require a publicly reachable HTTPS endpoint.

Install

With uv, which fetches a suitable Python and the dependencies itself:

uv --directory /absolute/path/to/local-web-mcp run local-web-mcp

Run that once in a terminal before registering the server. The first run resolves and builds the environment, which takes long enough that a client launching it cold may give up and report the server as failed even though the configuration is correct.

It prints Starting local-web and then waits for a client on stdin, which is what a working server looks like. Once you see that line, the environment is built: press Ctrl+C and carry on.

Or with a plain virtual environment:

python3 -m venv .venv
.venv/bin/pip install -r requirements.txt

Register with Claude Desktop

Add to claude_desktop_config.json (macOS: ~/Library/Application Support/Claude/, Linux: ~/.config/Claude/):

{
  "mcpServers": {
    "local-web": {
      "command": "uv",
      "args": [
        "--directory",
        "/absolute/path/to/local-web-mcp",
        "run",
        "local-web-mcp"
      ]
    }
  }
}

If uv is not on the PATH that the client sees, use its absolute path as command. The virtual-environment equivalent is:

{
  "mcpServers": {
    "local-web": {
      "command": "/absolute/path/to/.venv/bin/python",
      "args": ["/absolute/path/to/local_web_mcp.py"]
    }
  }
}

Claude Code:

claude mcp add local-web -- uv --directory /absolute/path/to/local-web-mcp run local-web-mcp

Configuration

All optional. Every setting has a working default.

The default Accept-Language asks for Danish first. That is a deliberate default rather than a neutral one, so set LOCALWEB_ACCEPT_LANGUAGE if you want English or anything else:

"env": { "LOCALWEB_ACCEPT_LANGUAGE": "en-GB,en;q=0.9" }

Session cookies

Point LOCALWEB_COOKIE_FILE at an export from a browser extension. The loader accepts Netscape cookies.txt with any header or none, and JSON exports from the common extensions. See cookies.txt.example.

Always set LOCALWEB_COOKIE_DOMAINS. Without it the whole jar is live and any fetch can carry an unrelated session. With it, every cookie outside those domains is dropped at load time, before the jar reaches a client. Matching covers subdomains (example.com covers www.example.com) but not lookalikes (evil-example.com).

Three things to keep in mind:

1. Content read through your session is sent to the model as tool output.

2. A cookie file is a credential file. chmod 600, and it is gitignored here.

3. Cookies expire. A previously working source returning login_required is usually a stale export rather than lost access.

Pass use_session=false to check whether a page is genuinely public.

Reaching a local network

Private addresses are blocked by default, and deliberately so. The model chooses URLs partly from text it has just read, so a fetched page can attempt to steer it at 192.168.1.1. The blocklist means a prompt injection in page content cannot turn this into a network scanner.

Prefer a narrow allowlist over the blanket switch:

"env": { "LOCALWEB_ALLOWLIST": "nas.local,nas" }

Security

• Scheme restricted to http and https; URLs carrying credentials rejected

• Hostname resolved and every returned address validated before the request

• IPv6-mapped IPv4 (::ffff:10.0.0.1) unwrapped before checking

• Cloud metadata endpoints blocked unconditionally: the hostname is resolved before both the allowlist and ALLOW_PRIVATE, so neither can open them

• Redirects and meta refreshes followed manually, revalidated at each hop

• Cookie counts logged, never names, values or domains

Known limitations

• No JavaScript. The hard boundary above.

• DNS rebinding. The hostname is resolved for validation, then resolved again on connect. Closing that gap needs the connection pinned to the validated IP. Worth doing before enabling ALLOW_PRIVATE on an untrusted network.

• Challenge detection is signature-based and will drift as vendors change their markup. Run inspect_response.py if a verdict looks wrong.

• HTML-to-text extraction is dependency-free and crude. trafilatura would be markedly better for serious article extraction.

Files

Diagnosing a wrong verdict

uv run python inspect_response.py https://example.com/article

Prints the status, the interesting headers, the start of the body, and then runs the server's own detection against it. Always check this before changing detection logic. A browser and this fetcher are frequently served entirely different responses, so what you see on screen is not evidence about what the tool received.

Tests

uv run python unit_test.py
uv run python smoke_test.py

unit_test.py is offline and exits non-zero on failure, so it works as a pre-commit gate. smoke_test.py drives the server over stdio and makes one live request, so it needs a network connection. Run both from the repository root.

On the plain-venv route, substitute .venv/bin/python for uv run python.

Licence

Copyright (C) 2026 David Lindholm.

GNU Affero General Public License v3.0 or later. See LICENSE.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.