Web Reader

Web pages cost agents more tokens than anything else they read, and most of those tokens are navigation, footers and page furniture. Web Reader returns the main content of a page or PDF as clean Markdown (the extraction behind Firefox's Reader View), and lets an agent read only what it needs:

• Ask the page a question: read_page with query ranks every passage of the page and returns the ones that answer it, with their heading path. A 435,000-character RFC answers in 2,000.

• Navigate long pages: an outline with section ids, one section at a time, or the page in pages (next_start).

• Search and count: exact text or a regular expression, every match with context and its section; links with their text, tracking parameters removed.

• PDFs are read as text page by page.

Pages are fetched once and cached for 10 minutes, so an outline, a search and a section read cost one download.

Safe to point anywhere. It reads the public web only: private, loopback, link-local (including cloud metadata at 169.254.169.254) and other reserved addresses are refused, checked on the address actually connected to and again on every redirect. It respects robots.txt (RFC 9309), identifies itself honestly in its User-Agent, and does not get around bot protection. No key needed.

Built and maintained by Arhan Canli.

Install

Needs Node.js 20 or newer. No account or key.

Claude Code

claude mcp add web-reader -- npx -y web-reader-mcp

Claude Desktop: download web-reader-mcp-<version>.mcpb from the latest release and open it. The bundle is signed; verify it with gh attestation verify <file> --repo arhancanli/web-reader-mcp.

Any other client (Windsurf, Zed, Cline, Continue and others), in its MCP config file:

{
  "mcpServers": {
    "web-reader": {
      "command": "npx",
      "args": [
        "-y",
        "web-reader-mcp"
      ]
    }
  }
}

Docker

docker build -t web-reader-mcp https://github.com/arhancanli/web-reader-mcp.git && docker run -i --rm web-reader-mcp

Hosted (Streamable HTTP): node src/server.mjs --http serves stateless MCP at POST /mcp (port from PORT, default 3000).

Related MCP server: Fetch MCP Server

Example

An agent calls read_page with:

{
  "url": "https://www.rfc-editor.org/rfc/rfc9110.html",
  "query": "Content-Location header field",
  "max_chars": 2000
}

and gets back (recorded from the live server on 2026-09-26):

{
  "url": "https://www.rfc-editor.org/rfc/rfc9110.html",
  "title": "RFC 9110: HTTP Semantics",
  "passages": "2 of 419 matching",
  "chars": 434823,
  "content": "[...]\n\n### 8. Representation Data and Metadata > 8.7. Content-Location\n\nThe \"Content-Location\" header field references a URI that can be used as an identifier for a specific resource corresponding to the representation in this message's content. In other words, if one were to perform a GET request on this URI at the time of this message's generation, then a 200 (OK) response would contain the same representation that is enclosed as content in this message.\n\n  Content-Location = absolute-URI / partial-URI\n\nThe field value is either an absolute-URI or a partial-URI. In the latter case (Section 4), the referenced URI is relative to the target URI ([URI], [Section 5](https://www.rfc-editor.org/rfc/rfc3986#section-5)).\n\nThe Content-Location value is not a replacement for the target URI (Section 7.1). It is representation metadata. It has the same syntax and semantics as the header field of the same name defined for MIME body parts in [Section 4](https://www.rfc-editor.org/rfc/rfc2557#section-4) of [RFC2557]. However, its appearance in an HTTP message has some special implications for HTTP recipients.\n\n[...]\n\nA user agent that sends Content-Location in a request message is stating that its value refers to where the user agent originally obtained the content of the enclosed representation (prior to any modifications made by that user agent). In other words, the user agent is providing a back link to the source of the original representation.\n\nAn origin server that receives a Content-Location field in a request message MUST treat the information as transitory request context rather than as metadata to be saved verbatim as part of the representation. An origin server MAY use that context to guide in processing the request or to save it for other uses, such as within source links or versioning metadata. However, an origin server MUST NOT use such context information to alter the request semantics."
}

Tools

How it behaves

• Read-only: only GET requests, to the URLs you give (and their robots.txt). Nothing is logged except unexpected failures (to stderr, without your inputs).

• Refused: non-http(s) URLs, URLs with credentials, and any address that is not public (checked at connect time and on each of at most 5 redirects). Bodies are capped at 15 MB after decompression; requests time out after 20 seconds.

• robots.txt is honoured for the reader's product token web-reader-mcp and *, longest match first, as RFC 9309 specifies. A site that refuses (a 403, a bot challenge) is reported as such.

• Options, as environment variables: WEB_READER_ALLOW_PRIVATE=1 to read local and intranet sites (for example a dev server on localhost), WEB_READER_IGNORE_ROBOTS=1 to skip robots.txt on your own responsibility.

• Results are compact JSON with a matching output schema. Long pages come with an outline and next_start; lists say how many items were left out.

How it reads a page

HTML goes through Mozilla Readability (main content) and Turndown with GitHub tables (Markdown). Images and citation markers are dropped, links made absolute. When a page is not an article (an index, a listing), the whole page without navigation is used. PDFs are read with unpdf (Mozilla pdf.js). JSON is pretty-printed; plain text and Markdown are returned as they are.

Benchmark

Measured 2026-09-26 with gpt-5.4-mini, 10 fixed tasks graded by fixed checks (bench/tasks.json, raw results in bench/results/).

Performance

Measured 2026-09-26 from Dubai, home connection against the live upstream, Node 24.19.0 (bench/perf.json, scripts/perf.mjs in the factory).

First call: a fresh server process, including the TLS connection and the upstream's own time. Repeat: the same call again, answered from the in-process cache, so it shows this server's own overhead.

Tool definitions the model reads on every turn (name, description, input schema): 2,290 characters, against 1,105 for mcp-server-fetch, the official reference fetch server. The full tool list, with the output schemas and annotations clients use to validate results, is 3,635 characters (1,104 for the alternative).

More MCP servers by Arhan Canli

• World Time: Time anywhere, DST-safe conversions, holidays for 200+ countries, business days and meeting slots.

• Actions Check: Checks GitHub Actions workflows: outdated actions, old Node runtimes, retired runners, injection.

• Citation Check: Verifies citations: finds fabricated or mismatched references and retractions, returns clean BibTeX.

• Config Check: Validates config files against their official schemas: tsconfig, compose, workflows, 1,400+ more.

• Contact Check: Validates and formats phone numbers, email addresses and postal addresses for any country.

• Cron Check: Explains cron expressions, lists next run times in any time zone, converts between cron dialects.

• Dockerfile Check: Checks Dockerfiles: build-breaking mistakes, base image tags that exist, digests, platforms, EOL.

• Domain Health: Email and domain checks: SPF lookup limits, DKIM keys, DMARC, DNS records, registration expiry.

• The whole collection, 12 more

License

MIT, Copyright (c) 2026 Arhan Canli.