web-reader-mcp
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@web-reader-mcpRead this page and summarize the main points: https://en.wikipedia.org/wiki/Web_scraping"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Web Reader
Web pages cost agents more tokens than anything else they read, and most of those tokens are navigation, footers and page furniture. Web Reader returns the main content of a page or PDF as clean Markdown (the extraction behind Firefox's Reader View), and lets an agent read only what it needs:
Ask the page a question:
read_pagewithqueryranks every passage of the page and returns the ones that answer it, with their heading path. A 435,000-character RFC answers in 2,000.Navigate long pages: an outline with section ids, one section at a time, or the page in pages (
next_start).Search and count: exact text or a regular expression, every match with context and its section; links with their text, tracking parameters removed.
PDFs are read as text page by page.
Pages are fetched once and cached for 10 minutes, so an outline, a search and a section read cost one download.
Safe to point anywhere. It reads the public web only: private, loopback, link-local (including cloud metadata at 169.254.169.254) and other reserved addresses are refused, checked on the address actually connected to and again on every redirect. It respects robots.txt (RFC 9309), identifies itself honestly in its User-Agent, and does not get around bot protection. No key needed.
Built and maintained by Arhan Canli.
Install
Needs Node.js 20 or newer. No account or key.
Claude Code
claude mcp add web-reader -- npx -y web-reader-mcpClaude Desktop: download web-reader-mcp-<version>.mcpb from the latest release and open it. The bundle is signed; verify it with gh attestation verify <file> --repo arhancanli/web-reader-mcp.
Any other client (Windsurf, Zed, Cline, Continue and others), in its MCP config file:
{
"mcpServers": {
"web-reader": {
"command": "npx",
"args": [
"-y",
"web-reader-mcp"
]
}
}
}Docker
docker build -t web-reader-mcp https://github.com/arhancanli/web-reader-mcp.git && docker run -i --rm web-reader-mcpHosted (Streamable HTTP): node src/server.mjs --http serves stateless MCP at POST /mcp (port from PORT, default 3000).
Related MCP server: Fetch MCP Server
Example
An agent calls read_page with:
{
"url": "https://www.rfc-editor.org/rfc/rfc9110.html",
"query": "Content-Location header field",
"max_chars": 2000
}and gets back (recorded from the live server on 2026-09-26):
{
"url": "https://www.rfc-editor.org/rfc/rfc9110.html",
"title": "RFC 9110: HTTP Semantics",
"passages": "2 of 419 matching",
"chars": 434823,
"content": "[...]\n\n### 8. Representation Data and Metadata > 8.7. Content-Location\n\nThe \"Content-Location\" header field references a URI that can be used as an identifier for a specific resource corresponding to the representation in this message's content. In other words, if one were to perform a GET request on this URI at the time of this message's generation, then a 200 (OK) response would contain the same representation that is enclosed as content in this message.\n\n Content-Location = absolute-URI / partial-URI\n\nThe field value is either an absolute-URI or a partial-URI. In the latter case (Section 4), the referenced URI is relative to the target URI ([URI], [Section 5](https://www.rfc-editor.org/rfc/rfc3986#section-5)).\n\nThe Content-Location value is not a replacement for the target URI (Section 7.1). It is representation metadata. It has the same syntax and semantics as the header field of the same name defined for MIME body parts in [Section 4](https://www.rfc-editor.org/rfc/rfc2557#section-4) of [RFC2557]. However, its appearance in an HTTP message has some special implications for HTTP recipients.\n\n[...]\n\nA user agent that sends Content-Location in a request message is stating that its value refers to where the user agent originally obtained the content of the enclosed representation (prior to any modifications made by that user agent). In other words, the user agent is providing a back link to the source of the original representation.\n\nAn origin server that receives a Content-Location field in a request message MUST treat the information as transitory request context rather than as metadata to be saved verbatim as part of the representation. An origin server MAY use that context to guide in processing the request or to save it for other uses, such as within source links or versioning metadata. However, an origin server MUST NOT use such context information to alter the request semantics."
}Tools
Tool | What it does |
| Finds exact text (case-insensitive) or a regular expression in a page or PDF and returns each match with surrounding context and its section id. For facts whose wording you know: a name, a number, an error message. |
| The links on a page (text and absolute URL, tracking parameters removed, duplicates dropped), optionally only those containing a word in text or URL, or only same-site ones. For crawling docs or finding a download. |
| The headings of a page or PDF with section ids and sizes, plus title, site, date and length, to read one part with read_page section instead of the whole page. |
| Reads a URL as clean Markdown (main content only). With query: only the passages that answer it, from anywhere in the page. With section (from the outline): that section. Otherwise from start; long pages give an outline and next_start. |
How it behaves
Read-only: only GET requests, to the URLs you give (and their robots.txt). Nothing is logged except unexpected failures (to stderr, without your inputs).
Refused: non-http(s) URLs, URLs with credentials, and any address that is not public (checked at connect time and on each of at most 5 redirects). Bodies are capped at 15 MB after decompression; requests time out after 20 seconds.
robots.txt is honoured for the reader's product token
web-reader-mcpand*, longest match first, as RFC 9309 specifies. A site that refuses (a 403, a bot challenge) is reported as such.Options, as environment variables:
WEB_READER_ALLOW_PRIVATE=1to read local and intranet sites (for example a dev server on localhost),WEB_READER_IGNORE_ROBOTS=1to skip robots.txt on your own responsibility.Results are compact JSON with a matching output schema. Long pages come with an outline and
next_start; lists say how many items were left out.
How it reads a page
HTML goes through Mozilla Readability (main content) and Turndown with GitHub tables (Markdown). Images and citation markers are dropped, links made absolute. When a page is not an article (an index, a listing), the whole page without navigation is used. PDFs are read with unpdf (Mozilla pdf.js). JSON is pretty-printed; plain text and Markdown are returned as they are.
Benchmark
Measured 2026-09-26 with gpt-5.4-mini, 10 fixed tasks graded by fixed checks (bench/tasks.json, raw results in bench/results/).
Server | Correct | Input tokens | Output tokens | Tool calls | Median time |
This server | 10/10 | 28243 | 833 | 17 | 4.0 s |
mcp-server-fetch, the official reference fetch server | 8/10 | 418342 | 1682 | 39 | 13.1 s |
Performance
Measured 2026-09-26 from Dubai, home connection against the live upstream, Node 24.19.0 (bench/perf.json, scripts/perf.mjs in the factory).
Call | First call | Repeat | Result size |
read_page: RFC 9110 (435,000 chars) with a query | 1527 ms | 18.1 ms | 2,085 chars |
read_page: RFC 9309 from the start (long: outline and next_start) | 992 ms | 0.4 ms | 9,829 chars |
page_outline: RFC 9309, two levels | 1308 ms | 0.3 ms | 1,019 chars |
find_in_page: 'Pemberley' in Pride and Prejudice | 2658 ms | 0.6 ms | 3,445 chars |
read_page: a PDF (RFC 9309) with a query | 1064 ms | 1.2 ms | 1,417 chars |
read_page: one section of RFC 9309 (2.5. Limits) | 907 ms | 0.2 ms | 306 chars |
page_links: Python functools docs, links about itertools | 1083 ms | 0.4 ms | 504 chars |
First call: a fresh server process, including the TLS connection and the upstream's own time. Repeat: the same call again, answered from the in-process cache, so it shows this server's own overhead.
Tool definitions the model reads on every turn (name, description, input schema): 2,290 characters, against 1,105 for mcp-server-fetch, the official reference fetch server. The full tool list, with the output schemas and annotations clients use to validate results, is 3,635 characters (1,104 for the alternative).
More MCP servers by Arhan Canli
World Time: Time anywhere, DST-safe conversions, holidays for 200+ countries, business days and meeting slots.
Actions Check: Checks GitHub Actions workflows: outdated actions, old Node runtimes, retired runners, injection.
Citation Check: Verifies citations: finds fabricated or mismatched references and retractions, returns clean BibTeX.
Contact Check: Validates and formats phone numbers, email addresses and postal addresses for any country.
Cron Check: Explains cron expressions, lists next run times in any time zone, converts between cron dialects.
Domain Health: Email and domain checks: SPF lookup limits, DKIM keys, DMARC, DNS records, registration expiry.
Drug Label: FDA drug label answers with section citations, RxNorm name resolution, recalls and shortages.
End of Life: Is this version still supported? EOL dates, latest patch and upgrade target for 470+ products.
The whole collection, 7 more
License
MIT, Copyright (c) 2026 Arhan Canli.
Available Tools
4 toolsfind_in_pageSearch a page for textARead-onlyIdempotent
Finds exact text (case-insensitive) or a regular expression in a page or PDF and returns each match with surrounding context and its section id. For facts whose wording you know: a name, a number, an error message.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | http(s) URL | |
| text | Yes | ||
| regex | No | ||
| context | No | chars around each match, default 200 |
Output Schema
| Name | Required | Description |
|---|---|---|
| url | Yes | |
| total | Yes | |
| matches | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already carry readOnlyHint, openWorldHint, idempotentHint, and destructiveHint. Beyond those, the description adds useful behavioral detail: case-insensitive matching, regex support, per-match surrounding context, and section-id association—none of which the annotations convey. No contradiction with annotations; the description complements them well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler. The first sentence front-loads the mechanism and result shape; the second gives a crisp usage pointer. Every phrase earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present and strong annotations, much of the return contract is already structured. The description covers the core behaviors an agent needs: target scope (page/PDF), matching modes, and result content. Minor gaps remain—match limits, behavior on zero matches, and what 'section id' means—but they're not critical for a read-only, idempotent tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 50% (url and context are described; text and regex are not). The description compensates by clarifying that 'text' is exact text or a regular expression, and by implying the regex boolean toggles that mode. This adds meaning beyond the bare schema for the two undocumented parameters, though it doesn't specify regex flags or the default behavior of the regex flag.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb ('Finds'), resource ('a page or PDF'), and two distinct modes (exact case-insensitive text or regular expression) plus the output shape (matches with surrounding context and section id). This clearly differentiates it from siblings like read_page, page_links, and page_outline, which are structurally different operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a concrete when-to-use signal: 'For facts whose wording you know: a name, a number, an error message.' This helps the agent choose search over reading a whole page. However, it stops short of explicitly naming alternatives or stating when not to use this tool (e.g., when the tool should defer to read_page), so it doesn't earn a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
page_linksLinks on a pageARead-onlyIdempotent
The links on a page (text and absolute URL, tracking parameters removed, duplicates dropped), optionally only those containing a word in text or URL, or only same-site ones. For crawling docs or finding a download.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | http(s) URL | |
| limit | No | default 100 | |
| contains | No | ||
| same_site | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| url | Yes | |
| links | Yes | |
| total | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds valuable behavior not covered by annotations: tracking parameters are removed and duplicates dropped, plus optional filters for same-site and text/URL containment. This is useful, though it doesn't cover aspects like pagination or rate limits, which are minor for this read-only tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single well-structured sentence: core purpose first, then output transformations in parentheses, then optional filters, ending with a use-case clause. Every phrase earns its place with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists and annotations carry the safety profile, the description covers the necessary behavior, filters, and use cases. It does not explain return format (handled by output schema) or edge cases like limit application order, which are minor for a tool of this simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 50%, but the description compensates by semantically explaining 'contains' ("containing a word in text or URL") and 'same_site' ("same-site ones"). It also describes the output preprocessing. It doesn't re-explain 'url' or 'limit', but those are self-explanatory from the schema and default behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb+resource: "The links on a page" with precise output transformations (text and absolute URL, tracking parameters removed, duplicates dropped). This clearly differentiates it from sibling tools like find_in_page (text search), page_outline (outline), and read_page (full content) without needing to inspect schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The closing phrase "For crawling docs or finding a download" gives concrete use cases and implies a lightweight extraction role. However, it does not explicitly name alternatives or state when not to use this tool versus siblings, so it falls short of full exclusion guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
page_outlineOutline of a pageARead-onlyIdempotent
The headings of a page or PDF with section ids and sizes, plus title, site, date and length, to read one part with read_page section instead of the whole page.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | http(s) URL | |
| depth | No | deepest heading level, default all |
Output Schema
| Name | Required | Description |
|---|---|---|
| url | Yes | |
| chars | Yes | |
| sections | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already establish that this is read-only, idempotent, and non-destructive, so the description does not need to repeat that. It adds useful context by specifying that the input can be a page or PDF and by enumerating the returned metadata, going beyond what annotations alone provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence that packs in the output contents, supported input types, and the read_page relationship. It is slightly dense and grammatically awkward near the end, but every phrase earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema, 100% schema parameter coverage, and safety annotations, nothing essential is missing. The description explains what data is returned and how to use it with read_page, making the tool fully callable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%: both url and depth have descriptions. The tool description adds no extra parameter-level meaning, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool's output: headings of a page or PDF with section ids, sizes, title, site, date, and length. It distinguishes itself from read_page by presenting itself as the outline rather than the full page content.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives an explicit workflow: use page_outline to locate a section, then call read_page with that section instead of reading the whole page. This tells the agent when this tool is the right choice and when read_page is the alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
read_pageRead a web page or PDFARead-onlyIdempotent
Reads a URL as clean Markdown (main content only). With query: only the passages that answer it, from anywhere in the page. With section (from the outline): that section. Otherwise from start; long pages give an outline and next_start.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | http(s) URL | |
| query | No | what you need from the page | |
| start | No | ||
| section | No | section id, e.g. s4 | |
| max_chars | No | default 8000 |
Output Schema
| Name | Required | Description |
|---|---|---|
| url | Yes | |
| chars | Yes | |
| content | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, covering safety. The description adds behavioral context beyond annotations: it specifies that only main content is extracted, that query filters passages, that section selects from an outline, and that long pages provide an outline and next_start for pagination. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place. The core purpose is front-loaded, then query/section behavior is explained succinctly, and default behavior and pagination are covered. No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main behaviors: main content extraction, query filtering, section selection, start behavior, and pagination hint. An output schema exists, so return format details are likely covered there. Minor gap: max_chars behavior is not explicitly described, but it's in the schema with a default. Overall, the description is complete enough for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 80% (4 of 5 params documented), so baseline is 3. The description adds significant meaning for query (filters passages) and section (selects from outline) beyond the schema's minimal descriptions. It also clarifies start behavior ('Otherwise from start') and mentions pagination via next_start, adding value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Reads') and resource ('a URL') and immediately clarifies output format ('clean Markdown (main content only)'). It also distinguishes from siblings by explaining query, section, and start behaviors, making the tool's purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains how to use query, section, and start parameters but does not explicitly state when to use this tool versus siblings like find_in_page, page_links, or page_outline. The usage context is implied through the description of behavior, but no exclusions or alternative recommendations are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
4 tool updates
v0.1.0- First observed
find_in_page - First observed
page_links - First observed
page_outline - First observed
read_page
TDQS
Scored across 4 tools
Each tool has a distinct primary role: read content, list links, get outline, or find literal text. The only real overlap is between find_in_page and read_page's query mode, since both can return relevant passages, but the descriptions distinguish exact/regex matching from answer-oriented extraction.
All names are lowercase snake_case and share the 'page' vocabulary, making them readable, but the pattern is mixed: find_in_page and read_page are verb-first while page_links and page_outline are noun-first. This is not chaotic, but it is not a uniform verb_noun convention.
Four tools is well-scoped for a web reader server: read, search, list links, and outline cover the core interactions without redundancy or bloat. Every tool earns its place in the set.
The tool surface covers the main web-reading workflow: fetching clean content, navigating via links, finding specific text, and using headings to selectively read sections. There are no obvious dead ends—page_links and page_outline feed naturally into read_page, and find_in_page complements full-page reading.
Maintenance
Related MCP Connectors
Read a URL as clean markdown, screenshot a website, url to PDF. Web access for agents, no signup.
Read any web page as clean Markdown for AI agents: fetch, search, metadata, links. SSRF-safe.
Web search, URL content extraction to Markdown, site mapping, and recursive web crawler.
A real browser for your agent: render any page, or 25 pages of a site, to clean text.
Related MCP Servers
- AlicenseBqualityDmaintenanceEnables LLMs to retrieve and process web content by fetching URLs and converting HTML to markdown format. Supports chunked reading of large pages and can access both public websites and local networks.1MIT
- AlicenseAqualityDmaintenanceEnables LLMs to fetch and process web content by converting HTML into markdown for easier consumption. It supports chunked reading via pagination and provides configuration options for robots.txt compliance and proxy usage.1MIT
- AlicenseAqualityCmaintenanceEnables AI agents to read web pages reliably, returning clean markdown content, hyperlinks, and metadata without navigation or ad noise.36 npmMIT
- AlicenseAqualityDmaintenanceEnables AI agents to fetch any web page as clean markdown or screenshot it, turning URLs into LLM-ready context.21 npmMIT