Skip to main content
Glama
Duffey234

webread

by Duffey234
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Hybrid

WebRead

URL to clean Markdown for AI agents (MCP server + x402 API).

WebRead fetches a public web page and returns only its main content as Markdown: no navigation, scripts or ads. Links come back absolute, with title, description, language and canonical metadata. It respects robots.txt and refuses private/internal addresses (SSRF-safe).

  • Hosted remote MCP: https://api.youraitools.co/webread/mcp (Streamable HTTP)

  • Free tier: 30 full reads/day per IP (full Markdown, up to 200,000 characters) on the remote MCP endpoint, then $0.002 USDC per successful read over x402, in USDC on Base, Polygon, Arbitrum One or Solana. No account or API key.

  • Hosted API: https://api.youraitools.co/webread (GET /v1/read?url=<page> ($0.002; failed reads are free), free GET /v1/read/preview?url=<page> (first 2,000 characters, 10/min per IP))

  • Local stdio MCP server: this repo. Starts with no key, no account and no npm registry package.

  • MCP Registry: co.youraitools/webread

  • License: MIT

Use the hosted server (no install)

Claude Code:

claude mcp add --transport http webread https://api.youraitools.co/webread/mcp

Claude Desktop: Settings > Connectors > Add custom connector with the URL above, or in claude_desktop_config.json:

{ "mcpServers": { "webread": { "command": "npx", "args": ["-y", "mcp-remote", "https://api.youraitools.co/webread/mcp"] } } }

Cursor (~/.cursor/mcp.json), or one click: Add to Cursor

{ "mcpServers": { "webread": { "url": "https://api.youraitools.co/webread/mcp" } } }

Each free result includes an upgrade field with the calls left today. Past the daily limit the tool returns a payment_required (402) result with the paid x402 URL.

Related MCP server: crawl-mcp-server

Self-host: run from source (no npm registry)

Requires Node.js 20+ (or Docker). The server has no runtime dependencies.

git clone https://github.com/Duffey234/webread.git
cd webread
npm ci --omit=dev
node src/mcp-stdio.js        # stdio MCP server (newline-delimited JSON-RPC 2.0)

No git? Download the source tarball from the mirror instead: curl -sL https://agentgate-source.pages.dev/x402-url-to-markdown/x402-url-to-markdown.tar.gz | tar xz.

MCP client config (use the absolute path of your checkout):

{ "mcpServers": { "webread": { "command": "node", "args": ["/absolute/path/to/webread/src/mcp-stdio.js"] } } }

Claude Code: claude mcp add webread -- node /absolute/path/to/webread/src/mcp-stdio.js

Tool read_url reads pages locally on your machine with the same SSRF-safe, robots.txt-respecting reader and HTML-to-Markdown converter the hosted API uses (default 50,000 characters, max 200,000). Nothing is sent to WebRead.

Docker

docker build -t webread-mcp .
docker run -i --rm webread-mcp
{ "mcpServers": { "webread": { "command": "docker", "args": ["run", "-i", "--rm", "webread-mcp"] } } }

Test

npm test        # full test suite (node:test, Node 20 and 22, no network, no key)

Full source

The complete hosted service (Cloudflare Worker, x402 v2 + v1, remote MCP) is on the source mirror: https://agentgate-source.pages.dev/x402-url-to-markdown/

Built by YourAITools. Informational tooling; no warranty.

Available Tools

1 tool
read_urlRead a web page as MarkdownA
Read-only

Fetch a public web page and return its main content as clean Markdown (no navigation, scripts or ads), with title and description. Respects robots.txt and refuses private/internal addresses. Runs locally; maxChars defaults to 50,000 (max 200,000).

NameRequiredDescriptionDefault
urlYesAbsolute http(s) URL
maxCharsNoMaximum Markdown characters to return (default 50000)

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations cover only readOnly and openWorld, so the description carries real weight here: it discloses content cleaning, robots.txt compliance, SSRF-style address blocking, local execution, and the maxChars default/ceiling. These are behavioral facts an agent could not infer from the annotations or schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tightly packed sentence with zero filler; the core action and return format come first, with constraints and limits trailing. Every clause (cleaning, robots.txt, address blocking, char limit) contributes distinct information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description adequately describes the return payload (Markdown plus title/description) and the size cap. It stops short of covering failure modes — non-HTML content, unreachable hosts, or whether truncated output is flagged — which an agent would want before relying on it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both parameters are fully documented there. The description restates the maxChars default and maximum verbatim and adds no new semantics (e.g. truncation behavior) beyond the schema, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Fetch a public web page') and precisely characterizes the output ('main content as clean Markdown (no navigation, scripts or ads), with title and description'). No siblings exist, so no differentiation is needed; the purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives the operative usage boundary: only public pages, robots.txt respected, private/internal addresses refused. With no sibling tools there are no alternatives to route to, so the guidance is complete in context, though it does not address what to do when a fetch is refused.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 1 tool updatev0.1.4
    • First observedread_url
Deploy Server
Give agents eyes on the webHosted screenshot MCP. Send a URL, get a pixel-perfect image. $0.002 per grab. Works with Claude, Cursor, and Codex.grabbit.liveAd

TDQS

A4.4/5.0

Scored across 1 tool

Disambiguation5/5

There is only one tool, so there is no possibility of misselection or overlapping purpose. Its scope (fetch a URL, return clean Markdown) is unambiguous.

Naming Consistency5/5

read_url follows a clear verb_noun snake_case convention. With a single tool there is no inconsistency, and the name directly reflects the action.

Tool Count4/5

A single tool is thin in absolute terms, but it is well matched to a narrowly scoped server whose sole job is fetching and cleaning web page content. The count is defensible rather than excessive or arbitrary.

Completeness4/5

For its stated purpose of reading a page as Markdown, the core operation is fully covered with sensible safety constraints (robots.txt, private address refusal, char limits). Minor gaps remain, such as batch fetching, link extraction, or explicit fallbacks when a page is JS-rendered or blocked.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • crawl-mcp-server

    Git-Fg
    F
    license
    Not graded
    quality
    Not graded
    maintenance
    An MCP server for web content extraction that converts HTML pages into clean, LLM-optimized Markdown using Mozilla's Readability. It supports batch processing, intelligent multi-page crawling, and configurable caching while respecting robots.txt standards.
    32 npm
    -

  • WebforAI Text Extractor

    MCP-Mirror
    A
    license
    Not graded
    quality
    Not graded
    maintenance
    An MCP server that extracts clean, structured Markdown content from web page URLs using the WebforAI library. It simplifies feeding web content into AI models by removing HTML noise and intelligently processing tables and links.
    -

  • pulldown

    anthony-maio
    A
    license
    Not graded
    quality
    C
    maintenance
    MCP server for retrieving web pages as clean Markdown, with configurable detail levels and optional Chromium rendering for JavaScript-heavy pages.
    1
    MIT