@readfetch/mcp-x402-proxy

readfetch_crawl

Read-only

Start a bounded asynchronous crawl from a seed URL, then poll the returned job ID for completed pages with per-page Markdown; cap scope with max_pages and max_depth.

Instructions

Start a bounded async crawl from one seed URL. Returns job_id, status (queued), pages_quoted, and price_usd — then poll readfetch_crawl_status with that id until status is completed or failed. Completed jobs include per-page url and Markdown in the status payload. Cap scope with max_pages (default 20, max 100) and max_depth (default 2, max 3); include/exclude filter paths by prefix (* suffix on include). Prefer readfetch_batch for a fixed URL list; readfetch_map to list URLs without fetching bodies; readfetch_text for a single page.

Input Schema

Name	Required	Description
`url`	Yes	Seed URL; crawl stays on the same registrable domain.
`exclude`	No	Path prefixes to skip during the crawl.
`include`	No	Path prefixes to allow; * suffix wildcard.
`max_depth`	No	Link depth from the seed URL (default 2).
`max_pages`	No	Maximum pages to fetch in this job (quoted up front for payment).

Output Schema

Name	Required	Description	Default
`hint`	No
`job_id`	No
`status`	No
`price_usd`	No
`pages_quoted`	No

Schema Changelog

Changes observed during successful MCP inspections.

Changed6 schema fields changedv0.1.13Nov 1, 2026

addedInput schema / properties / exclude / description

Added value: +"Path prefixes to skip during the crawl."

changedInput schema / properties / include / description

Previous value: -"Path prefixes; * suffix wildcard."New value: +"Path prefixes to allow; * suffix wildcard."

addedInput schema / properties / max_depth / description

Added value: +"Link depth from the seed URL (default 2)."

addedInput schema / properties / max_pages / description

Added value: +"Maximum pages to fetch in this job (quoted up front for payment)."

addedInput schema / properties / url / description

Added value: +"Seed URL; crawl stays on the same registrable domain."

changedOutput schema / (root)

Previous value: -nullNew value: +{
+  "properties": {
+    "hint": {
+      "type": "string"
+    },
+    "job_id": {
+      "type": "string"
+    },
+    "pages_quoted": {
+      "type": "integer"
+    },
+    "price_usd": {
+      "type": "number"
+    },
+    "status": {
+      "type": "string"
+    }
+  },
+  "type": "object"
+}

First observedv0.1.0Nov 1, 2026

TDQS

A4.4/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only say readOnlyHint/openWorldHint; the description adds real behavior beyond that — asynchronous job semantics, the exact return fields (job_id, status, pages_quoted, price_usd), the polling contract, the terminal states, and what a completed payload contains. It doesn't spell out failure/error handling or what happens if the poll never completes, which keeps it short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose, then returns, then polling, then scope caps, then the alternative routing — a sensible priority order with no filler sentences. Minor redundancy in restating schema defaults costs it a point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an async, paid, open-world crawl with 5 params and an output schema, the description covers initiation, cost quoting, poll cadence, terminal states, scoping knobs, and sibling routing. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so url, include, exclude, max_pages and max_depth are already documented with defaults and bounds. The description mostly repeats those defaults and maxima ('default 20, max 100', 'default 2, max 3') rather than adding semantics, though the wildcard note on include is a useful confirmation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb and resource ('Start a bounded async crawl from one seed URL') and immediately situates it among siblings: batch for a fixed list, map for URL listing, text for a single page. An agent can pick this tool over readfetch_scrape or readfetch_fetch without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit routing rules are given for all three alternatives, plus the required follow-up call ('poll readfetch_crawl_status with that id until status is completed or failed'). It states both when to use this tool and when not to.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server