Site Crawl to Markdown
site_crawlWebsite crawler: read a whole website or one section of it (for example every page under /docs) as clean Markdown: starts at a URL, follows same-site links breadth-first and returns up to 20 pages with title and description. Use for documentation, product catalogs or company sites. For one page use web-scrape, for a known list of URLs web-read-batch (up to 5) or web-scrape-batch (up to 20). Charged only if at least one page was read. Params: url (required), max_pages (1 to 20, default 5), max_chars per page (500 to 20000, default 4000), path_prefix (e.g. /docs). Price: $0.005 USDC per call, paid inside this MCP call via x402 (USDC on Base, Solana, Polygon or Arbitrum). x402 MCP clients such as @x402/mcp pay automatically. Free sample: call with the example input {"url":"https://agentsvc.io/docs","max_pages":3,"max_chars":2000}.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| max_chars | No | ||
| max_pages | No | ||
| path_prefix | No | Only follow links whose path starts with this, e.g. /docs |