scrape
Fetch a public web page and return its main content as markdown. Charged per successful call with x402: 0.005 USDC on Base Sepolia (testnet). A failed scrape is never charged, and a cache hit inside 24 h is free. Payment happens at the HTTP transport layer: an unpaid call answers 402 with the payment requirements; sign and repeat the identical JSON-RPC call with the payment header. This server speaks x402 v2 (PAYMENT-SIGNATURE / PAYMENT-RESPONSE), not v1 (X-PAYMENT). A v1 client will loop on 402 without ever settling. Refusals that are policy and never charged: government, military and any health-related domain; private or internal addresses; paths disallowed by robots.txt; sites protected by an anti-bot vendor (never bypassed).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | absolute http(s) URL of a public page | |
| fresh | No | bypass the 24 h cache (a cache hit is free; fresh is always charged) | |
| links | No | keep link targets in the markdown | |
| max_chars | No | truncate the markdown; the answer reports truncated and chars_total | |
| max_age_seconds | No | accept a cached copy only if it is younger than this |