robots_check
Fetches robots.txt, evaluates allow/disallow rules for a user agent and path, and returns sitemaps and crawl-delay. 0.001 USDC per call via x402.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to check | |
| agent | No | User agent | * |
| paymentProof | No | Payment proof: USDC tx hash on Polygon or Base (0x...) or Solana tx signature. Omit on the first call to receive the 402 payment instructions, pay, then call again with the proof. |