robots_check
Fetch a site's robots.txt + llms.txt and report whether the major AI crawlers are allowed/blocked, plus sitemaps. POST { url }; unpaid returns 402.
Instructions
Fetch a site's robots.txt + llms.txt and report whether the major AI crawlers are allowed/blocked, plus sitemaps. POST { url }; unpaid returns 402. (PAID x402 service — USDC on Base; the MCP server needs a funded wallet to settle.) Price: $0.003 USD (fixed).
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Any URL on the site |