check_robots_sitemap
Check a site's robots.txt and sitemap.xml in one call: raw contents (truncated), sitemap presence, an AI-crawler policy analysis (explicit rules for GPTBot, ClaudeBot, CCBot, PerplexityBot, Google-Extended, Bytespider, Amazonbot and other AI crawlers, vs wildcard/default), and llms.txt presence. For a scored, whole-site AI-readiness report use ai_readiness_audit instead.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Any URL on the site to check (origin is derived from it) |