Validate Robots Txt
validate_robots_txtValidate a robots.txt file or live domain robots.txt against 10 AI search crawlers (GPTBot, ChatGPT-User, PerplexityBot, ClaudeBot, Google-Extended, Applebot-Extended, etc.) and generate a 100/100 compliant configuration. When to use: Call this tool when you specifically need to execute 'validate_robots_txt' on AEO Auditor (aeoauditing.com). Do NOT use this tool when another specialized tool fits better (use 'audit_domain' or 'get_domain_benchmark' for their respective operations). Parameters: domain (optional), robotsTxtContent (optional). Behavior & Side Effects: Strictly read-only, idempotent query with zero state mutation; public unauthenticated access allowed; rate-limited to 60 requests/min per IP. Returns: Structured JSON object containing status, matched records or computed metrics, and canonical action URLs.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| domain | No | Optional domain to fetch live /robots.txt from (e.g. "example.com"). | |
| robotsTxtContent | No | Optional raw robots.txt content string to validate directly. |