ai_crawler_check
Fetches a domain's robots.txt and reports which known AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot and others) are allowed or blocked, including wildcard rules. Reflects robots.txt only, not server-side blocking.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| domain | Yes | Domain to check, e.g. example.com. robots.txt is fetched from https://<domain>/robots.txt. |