Check whether AI crawlers are blocked in robots.txt
check_ai_crawler_accessUse this when someone asks why their site never shows up in AI answers, or whether GPTBot, ClaudeBot, PerplexityBot or Google-Extended can crawl it. Reads the site's robots.txt and reports which of those crawlers are blocked, including by a blanket rule the owner may not have meant. One fetch, no AI.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"url": "<url>"
},
"name": "check_ai_crawler_access"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The site or page URL to check. A bare domain like example.com is fine. |