check_ai_crawlers
Check if AI crawlers like GPTBot, ClaudeBot, or PerplexityBot can access a website by analyzing its robots.txt file. Useful for diagnosing why a site does not appear in AI answer engines.
Instructions
Check whether AI crawlers like GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, and Bingbot are allowed to read a website, based on its robots.txt. Use when asked about AI visibility, AEO, crawler permissions, or why a site does not appear in ChatGPT or Perplexity.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"url": "<url>"
},
"name": "check_ai_crawlers"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Website URL, e.g. https://example.com (robots.txt is derived from it) |