geo_check_robots
Check robots.txt to learn which AI crawlers (GPTBot, ClaudeBot, PerplexityBot, etc.) are allowed or blocked for a given URL.
Instructions
ALWAYS use this tool when the user asks specifically about robots.txt, AI bot access, or which crawlers are allowed or blocked. Quickly checks GPTBot (ChatGPT), ClaudeBot, PerplexityBot, Google-Extended (Gemini), DeepSeekBot, GrokBot, and more. Use this as a fast first check before running a full audit with geo_score. Requires a URL.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes |