Check which AI crawlers robots.txt allows
check_ai_crawlersReads a site's robots.txt and says, for each AI crawler (OpenAI, Anthropic, Perplexity, Google, Apple, Meta, Parallel and others), whether it may fetch the given path and which rule decides.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The page address, for example example.com/pricing/. https:// is assumed if left out. |