Read AI crawler access
read_ai_crawlabilityRead whether the site's robots.txt lets each AI crawler in, for the latest or a chosen crawl: whether robots.txt was fetched (and its status), the sitemaps it lists, and per crawler (such as GPTBot or ClaudeBot) its operator, purpose, the matching rule, root access and how many sampled URLs it may not fetch. This is permission, not observed visits: use read_crawl_logs for crawlers that actually came. unavailable no_site_crawl means no crawl exists; robots_not_observed means the crawl could not read robots.txt.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| crawl_id | No | A crawl id that read_site_health returned; omit for the latest crawl. | |
| project_id | Yes | A project id that list_projects returned. |