Which AI crawlers does this file actually stop?
audit_ai_accessEvaluate a pasted robots.txt against every AI crawler in this index and return the two lists that matter: blocked and allowed, per operator and category. Also names the tokens in your file that match no known crawler (a typo blocks nothing) and separates the crawlers that document obedience from the ones observed ignoring robots.txt, which need an IP or WAF rule instead.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | Path to test the verdict at. Defaults to "/". | |
| robots_txt | Yes | The contents of the robots.txt file. |