Which AI crawlers does this file actually stop?
audit_ai_accessFIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Evaluate a pasted robots.txt against every AI crawler in this index and return the two lists that matter: blocked and allowed, per operator and category. Also names the tokens in your file that match no known crawler (a typo blocks nothing) and separates the crawlers that document obedience from the ones observed ignoring robots.txt, which need an IP or WAF rule instead. Example: path='/' with your robots_txt pasted in — the verdict is per crawler, at that path. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/ai-access?robots_txt=&s=client-dossiers
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | Path to test the verdict at. Defaults to "/". | |
| robots_txt | Yes | The contents of the robots.txt file. |