Evaluate a robots.txt for one crawler or every AI crawler in the directory, per RFC 9309
checkRobotsTxtEvaluate a robots.txt for one crawler or every AI crawler in the directory, per RFC 9309. Pass the robots.txt text; nothing is fetched. Returns allowed or disallowed per crawler for the path, the matching group and rule, sitemaps, Content-Signal lines, and warnings. Directory crawlers that may ignore robots.txt for user-initiated fetches are flagged. Same as GET https://bozos.si/api/v1/robots-check.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | Path (with optional query) or full URL to check (default /) | |
| agent | No | A crawler token (GPTBot), a full user-agent string, or all (default: every crawler in the directory) | |
| robots | Yes | The robots.txt text, URL-encoded (up to 65,536 characters) |