ai_visibility_audit
Check a URL origin's robots.txt and llms.txt to determine AI crawler access and discoverability.
Instructions
Check robots.txt AI crawler access and llms.txt presence for a URL's origin.
Fetches {origin}/robots.txt and checks each known AI crawler agent. Also checks for {origin}/llms.txt (Model Context Protocol discoverability file). No authentication required.
Verdicts: open (all AI crawlers allowed) | partial (some blocked) | closed (all blocked or disallow all) | fetch_error.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |