check_ai_crawler_access
Returns a site's robots.txt verdict for AI crawlers, including allowed and blocked bots, with a paid upgrade link. Use to inspect one public website's declared crawler policy. Not for proving that a CDN or firewall actually permits a bot, or for ongoing monitoring. Apify Actor links can be paid with x402 via Apify, no Apify account needed.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Public website URL or domain. |