citefiles_crawler_access
Test how a website answers requests that carry the user-agent strings of four AI crawlers — GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot — by requesting its homepage as a browser and again as each, then comparing. The requests come from our server, not the vendors' networks, so a rule that checks the sender's address is not tested. Catches CDN and WAF rules that block assistants without the owner knowing. Use when a site looks correct but is not being cited.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The public website address. |