Check AI crawler robots.txt policy
check_ai_crawler_policyCheck the homepage robots.txt rule for eight named OpenAI, Anthropic, Perplexity, and Google AI crawler tokens. The free result stays complete and includes a bounded optional handoff to exact monitoring terms. Results describe robots policy only and do not prove network access, authentic bot identity, crawling, indexing, citation, or ranking.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"url": "<url>"
},
"name": "check_ai_crawler_policy"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Public HTTP or HTTPS website URL. Private and local-network targets are rejected. |