site_ai_crawler_policy
Which AI crawlers does a website block in robots.txt? Looks up a domain in our scan of the top 10,000 websites (Tranco list) and returns, per AI bot (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, ...), whether it is blocked, partly restricted or explicitly allowed, plus llms.txt presence.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"domain": "<domain>"
},
"name": "site_ai_crawler_policy"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| domain | Yes | Domain or URL, e.g. 'nytimes.com' or 'https://www.nytimes.com/section'. |