Check AI crawler access
check_ai_crawlersReads a website's robots.txt and reports which of 15 AI crawlers (from OpenAI, Anthropic, Google, Apple, Meta and others) may crawl its homepage, separating bots that fetch pages for AI answers from bots that collect training data, plus the robots.txt lines that would let blocked answer bots in.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"website": "<website>"
},
"name": "check_ai_crawlers"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| website | Yes | The product's website, like acme.com (a full URL works too). |