Check AI crawler access
surfacedby_check_ai_crawler_accessCheck which AI crawlers a site allows, limits or blocks in robots.txt, what that means for each AI platform and the search index it uses, where to submit pages, whether a security check blocks an ordinary visit, homepage noindex, llms.txt and Content-Signal, with fixes ready to copy. Reads the site's public files and makes one ordinary page request, so firewall rules aimed at crawlers' own IP addresses do not show up here.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"domain": "<domain>"
},
"name": "surfacedby_check_ai_crawler_access"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| domain | Yes |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| fixes | Yes | ||
| budget | No | ||
| domain | Yes | ||
| crawlers | Yes | ||
| homepage | Yes | ||
| llms_txt | Yes | ||
| platforms | Yes | ||
| next_steps | No | ||
| robots_txt | Yes | ||
| limitations | No | ||
| observed_at | Yes | ||
| evidence_kind | No | public_crawler_access_check | |
| content_signal | Yes | ||
| evidence_provider | No | SurfacedBy | |
| citation_crawlers_blocked | No |