Read a site's robots.txt rules per crawler
resolve_robotsResolve every named answer engine, search index and training crawler against a domain's robots.txt the way a crawler does: most-specific group only, longest match, allow wins a tie. Shows when a Disallow under * does not apply to an agent with its own group.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"domain": "<domain>"
},
"name": "resolve_robots"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | Path to test, default / | |
| domain | Yes |