Test URLs against robots.txt
seo-technical_robots-txt-testEvaluate path access for a user-agent against a caller-supplied robots.txt using RFC 9309 longest-match rules. Price $0.02 USDC. POST /api/v1/seo-technical/robots-txt-test. An unpaid call returns the x402 payment terms and does not run.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"robotsTxt": "<robotsTxt>",
"userAgent": "<userAgent>",
"urls": "<urls>"
},
"name": "seo-technical_robots-txt-test"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| urls | Yes | urls, array. Required. | |
| robotsTxt | Yes | robotsTxt, string, length 0 to 512000. Required. | |
| userAgent | Yes | userAgent, string, length 1 to 256. Required. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| group | No | group. String. | |
| results | No | results. Array. | |
| sources | No | sources. Array. | |
| sitemaps | No | sitemaps. Array. | |
| crawlDelay | No | crawlDelay. Null. | |
| limitations | No | limitations. Array. | |
| unknownDirectives | No | unknownDirectives. Array. |