check_crawler_blocks
Check whether robots.txt or a live probe blocks AmazonAdBot / Google ad crawlers. robots.txt Disallow and Amazon Crawl-delay are high-confidence fails. A WAF 403 against our scanner is a medium-confidence warn that needs verification.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| domain | Yes | Publisher domain, for example example.com. | |
| outputFormat | No | markdown |