robots_txt_check
Fetch robots.txt as Googlebot to check status, Google's interpretation, 500 KiB size limit, groups, rules, sitemaps, syntax warnings, and changes since the last check; results are saved to history.
Instructions
Fetch robots.txt like Googlebot: status and Google's interpretation, size vs the 500 KiB limit, groups, Googlebot rules, sitemaps, syntax warnings, and a diff if it changed since the last check. Saved to history.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| host_or_url | Yes | Host, URL or sc-domain property, e.g. www.example.com |