Check robots.txt and sitemap
check_site_filesCheck a site's robots.txt and XML sitemap for crawler blocking, sitemap location, and URL count to identify indexing issues before an SEO audit.
Instructions
Check a site's robots.txt and XML sitemap: presence, crawler blocking, sitemap location and URL count.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Any URL or domain on the site |