What search and AI bots really crawl
analyze_logsAnalyse server access log lines to see what Googlebot, Bingbot and AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot, Applebot, Meta, Amazon, ByteDance, CCBot) actually request: hits per bot, the pages they crawl most, errors and redirects they get, crawl budget spent on parameters and assets, and, with the site's sitemap URLs, which pages Googlebot never visits. Pass raw lines in the common or combined log format (Apache, Nginx, Cloudflare, most hosts), up to 5000 lines. Logs can hold visitor IP addresses: only send them when the user agrees.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| log | Yes | Raw access log lines, one request per line. | |
| sitemap_urls | No | Optional: the URLs from the site's sitemap, to find pages bots never crawl. |