check_bots
Probes a URL with multiple AI bot user-agents in parallel and returns per-bot reachability (status, key WAF/CDN headers, and a verdict). Reveals WAF/CDN-layer blocks that robots.txt audits miss — e.g. a site that allows GPTBot in robots.txt but returns 403 to it at Cloudflare. Verdicts: reachable, blocked-403, challenged-by-cdn, rate-limited-429, server-error-5xx, redirected-to-block-page, timeout, fetch-error. The summary.blockingPattern field names exactly which bots got blocked when others got through.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Absolute http(s) URL to probe | |
| timeoutMs | No | Per-fetch timeout in ms (default 5000) | |
| userAgents | No | Optional UA strings to test. Defaults to GPTBot, ClaudeBot, Claude-Web, PerplexityBot, GoogleOther, Google-Extended, CCBot, Applebot-Extended, ChatGPT-User, meta-externalagent. |