AI Crawler Access Check by Citant.ai
Server Details
Can ChatGPT, Claude and Perplexity crawlers reach a site? Real per-bot fetch, firewall/CDN blocks.
- Status
- Healthy
- Uptime
- 100.0% over the last 24 hours
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-11-25
- URL
- ChatGPT
- –
- Claude
- –
TDQS
Scored across 1 tool
There is only one tool, so there is no possibility of an agent misselecting between overlapping capabilities. Its purpose is stated unambiguously in the description.
The single tool uses a clear snake_case verb_noun pattern (check_ai_crawler_access) that is self-descriptive and idiomatic for MCP tooling. There are no competing conventions to introduce inconsistency.
One tool is thin by general standards, but the server's scope is a single focused check, and the tool is substantive rather than trivial, returning a score, per-crawler verdicts, and fixes. It fully earns its place with no bloat.
The tool covers the core domain well: per-crawler reachability testing, firewall/CDN detection, noindex and llms.txt inspection, plus actionable output. Minor gaps exist, such as no batch/site-list checking, no custom user-agent parameter, or no way to retrieve raw fetch traces.
Available Tools
1 toolcheck_ai_crawler_accessCheck AI crawler accessARead-onlyIdempotentInspect
Checks whether AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended and others) can actually reach a website. Goes beyond robots.txt: fetches the site as each crawler and detects firewall/CDN blocks (Cloudflare, Akamai, Imperva), challenge pages, noindex and llms.txt. Returns a 0-100 AI Access Score, a per-crawler verdict and the top issues with fixes. Takes 15-60 seconds.
Inputs: url (required).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Related MCP Connectors
Can ChatGPT, Claude and Perplexity reach and read a website? Score, problems found and fixes.
Can ChatGPT, Claude and Perplexity crawlers read your page? Audit, fixes, compare, monitoring.
Can AI crawlers read this site? GPTBot/ClaudeBot verdicts, robots vs edge blocking, 47k-domain index
Checks if GPTBot, ClaudeBot, PerplexityBot, Googlebot and 9 more AI crawlers can reach a page.
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceEnables users to check whether GPTBot, ClaudeBot, PerplexityBot, Googlebot and other AI/search crawlers can reach a specified page, distinguishing robots.txt blocks from server or firewall refusals.MIT
- AlicenseAqualityDmaintenanceAudits AI-bot visibility: robots.txt per-bot for 22 AI user-agents (GPTBot/ClaudeBot/PerplexityBot/etc), Cloudflare flags, JSON-LD, sitemap, llms.txt, SPA shell, plus cross-model brand mentions via Perplexity + OpenRouter. 0-100 score. SSRF-guarded, spend-capped.41MIT
- AlicenseAqualityDmaintenanceAnalyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.51MIT
- AlicenseAqualityDmaintenanceChecks a website's robots.txt and Cloudflare settings to identify AI crawler blocking. Also generates llms.txt content to improve visibility to AI answer engines.355 npmMIT
Glama MCP Gateway
Add one secure layer between your agents and this server.