Skip to main content
Glama

Server Details

Can ChatGPT, Claude and Perplexity crawlers reach a site? Real per-bot fetch, firewall/CDN blocks.

If you are the author of this connector, you can claim ownership. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Healthy
Uptime
100.0% over the last 24 hours
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL
ChatGPT
–
Claude
–

TDQS

A4.5/5.0

Scored across 1 tool

Disambiguation5/5

There is only one tool, so there is no possibility of an agent misselecting between overlapping capabilities. Its purpose is stated unambiguously in the description.

Naming Consistency5/5

The single tool uses a clear snake_case verb_noun pattern (check_ai_crawler_access) that is self-descriptive and idiomatic for MCP tooling. There are no competing conventions to introduce inconsistency.

Tool Count4/5

One tool is thin by general standards, but the server's scope is a single focused check, and the tool is substantive rather than trivial, returning a score, per-crawler verdicts, and fixes. It fully earns its place with no bloat.

Completeness4/5

The tool covers the core domain well: per-crawler reachability testing, firewall/CDN detection, noindex and llms.txt inspection, plus actionable output. Minor gaps exist, such as no batch/site-list checking, no custom user-agent parameter, or no way to retrieve raw fetch traces.

Available Tools

1 tool
check_ai_crawler_accessCheck AI crawler accessA
Read-onlyIdempotent
Inspect

Checks whether AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended and others) can actually reach a website. Goes beyond robots.txt: fetches the site as each crawler and detects firewall/CDN blocks (Cloudflare, Akamai, Imperva), challenge pages, noindex and llms.txt. Returns a 0-100 AI Access Score, a per-crawler verdict and the top issues with fixes. Takes 15-60 seconds.

Inputs: url (required).

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      D
      maintenance
      Audits AI-bot visibility: robots.txt per-bot for 22 AI user-agents (GPTBot/ClaudeBot/PerplexityBot/etc), Cloudflare flags, JSON-LD, sitemap, llms.txt, SPA shell, plus cross-model brand mentions via Perplexity + OpenRouter. 0-100 score. SSRF-guarded, spend-capped.
      4
      1
      MIT
    • A
      license
      A
      quality
      D
      maintenance
      Analyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.
      5
      1
      MIT
    • A
      license
      A
      quality
      D
      maintenance
      Checks a website's robots.txt and Cloudflare settings to identify AI crawler blocking. Also generates llms.txt content to improve visibility to AI answer engines.
      3
      55 npm
      MIT
    Try in Browser

    Glama MCP Gateway

    Add one secure layer between your agents and this server.

    Resources