Skip to main content
Glama

Server Details

Can AI crawlers read this site? GPTBot/ClaudeBot verdicts, robots vs edge blocking, 47k-domain index

Glama couldn't complete the latest health check. If this server requires authentication, missing or expired test credentials may be the cause. A test profile lets Glama authenticate for health checks and discover tools; it is separate from your personal connections.

If you are the author, claim ownership, then add or update a test profile under Admin → Test Profile.

Status
Unhealthy
Uptime
4.5% over 21 days
Last Tested
Transport
Streamable HTTP
URL

TDQS

A3.9/5.0

Scored across 5 tools

Disambiguation4/5

Most tools are clearly distinct: domain_readiness covers archived per-domain status, measure_domain covers fresh measurement, edge_blocking covers vendor attribution, recent_changes covers temporal flips, and web_openness covers aggregate statistics. The only close pair is domain_readiness and measure_domain, but the explicit 'right now' vs 'archive' distinction prevents serious confusion.

Naming Consistency3/5

All names are readable snake_case, but they do not follow a consistent pattern: domain_readiness and web_openness are noun compounds, edge_blocking is noun+gerund, measure_domain is verb+noun, and recent_changes is adjective+noun. The mixed conventions are still predictable enough to infer intent, but they are not uniform.

Tool Count5/5

Five tools is well-scoped for an analytics and measurement server. Each tool covers a distinct question a user would reasonably ask: one domain now, one domain from archived data, which edge vendor is responsible, what changed recently, and overall web openness. There is no obvious redundancy or bloat.

Completeness4/5

The tool surface covers the core workflows: per-domain readiness, live remeasurement, aggregate openness, vendor-level edge blocking, and change detection. A minor gap is the lack of a direct way to enumerate all domains that currently block a specific crawler, but this does not undermine the main use cases.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 1 tool update
    • Addedrecent_changes
  2. 4 tool updates
    • First observeddomain_readiness
    • First observededge_blocking
    • First observedmeasure_domain
    • First observedweb_openness

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    D
    maintenance
    Audits AI-bot visibility: robots.txt per-bot for 22 AI user-agents (GPTBot/ClaudeBot/PerplexityBot/etc), Cloudflare flags, JSON-LD, sitemap, llms.txt, SPA shell, plus cross-model brand mentions via Perplexity + OpenRouter. 0-100 score. SSRF-guarded, spend-capped.
    4
    1
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Lets an AI assistant check any URL to see how much of a page's main content is present in the raw HTML versus only after JavaScript runs, showing which text blocks, meta fields, canonical tags, JSON-LD and links AI crawlers like GPTBot and ClaudeBot cannot read. It also reports what robots.txt allows for 22 AI bot tokens and how the site responds to each bot's user-agent.
    AGPL 3.0
  • A
    license
    A
    quality
    C
    maintenance
    Analyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.
    5
    1
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Checks a website's robots.txt and Cloudflare settings to identify AI crawler blocking. Also generates llms.txt content to improve visibility to AI answer engines.
    3
    43 npm
    MIT
Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources