Skip to main content
Glama

AI Crawler Access Checker

by io.github.Nero-Engine
OverviewDiscussionsAdmin

Server Details

Shows if GPTBot, ClaudeBot, PerplexityBot and other AI crawlers can read each site in a list.

The last two health checks of this connector failed. If this server requires authentication, missing or expired test credentials may be the cause. A test profile lets Glama authenticate for health checks and discover tools; it is separate from your personal connections.

If you are the author, claim ownership, then add or update a test profile under Admin → Test Profile.

Status
Unhealthy
OAuth
Works in Glama
Last Tested
Transport
Streamable HTTP
URL

Related MCP Connectors

Related MCP Servers

  • robotstxt-ai-mcp

    sharozdawa
    A
    license
    A
    quality
    C
    maintenance
    Analyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.
    5
    1
    MIT

  • OpenGEO

    rohanjha2019
    A
    license
    A
    quality
    C
    maintenance
    Enables AI agents to audit and improve a site's visibility in AI search by checking robots.txt against 17 major AI crawlers, detecting llms.txt, and returning a 0-100 visibility score with per-crawler verdicts; it also lists known AI crawlers and their purposes.
    2
    MIT

  • ai-visibility-mcp

    bestaiinsider
    A
    license
    A
    quality
    D
    maintenance
    Audits AI-bot visibility: robots.txt per-bot for 22 AI user-agents (GPTBot/ClaudeBot/PerplexityBot/etc), Cloudflare flags, JSON-LD, sitemap, llms.txt, SPA shell, plus cross-model brand mentions via Perplexity + OpenRouter. 0-100 score. SSRF-guarded, spend-capped.
    4
    1
    MIT

  • unrendered-mcp

    imfurkana
    A
    license
    Not graded
    quality
    C
    maintenance
    Lets an AI assistant check any URL to see how much of a page's main content is present in the raw HTML versus only after JavaScript runs, showing which text blocks, meta fields, canonical tags, JSON-LD and links AI crawlers like GPTBot and ClaudeBot cannot read. It also reports what robots.txt allows for 22 AI bot tokens and how the site responds to each bot's user-agent.
    AGPL 3.0
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources