ai-visibility-mcp

An MCP server for checking and improving a website's visibility to AI crawlers and answer engines (ChatGPT, Claude, Perplexity, Google AI Overviews, etc.) against your own properties.

Most "AI SEO" advice stops at "add an llms.txt." In practice the bigger and more common problem is upstream of that: robots.txt — often auto-managed by a CDN like Cloudflare — silently blocking the exact crawlers you want to reach, which makes any llms.txt or content work moot. This server checks both, plus can audit and generate the fix.

Tools

• check_visibility(url) — fetches a site's live robots.txt and flags any named AI bot (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, Applebot-Extended, CCBot, Bytespider, Amazonbot, meta-externalagent, and others) blocked with Disallow: /, or a wildcard block. Also checks whether llms.txt exists as a real file versus falling through to a SPA catch-all route (a common false-positive: the file "200s" but it's actually your homepage).

• cloudflare_ai_bot_audit(domain) — reads a Cloudflare zone's Bot Management AI-bot settings (ai_bots_protection, ai_training, ai_search, ai_user, is_robots_txt_managed) for a domain you control. Requires CLOUDFLARE_API_EMAIL + CLOUDFLARE_API_KEY (Global API Key) env vars.

• generate_llms_txt(name, description, url, phrase?) — generates an llms.txt line and, if you supply an ownable niche phrase, a small content pack (meta description + FAQ block) anchored on that phrase. Pure text generation — no filesystem or git writes.

Related MCP server: ai-readiness

Install

git clone https://github.com/tsushanth/ai-visibility-mcp.git
cd ai-visibility-mcp
npm install
npm run build

Add to your MCP client config (e.g. Claude Code, Claude Desktop):

{
  "mcpServers": {
    "ai-visibility": {
      "command": "node",
      "args": ["/absolute/path/to/ai-visibility-mcp/dist/index.js"],
      "env": {
        "CLOUDFLARE_API_EMAIL": "you@example.com",
        "CLOUDFLARE_API_KEY": "your-global-api-key"
      }
    }
  }
}

The Cloudflare env vars are only needed for cloudflare_ai_bot_audit — check_visibility and generate_llms_txt work with no configuration.

Why this exists

Sites on Cloudflare frequently have AI-crawler blocking turned on by default (the "Block AI Bots" toggle), which rewrites the served robots.txt to disallow GPTBot, ClaudeBot, Google-Extended, and others — even when the site owner never explicitly decided that and actually wants AI-search visibility. This is easy to miss because the site still looks and functions normally to human visitors and to regular search engines; only AI crawlers are silently turned away. check_visibility surfaces this from the outside (what's actually being served) and cloudflare_ai_bot_audit confirms it from the account side (what Cloudflare's settings say), so you can tell the two apart.

License

MIT