# ai-visibility-mcp

An MCP server for checking and improving a website's visibility to AI crawlers and answer engines (ChatGPT, Claude, Perplexity, Google AI Overviews, etc.) against your own properties.

Most "AI SEO" advice stops at "add an llms.txt." In practice the bigger and more common problem is upstream of that: `robots.txt` — often auto-managed by a CDN like Cloudflare — silently blocking the exact crawlers you want to reach, which makes any llms.txt or content work moot. This server checks both, plus can audit and generate the fix.

## Tools

- **`check_visibility(url)`** — fetches a site's live `robots.txt` and flags any named AI bot (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, Applebot-Extended, CCBot, Bytespider, Amazonbot, meta-externalagent, and others) blocked with `Disallow: /`, or a wildcard block. Also checks whether `llms.txt` exists as a real file versus falling through to a SPA catch-all route (a common false-positive: the file "200s" but it's actually your homepage).
- **`cloudflare_ai_bot_audit(domain)`** — reads a Cloudflare zone's Bot Management AI-bot settings (`ai_bots_protection`, `ai_training`, `ai_search`, `ai_user`, `is_robots_txt_managed`) for a domain you control. Requires `CLOUDFLARE_API_EMAIL` + `CLOUDFLARE_API_KEY` (Global API Key) env vars.
- **`generate_llms_txt(name, description, url, phrase?)`** — generates an `llms.txt` line and, if you supply an ownable niche `phrase`, a small content pack (meta description + FAQ block) anchored on that phrase. Pure text generation — no filesystem or git writes.

## Install

```bash
git clone https://github.com/tsushanth/ai-visibility-mcp.git
cd ai-visibility-mcp
npm install
npm run build
```

Add to your MCP client config (e.g. Claude Code, Claude Desktop):

```json
{
  "mcpServers": {
    "ai-visibility": {
      "command": "node",
      "args": ["/absolute/path/to/ai-visibility-mcp/dist/index.js"],
      "env": {
        "CLOUDFLARE_API_EMAIL": "you@example.com",
        "CLOUDFLARE_API_KEY": "your-global-api-key"
      }
    }
  }
}
```

The Cloudflare env vars are only needed for `cloudflare_ai_bot_audit` — `check_visibility` and `generate_llms_txt` work with no configuration.

## Why this exists

Sites on Cloudflare frequently have AI-crawler blocking turned on by default (the "Block AI Bots" toggle), which rewrites the served `robots.txt` to disallow GPTBot, ClaudeBot, Google-Extended, and others — even when the site owner never explicitly decided that and actually wants AI-search visibility. This is easy to miss because the site still looks and functions normally to human visitors and to regular search engines; only AI crawlers are silently turned away. `check_visibility` surfaces this from the outside (what's actually being served) and `cloudflare_ai_bot_audit` confirms it from the account side (what Cloudflare's settings say), so you can tell the two apart.

## License

MIT