Checks a website's robots.txt to determine if Amazonbot (Amazon's AI crawler) is blocked, helping ensure visibility to Amazon's AI search features.
Checks a website's robots.txt to determine if Applebot-Extended (Apple's AI crawler) is blocked, helping ensure visibility to Apple's AI services.
Reads a Cloudflare zone's AI bot protection settings and robots.txt management via the Cloudflare API, enabling audit of AI-bot blocking toggles and identification of misconfigurations.
Checks a website's robots.txt to determine if Google-Extended (Google's AI crawler) is blocked, helping ensure visibility to Google AI Overviews and other AI features.
Checks a website's robots.txt to determine if meta-externalagent (Meta's AI crawler) is blocked, helping ensure visibility to Meta's AI search capabilities.
Checks a website's robots.txt to determine if PerplexityBot (Perplexity's AI crawler) is blocked, helping ensure visibility to Perplexity's answer engine.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@ai-visibility-mcp check visibility for example.com"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An MCP server for checking and improving a website's visibility to AI crawlers and answer engines (ChatGPT, Claude, Perplexity, Google AI Overviews, etc.) against your own properties.
Most "AI SEO" advice stops at "add an llms.txt." In practice the bigger and more common problem is upstream of that: robots.txt — often auto-managed by a CDN like Cloudflare — silently blocking the exact crawlers you want to reach, which makes any llms.txt or content work moot. This server checks both, plus can audit and generate the fix.
check_visibility(url) — fetches a site's live robots.txt and flags any named AI bot (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, Applebot-Extended, CCBot, Bytespider, Amazonbot, meta-externalagent, and others) blocked with Disallow: /, or a wildcard block. Also checks whether llms.txt exists as a real file versus falling through to a SPA catch-all route (a common false-positive: the file "200s" but it's actually your homepage).
cloudflare_ai_bot_audit(domain) — reads a Cloudflare zone's Bot Management AI-bot settings (ai_bots_protection, ai_training, ai_search, ai_user, is_robots_txt_managed) for a domain you control. Requires CLOUDFLARE_API_EMAIL + CLOUDFLARE_API_KEY (Global API Key) env vars.
generate_llms_txt(name, description, url, phrase?) — generates an llms.txt line and, if you supply an ownable niche phrase, a small content pack (meta description + FAQ block) anchored on that phrase. Pure text generation — no filesystem or git writes.
Related MCP server: ai-visibility-mcp
git clone https://github.com/tsushanth/ai-visibility-mcp.git
cd ai-visibility-mcp
npm install
npm run buildAdd to your MCP client config (e.g. Claude Code, Claude Desktop):
{
"mcpServers": {
"ai-visibility": {
"command": "node",
"args": ["/absolute/path/to/ai-visibility-mcp/dist/index.js"],
"env": {
"CLOUDFLARE_API_EMAIL": "you@example.com",
"CLOUDFLARE_API_KEY": "your-global-api-key"
}
}
}
}The Cloudflare env vars are only needed for cloudflare_ai_bot_audit — check_visibility and generate_llms_txt work with no configuration.
Sites on Cloudflare frequently have AI-crawler blocking turned on by default (the "Block AI Bots" toggle), which rewrites the served robots.txt to disallow GPTBot, ClaudeBot, Google-Extended, and others — even when the site owner never explicitly decided that and actually wants AI-search visibility. This is easy to miss because the site still looks and functions normally to human visitors and to regular search engines; only AI crawlers are silently turned away. check_visibility surfaces this from the outside (what's actually being served) and cloudflare_ai_bot_audit confirms it from the account side (what Cloudflare's settings say), so you can tell the two apart.
MIT
Check a site's actual AI-crawler visibility: fetch its live robots.txt and look for Disallow rules blocking named AI bots (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, etc.) or a wildcard block, plus whether llms.txt actually exists as a real file (vs. falling through to a SPA catch-all).
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Site URL or bare domain, e.g. example.com |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It explains the tool fetches a live resource and checks for specific bot names, which implies it's a read-only network operation. However, it doesn't disclose that it may make external requests, handle timeouts, or fail on invalid URLs. Given no annotations, the description does some work but misses key behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, well-structured, and front-loaded with the tool's purpose. It is concise but covers the main actions without unnecessary detail. It earns its place but could be slightly more efficient in phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description is reasonably complete. It explains what the tool fetches, what specific checks it performs (named bots, wildcard block, llms.txt existence), and the nuance of SPA catch-all. It doesn't explain the return format, but given the lack of an output schema, a 4 is appropriate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers the single parameter (url) 100%, and the description adds meaning by explaining the parameter is a site URL or bare domain (e.g., example.com). The description also provides typical usage context (robots.txt fetch and llms.txt check), which goes beyond the schema's basic description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool checks a site's AI-crawler visibility by fetching robots.txt and looking for Disallow rules for named AI bots and a wildcard block, and verifies llms.txt existence. It uses a specific verb (Check) and resource (site's AI-crawler visibility), distinguishing it from siblings like cloudflare_ai_bot_audit or generate_llms_txt.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on what the tool does (checking robots.txt and llms.txt). It doesn't explicitly state when NOT to use it or mention alternatives, but the sibling tools (cloudflare_ai_bot_audit and generate_llms_txt) imply different use cases. The description is sufficient for an agent to decide when to invoke this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read a Cloudflare zone's Bot Management AI-bot settings (ai_bots_protection, ai_training, ai_search, ai_user, is_robots_txt_managed) for a domain. Requires CLOUDFLARE_API_EMAIL and CLOUDFLARE_API_KEY env vars (Global API Key, X-Auth-Email/X-Auth-Key auth).
| Name | Required | Description | Default |
|---|---|---|---|
| domain | Yes | Root domain as registered in Cloudflare, e.g. example.com |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behavior. It explicitly states 'Read' (non-destructive), which is the most critical trait, and additionally documents required environment variables and authentication method. It does not elaborate on error cases or return format, but for a simple read operation this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, each earning its place: the first states the core purpose and scope, the second specifies authentication requirements. No filler or redundant content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with one parameter and no output schema, the description is largely complete: it lists the settings read, the domain, and the auth prerequisites. It does not describe the return structure, but the absence of an output schema makes that less critical. Minor gaps like error handling or zone-not-found behavior keep it from a 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of the single parameter (domain), giving its format and example. The description adds no new parameter semantics beyond what the schema already provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Read a Cloudflare zone's Bot Management AI-bot settings') and enumerates the exact settings covered. The resource (Cloudflare zone) and the tool's focus are unambiguous, naturally distinguishing it from sibling tools despite not naming them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by stating what it reads and for which domain, but it does not provide explicit guidance on when to use this tool over alternatives, nor any exclusions. There is no mention of when not to use it or how it relates to sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate an llms.txt line plus a content pack (ownable phrase, llms.txt sentence, meta description, FAQ block) for a product, ready to hand-place on a landing page or drop as a file. Does not write files or touch git — pure text generation.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Canonical URL | |
| name | Yes | Product name | |
| phrase | No | Ownable niche phrase to anchor on, e.g. "one-click autofill for Greenhouse, Lever, and Ashby job applications". If omitted, only llms.txt is generated (no content pack). | |
| description | Yes | One-sentence plain description of what it does |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries the burden of behavioral transparency. It clearly states that the tool is pure text generation (no file writing, no git interaction), which is helpful. However, it doesn't disclose potential limits, response format, or whether generation is deterministic or uses external services.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no wasted words. It front-loads the main purpose, specifies the output components, and clarifies constraints in a concise manner.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters with 100% schema coverage, no output schema, and no annotations, the description provides a clear functional overview. However, it could be more complete by mentioning that the output includes the generated text (since no output schema exists) and whether the FAQ block is a fixed format or customizable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds value by explaining that the 'phrase' parameter is optional and that omitting it triggers only llms.txt generation (no content pack), which is not stated in the schema. This clarifies behavior beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool generates an llms.txt line plus a content pack (ownable phrase, llms.txt sentence, meta description, FAQ block) for a product, distinguishing it from siblings like check_visibility and cloudflare_ai_bot_audit by focusing on text generation for SEO/content, not visibility checks or bot audits.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies when to use the tool (to generate content for a landing page or file drop) and what it does not do (does not write files or touch git), providing clear context. However, it doesn't explicitly exclude alternatives or mention when not to use this tool versus other content generation tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
v0.1.0check_visibilitycloudflare_ai_bot_auditgenerate_llms_txtEach tool has a clearly distinct purpose: one checks live robots.txt and llms.txt, one audits Cloudflare's specific AI bot settings, and one generates llms.txt content. There is no overlap or ambiguity.
All names use snake_case and include a verb, but check_visibility and generate_llms_txt follow a verb_noun pattern while cloudflare_ai_bot_audit places the verb at the end of a compound noun. This slight inconsistency is minor and does not hinder readability.
Three tools is a reasonable number for a focused domain like AI crawler visibility. The scope is narrow enough that each tool earns its place without feeling sparse or excessive.
The tool set covers the core workflows: checking visibility, auditing a specific provider's settings, and generating content. However, it lacks support for other hosting providers beyond Cloudflare and does not include any write/modify capabilities, which may be expected in a visibility management server but is likely intentional as an analysis-only tool.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
Checks llms.txt, AI crawler access in robots.txt, and sitemap - with a 0-100 AI readiness score.
Check website AI-readiness: Schema.org, llms.txt, E-E-A-T, robots.txt. Works in Cursor & Claude.
Generate 18 AI readiness files (llms.txt, ai.txt, RAG indexes, schema) for any website.
Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/tsushanth/ai-visibility-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
