Skip to main content
Glama
Bigsupe55

geo-inspector-mcp

by Bigsupe55
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Remote

geo-inspector-mcp

npm MCP Registry license

An MCP server that gives an AI assistant four tools for inspecting how a website presents itself to other AI systems: which AI crawlers it blocks, whether it publishes llms.txt, what schema markup it ships, and how its indexing directives are set.

Published on npm and listed in the official MCP Registry as io.github.Bigsupe55/geo-inspector-mcp. Works with Claude Code, Claude Desktop, or any MCP client.

geo-inspector-mcp inspecting three sites

Tool output above is verbatim from a live run against nytimes.com, docs.anthropic.com, and stripe.com, captured by driving the built server over MCP stdio. The sitemap list is collapsed to a count; nothing else is edited. Regenerate with npm run demo.

Why this exists

AI assistants are becoming a primary way people find and cite content, and sites signal their intent to AI systems through a handful of plumbing files: robots.txt rules for AI crawlers, the emerging llms.txt standard, schema.org structured data, and meta directives. Checking those by hand means juggling curl, a robots.txt parser in your head, and view-source. This server turns all of it into questions you can just ask Claude.

Related MCP server: aeo-mcp

Quickstart

npx -y geo-inspector-mcp

That is the whole install. Point your MCP client at it:

Claude Code

claude mcp add geo-inspector -- npx -y geo-inspector-mcp

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "geo-inspector": {
      "command": "npx",
      "args": ["-y", "geo-inspector-mcp"]
    }
  }
}

Then ask things like: "Which AI crawlers does nytimes.com block?" or "Does stripe.com publish an llms.txt?"

Tools

Tool

What it checks

Example question

check_robots_txt

Which AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, and more) are allowed or blocked, per RFC 9309, plus sitemaps

"Can OpenAI train on example.com?"

fetch_llms_txt

Presence and spec-validity of /llms.txt and /llms-full.txt

"Has example.com adopted llms.txt?"

detect_schema_markup

JSON-LD blocks, schema.org type inventory, AI-relevant types, sameAs disambiguation

"What structured data does this article have?"

check_meta_directives

Meta robots tags (including noai/noimageai and bot-specific tags) and X-Robots-Tag headers

"Is this page indexable?"

Every tool returns a readable summary plus structured JSON (structuredContent) for programmatic use.

How it is built

The interesting part of an MCP server is not the tools, it is the contract around them.

Every tool returns two things. A readable summary for the model to reason over, and structuredContent for anything downstream that needs to compute. That split is what lets a separate scoring layer (ai-visibility-audit) derive deterministic numbers from the same call the model is reading in prose. The model never has to parse its own tool output back into data.

Parsers are pure functions. robots.txt, llms.txt, JSON-LD, and meta directives each parse in isolation, with no I/O and fixture-based tests. A tool handler is a thin shell: fetch, parse, format. This is what makes the behavior testable without a network, and it is why the test suite runs with no fixtures to record and no site to hit.

All network access goes through one helper. A single fetch path with a size cap, a redirect limit, and a timeout. An MCP server runs inside someone else's agent loop with their API budget attached, so a tool that can hang or stream an unbounded response is a tool that can ruin a session. One choke point means those limits cannot be forgotten in a new tool.

robots.txt parsing follows RFC 9309 rather than a regex, because the whole value of the tool is being right about whether a specific crawler is allowed. Longest-match wins, user-agent groups merge, and Allow can override a broader Disallow.

Development

npm install
npm test        # vitest unit + integration tests
npm run build   # bundle to dist/
npx @modelcontextprotocol/inspector node dist/index.js   # poke it interactively

Parsers are pure functions with fixture-based tests; all HTTP goes through one capped, redirect-limited fetch helper.

Regenerating the demo GIF

npm run demo          # build, capture, render

Or the steps separately:

npm run demo:capture  # drives the built server over MCP stdio, writes scripts/demo-data.json
npm run demo:render   # renders docs/demo.gif from that JSON (needs: pip install Pillow)

demo-data.json is committed, so rendering works offline and the GIF's claims stay auditable: diff it against what the tools return today. The capture script mocks nothing, so if a site changes its robots.txt, the demo changes with it. The only authored text in the pipeline is the human question line; the sites to inspect are configured at the top of scripts/capture-demo.mjs.

Related

ai-visibility-audit is the workflow layer on top of this one: it orchestrates these four tools into a scored, client-ready report. This repo is the raw tools.

License

MIT, see LICENSE.

Available Tools

4 tools
check_meta_directivesCheck indexing and AI meta directivesA

Report indexing directives from meta robots tags (including bot-specific and noai/noimageai tags) and X-Robots-Tag response headers for a page.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesWebsite URL. Scheme is optional, example.com works.

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It describes what the tool reports but does not disclose behavioral traits like read-only nature, error handling, or authentication requirements. It is adequate but lacks depth for a standalone description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that is front-loaded with the core action ('report indexing directives') and includes specific details. Every word earns its place, with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with no output schema, the description covers the main functionality. It could mention what happens on error or hint at output format, but overall it is reasonably complete for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single 'url' parameter, which already documents its type and format. The tool description adds no additional meaning beyond the schema, so the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: reporting indexing directives from meta robots tags and X-Robots-Tag headers. It specifically mentions bot-specific and noai/noimageai tags, distinguishing it from siblings like check_robots_txt (site-level), fetch_llms_txt, and detect_schema_markup.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for checking page-level meta directives but provides no explicit when-to-use or when-not-to-use guidance. It does not mention alternatives or exclude scenarios, leaving the agent to infer from siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

check_robots_txtCheck AI crawler access in robots.txtA

Fetch and parse a site's robots.txt per RFC 9309, then report which AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, and more) are allowed or blocked for a given path. Also lists sitemaps.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesWebsite URL. Scheme is optional, example.com works.
pathNoPath to evaluate access for. Defaults to /

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses that it fetches and parses per RFC 9309 and reports allowed/blocked crawlers plus sitemaps, but does not mention error handling (e.g., missing robots.txt), rate limiting, caching behavior, or whether it respects crawl-delay directives.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, each serving distinct purpose: first describes the core action and scope, second adds the sitemap feature. No filler words, and the most important information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple fetch-and-parse tool with no output schema, the description provides enough context: it reports which AI crawlers are allowed/blocked and lists sitemaps. It lacks details on output format (e.g., JSON vs text) but is otherwise adequate for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already describes both parameters. The description adds no extra meaning beyond the schema (e.g., it does not clarify URL format or path default behavior beyond what the schema states). Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb (fetch, parse, report), the resource (site's robots.txt), and the specific scope (AI crawler access). It explicitly lists AI crawlers and notes sitemaps, which distinguishes it from sibling tools that deal with LLMs.txt, schema markup, or meta directives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus the siblings (fetch_llms_txt, detect_schema_markup, check_meta_directives). It does not mention prerequisites, limitations, or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

detect_schema_markupDetect JSON-LD schema markupA

Extract JSON-LD structured data from a page and inventory the schema.org types found. Flags the types that matter for AI discoverability (Organization, WebSite, Article, FAQPage, BreadcrumbList, Person) and sameAs entity disambiguation.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesWebsite URL. Scheme is optional, example.com works.

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without any annotations, the description effectively communicates the tool's behavior: extracting JSON-LD, inventorying types, flagging important ones, and performing sameAs disambiguation. It is transparent about its functionality, though it doesn't mention edge cases or error handling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two concise sentences. The first sentence covers the core function, and the second adds important specifics about flagged types and disambiguation. No extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description explains that the tool extracts JSON-LD and provides an inventory of types with flags. This covers the main output expectation. However, it could be more explicit about the return format or what happens when no JSON-LD is found. Still, it's sufficiently complete for a single-parameter tool without annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides 100% coverage for the single parameter 'url' with a clear description. The tool description adds context that the URL points to a page containing JSON-LD, but this is inherent from the tool's purpose. No additional parameter semantics beyond the schema are needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool extracts JSON-LD structural data and inventories schema.org types, specifically naming the relevant types for AI discoverability. This distinguishes it from sibling tools like check_robots_txt or fetch_llms_txt which handle other SEO aspects.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is used to inspect schema markup on a page but does not explicitly state when to use it versus alternatives. No exclusion criteria or specific context is provided to guide tool selection among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_llms_txtFetch and validate llms.txtA

Check whether a site publishes /llms.txt and /llms-full.txt, and validate llms.txt structure against the llmstxt.org spec (H1 title, summary blockquote, H2 link sections).

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesWebsite URL. Scheme is optional, example.com works.

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description only states what it checks and validates. It does not disclose return format, error handling, rate limits, or authentication needs, which are important for a query tool without output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence with clear action and scope. No fluff, front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description covers the core functionality. However, it could mention expected return structure or error cases for completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter with full schema description coverage. The description does not add information beyond what the schema already states, meeting the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool checks for the existence of /llms.txt and /llms-full.txt and validates structure against a spec, distinguishing it from sibling tools like check_robots_txt.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use versus alternatives or when not to use. The context is implied but not explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv0.1.2
    • First observedcheck_meta_directives
    • First observedcheck_robots_txt
    • First observeddetect_schema_markup
    • First observedfetch_llms_txt
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4/5.0

Scored across 4 tools

Disambiguation5/5

Each tool targets a distinct aspect of website metadata for AI discoverability: robots.txt, llms.txt, schema markup, and meta directives. No functional overlap.

Naming Consistency5/5

All tool names follow a clear verb_noun pattern: check_robots_txt, fetch_llms_txt, detect_schema_markup, check_meta_directives, with consistent snake_case.

Tool Count5/5

4 tools is an ideal size for this focused domain—each tool addresses a necessary and distinct inspection task without redundancy or clutter.

Completeness4/5

The set covers the key AI-related metadata surfaces: robots.txt, llms.txt, schema.org markup, and meta robots directives. A minor gap is direct sitemap inspection, though sitemaps are referenced in robots.txt.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • maxaeo-ai-visibility-mcp

    maxaeo
    A
    license
    A
    quality
    B
    maintenance
    Enables AI agents to check whether a public website is crawlable, understandable, and ready for AI search workflows through local-only audits of robots.txt, sitemaps, metadata, and llms.txt.
    3
    50 npm
    1
    MIT

  • aeo-mcp

    arthurcarlsonn
    A
    license
    A
    quality
    D
    maintenance
    An MCP server that gives AI agents tools to inspect a website's visibility to AI answer engines, including crawler permissions, llms.txt, structured data, on-page signals, and a full 29-check AI-readiness audit.
    5
    37 npm
    MIT

  • growth-mcp

    abhi725
    A
    license
    Not graded
    quality
    C
    maintenance
    Open-source MCP server that audits websites for AI search readiness, providing deterministic scoring (0-100) and prioritized fix lists for metrics like JSON-LD, llms.txt, heading hierarchy, and AI crawler access.
    1
    MIT

  • ai-readiness

    silverbackmarketing
    F
    license
    Not graded
    quality
    C
    maintenance
    Generates a complete suite of AI readiness files (llms.txt, ai.txt, schema, RAG indexes) for any website to optimize representation in ChatGPT, Claude, Gemini, and Perplexity.
    3
    -