Skip to main content
Glama
apexseo-geo-mcp

apexseo-geo-mcp

AI Crawler, robots.txt & llms.txt Audit ($0.01)

ai_crawler_audit
Read-onlyIdempotent

Audits live robots.txt, llms.txt, sitemap, and agent/MCP files to return per-bot AI crawler permissions, a readiness score, and fixes for crawl-blocking issues.

Instructions

Live-fetches a domain's /robots.txt, /llms.txt, /llms-full.txt, /sitemap.xml, and /.well-known/{x402,mcp,agent-card.json} files, returning a per-bot permission matrix (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended, Bingbot, Applebot-Extended, CCBot), AI-crawler readiness score (0-100), and ICE fixes. Use geo_page_audit for HTML/TTFB signals or schema_citation_check for JSON-LD validation.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "url": "<url>"
    },
    "name": "ai_crawler_audit"
  }
}

Input Schema

NameRequiredDescriptionDefault
urlYesPublic HTTPS/HTTP URL to audit live (e.g. https://example.com).

Output Schema

NameRequiredDescriptionDefault
originYes
llmsTxtYes
summaryYes
robotsTxtYes
wellKnownNo
sitemapXmlNo
iceActionPlanYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=true, so safety is covered. The description adds real context beyond that: it performs a live network fetch against a remote host, hits specific well-known paths, and emits a scoring artifact plus remediation fixes. It stops short of disclosing failure behavior (missing files, 404s, timeouts, rate limits), which keeps it off a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, both front-loaded: capability first, sibling routing second. The long enumeration of paths and bot names is informative rather than filler, though the parenthetical endpoint list is dense enough to slow scanning slightly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be spelled out, and the single parameter is fully documented. Annotations carry the safety profile. The remaining gap is operational edge-case behavior when endpoints are absent or unreachable, which matters for a live-fetch tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single required parameter and schema description coverage is 100%, with the schema already stating 'Public HTTPS/HTTP URL to audit live (e.g. https://example.com)'. The description adds no additional meaning about URL form, domain-vs-path handling, or redirects, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Live-fetches') plus an enumerated resource set (/robots.txt, /llms.txt, /llms-full.txt, /sitemap.xml, .well-known files) and the outputs produced (per-bot permission matrix, 0-100 readiness score, ICE fixes). The named bot list further pins down scope, so an agent can distinguish this from geo_page_audit and schema_citation_check without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The final sentence explicitly routes to alternatives with the condition that selects each: geo_page_audit for HTML/TTFB signals, schema_citation_check for JSON-LD validation. Combined with the front-loaded statement of what this tool inspects, the when-to-use boundary is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.