Skip to main content
Glama

Server Details

Cited AI crawler registry (training, search or user fetch) plus robots.txt checks. No fetching.

If you are the author of this connector, you can claim ownership by verifying the domain or GitHub account it belongs to. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Healthy
Uptime
100.0% over the last 24 hours
Last Tested
Transport
Streamable HTTP · MCP 2025-06-18
URL
ChatGPT
–
Claude
–

TDQS

A4.2/5.0

Scored across 4 tools

Disambiguation5/5

Each tool has a clearly distinct verb+object purpose: list_crawlers enumerates tokens, get_crawler returns one card by id, check_robots analyzes a pasted robots.txt, and suggest_robots generates a new robots.txt. The inverse pair check/suggest could theoretically be swapped, but the descriptions make the direction of each operation unambiguous.

Naming Consistency5/5

All four tools follow the same snake_case verb_noun pattern (check_robots, get_crawler, list_crawlers, suggest_robots). No deviation in style or casing, making the surface highly predictable.

Tool Count5/5

Four tools is well within the appropriate range for a narrow, read-only dataset server. Each tool earns its place by covering a distinct operation (browse, detail, analyze, synthesize) without redundancy.

Completeness4/5

The read-only lifecycle is well covered: enumerate tokens, fetch details, validate an existing robots.txt, and synthesize a new one, with no create/update/delete needed for a reference dataset. Minor gaps remain, such as filtering or searching by vendor, but these are workable around via list_crawlers.

Available Tools

4 tools
check_robotsCheck a pasted robots.txtA
Read-onlyIdempotent
Inspect

Match a robots.txt file the caller pastes against Botcard's AI crawler tokens, using RFC 9309. Each row is allow or disallow plus that token's vendor stance. This tool does not fetch any website. Do not send url, host, target, website, robots_url, or fetch.

NameRequiredDescriptionDefault
idsNoCard ids to check, such as gptbot. Omit to check every card.
pathNoPath to test, starting with /. Defaults to /.
robots_txtYesThe robots.txt file contents. Not a URL.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, openWorld=false, and non-destructive, so the safety profile is covered. The description adds genuinely new behavioral context: no network fetch occurs, output is one allow/disallow row per token with the vendor stance, and RFC 9309 governs matching.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three front-loaded sentences with no filler; the core action leads and the anti-fetch constraint follows. The parameter blacklist is slightly long but earns its place by preventing a common wrong invocation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only, 3-param tool with no output schema, the description covers the action, the non-fetching behavior, and the return shape (allow/disallow plus vendor stance). Nothing needed to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so ids, path, and robots_txt are already fully documented in the schema, setting the baseline at 3. The description reinforces the key constraint (robots_txt is contents, not a URL) but adds no syntax or format detail beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Match) and resource (a pasted robots.txt against Botcard's AI crawler tokens) with a named standard (RFC 9309). It is clearly distinct from siblings like get_crawler and list_crawlers, which retrieve crawler data rather than evaluate a pasted file. An agent can select this without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit negative guidance — 'This tool does not fetch any website. Do not send url, host, target, website, robots_url, or fetch' — steers the agent away from the fetch-style misuse that the sibling tools might model. It gives clear context but does not name an alternative tool by name the way a top-tier definition would.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_crawlerGet one crawler cardA
Read-onlyIdempotent
Inspect

Return one card by id, including the vendor source URL and the date it was read. This tool does not fetch any website. Do not send url, host, target, website, robots_url, or fetch.

NameRequiredDescriptionDefault
idYesCard id, such as gptbot or google-extended.

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false; the description reinforces this by clarifying it performs no network fetch and returns stored card data. That adds useful context beyond the annotations, though it says nothing about auth needs, error behavior for unknown ids, or pagination.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three compact sentences: the retrieval contract is front-loaded, followed by the negative constraint. No filler, no repetition of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter, read-only lookup with no output schema, the description covers both the lookup key and the shape of the returned card (source URL, read date), and rules out misuse modes. Nothing needed to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the id parameter is already documented with concrete examples (gptbot, google-extended), so the description only restates 'by id'. The negative enumeration of forbidden parameters is the one addition, but no id syntax, format constraints, or prefix conventions are supplied beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specifies the verb (return), the resource (one card), and the retrieval key (by id), plus what the payload contains (vendor source URL and read date). An agent can distinguish this single-card reader from list_crawlers without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states what this tool does NOT do ('does not fetch any website') and enumerates parameters it must not receive (url, host, target, website, robots_url, fetch) — a strong exclusion list. However, it never names the sibling tool an agent should call instead for fetching or listing, so routing guidance is implied rather than complete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_crawlersList crawler cardsA
Read-onlyIdempotent
Inspect

List the AI crawler robots.txt tokens in this dataset. Each row names the vendor page the summary was checked against. This tool does not fetch any website. Do not send url, host, target, website, robots_url, or fetch.

NameRequiredDescriptionDefault

No parameters

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world. The description adds genuine behavioral context beyond them: no network fetch occurs, and each row carries the vendor page the summary was checked against, which tells the agent what kind of data comes back.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, purpose front-loaded, then return shape, then constraint. Each sentence earns its place; the middle sentence about the vendor page is slightly awkwardly worded but still informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, read-only list tool with no output schema, the description covers what it returns (crawler tokens, one per row with a vendor reference) and what it will not do (fetch). Sufficient for correct invocation, though more detail on row fields is absent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With zero parameters the baseline is 4, and the description adds useful value by explicitly forbidding invented arguments (url, host, target, website, robots_url, fetch) that an agent might otherwise hallucinate for a crawler-related tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List the AI crawler robots.txt tokens in this dataset') and scopes it to the local dataset, which implicitly separates it from the fetching siblings. It does not name check_robots or get_crawler explicitly, so the differentiation is left partly to inference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'This tool does not fetch any website' plus the explicit do-not-send list clearly signals the usage context: read a static local list rather than probe a URL. The alternative (check_robots) is only implied, not named, so the routing guidance is good but not airtight.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

suggest_robotsWrite a cited robots.txtA
Read-onlyIdempotent
Inspect

Write a robots.txt from this dataset for the kinds you allow or block. Each group cites the vendor page. effect request_only means the vendor does not say the crawler obeys that line. control_token is not training. The file has no User-agent: * group. This tool does not fetch any website. Do not send url, host, target, website, robots_url, or fetch.

NameRequiredDescriptionDefault
pathNoPath the Allow or Disallow names. Defaults to /.
allowNoKinds to Allow, such as search.
blockNoKinds to Disallow, such as training.
policyNoSame choice in words, such as "allow search, block training".

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/no-destructive/idempotent/no-open-world, but the description adds real behavioral context: no network fetch occurs, the emitted file omits a User-agent: * group, and each group cites the vendor page. It never explains the tension between the verb "Write" and readOnlyHint (no external state is mutated), which is the one thing left implicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded and there is little filler, but the text is a chain of clipped fragments ("effect request_only means the vendor does not say the crawler obeys that line.", "control_token is not training.") that are hard to parse without prior domain knowledge. It is terse rather than well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-output-schema tool with four optional params and solid annotations, the description covers output shape (no wildcard group, cited vendor pages), side effects (no fetching), and misuse constraints. It is slightly short on how `policy` interacts with `allow`/`block` when both are supplied.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description does add a little meaning ("control_token is not training", the negative list of fields not to send), but its "effect request_only" terminology does not match any enum value (the enum has owner_requested), which adds confusion rather than clarity.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb and resource ("Write a robots.txt") plus the scope ("for the kinds you allow or block") and the distinguishing output trait (cited groups). An agent can tell this apart from check_robots/list_crawlers without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use is implied by the purpose statement, and the closing sentence gives a negative constraint ("Do not send url, host, target, website, robots_url, or fetch"). However, it never says when to choose this over check_robots or what prerequisite data ("this dataset") must exist first.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updates
    • First observedcheck_robots
    • First observedget_crawler
    • First observedlist_crawlers
    • First observedsuggest_robots

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    C
    maintenance
    Checks a site for AI crawlers: which of 11 AI crawlers its robots.txt lets in, whether its llms.txt follows llmstxt.org, and what its home page says without scripts. Findings with evidence; no scores, no model calls, no telemetry.
    5
    79 npm
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    Audits AI-bot visibility: robots.txt per-bot for 22 AI user-agents (GPTBot/ClaudeBot/PerplexityBot/etc), Cloudflare flags, JSON-LD, sitemap, llms.txt, SPA shell, plus cross-model brand mentions via Perplexity + OpenRouter. 0-100 score. SSRF-guarded, spend-capped.
    4
    1
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables AI agents to audit and improve a site's visibility in AI search by checking robots.txt against 17 major AI crawlers, detecting llms.txt, and returning a 0-100 visibility score with per-crawler verdicts; it also lists known AI crawlers and their purposes.
    2
    MIT
Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources