AI Crawler Index

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is established. The description adds genuinely valuable behavioral context beyond those annotations: cursor advancement semantics ('advances only on a real change'), data staleness ('refreshes every 6h'), and first-call behavior. This exceeds what the structured annotations alone convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense with purposeful information, front-loading the most critical fact ('FIRST CALL, needs nothing') and an actionable first-step example. The embedded JSON-RPC whoami invocation adds bulk but serves as concrete guidance rather than filler. No sentence is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a cursor-based incremental sync tool, the core protocol is well covered: first-call behavior, cursor echo-back, advancement semantics, change-type payloads, and data freshness. The main gaps are the absence of an output schema (return shape is unspecified) and the unaddressed relationship to whats_changed, but for a read-only, idempotent, 2-parameter tool this is largely complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% — both 'limit' and 'since' are fully described in the input schema, so the baseline is 3. The description reinforces the cursor semantics with the since='0' example, which adds marginal clarity, but it does not add new parameter-level detail beyond what the schema already documents.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb and resource: returns what changed since the last call, enumerating concrete change types (prefixes added/removed, upstreams failed/recovered, records edited). However, it does not differentiate itself from the sibling tool 'whats_changed', which sounds nearly identical in scope, so it misses the sibling-distinction bar for a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear operational context: 'FIRST CALL, needs nothing', send back the cursor from the previous result, cursor advances only on a real change, data refreshes every 6h, example since='0'. But it offers no explicit when-to-use versus alternatives — notably it never addresses why one would choose changes_since over the similarly-named sibling whats_changed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive behavior, so the bar is lower. The description adds meaningful context: the tool outputs verification guidance, reflects blocking costs, and can be called via the same implementation over HTTP. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The core description is useful, but it is front-loaded with a lengthy JSON-RPC example for calling whoami, a different sibling tool. That embedded request is noise for an agent trying to understand classify_user_agent and consumes space that should describe this tool. The remaining content is otherwise efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter, read-only tool with no output schema, the description covers inputs, outputs, an example, a verification alternative, and even a non-MCP fallback endpoint. Nothing essential for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds value beyond the schema by giving a concrete example ('GPTBot/1.2'), showing what the parameter should contain ('raw User-Agent header value'), and illustrating the expected output mapping. This clarifies parameter usage without inventing ambiguity.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Identify the crawler behind a raw User-Agent header' and enumerates the exact output dimensions (operator, category, robots.txt stance, verification method, blocking cost). It also distinguishes itself from the sibling is_verified_crawler_ip by positioning that tool as the confirmation step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear contextual guidance: a User-Agent is only a claim, so results should be confirmed with is_verified_crawler_ip. It does not explicitly compare against lookup_crawler or list_crawlers, but the specific use case of classifying a raw UA string is unambiguous and the named alternative covers the main follow-up path.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, it adds concrete behavioral details: deterministic, read-only, touches no third party, requires no API key or account, uses only the host's own canned data, and returns a real structuredContent rather than a mock. This strongly supports safe agent invocation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded but highly repetitive: 'no third party' and 'server's own data' are stated multiple times, and several sentences restate the same idea. It could be reduced to two or three crisp sentences without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-argument, read-only example tool with an output schema, the description covers everything needed: what it does, why to call it, what it returns, and that it is safe and deterministic. Nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the description explicitly states 'Takes no arguments' and shows arguments={}. There is no parameter ambiguity, and the schema already covers this completely.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool runs this server's own worked example by invoking classify_user_agent on canned input and returning the real structuredContent. It distinguishes itself from sibling tools by being a safe, no-argument demo rather than a live classification or lookup call.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells the agent to use it to see the shape of an answer before sending real data. It does not enumerate formal exclusions or alternatives, but the intended usage context is clear and actionable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds useful behavior beyond annotations: the output is 'ready-to-paste', includes reasoning and the crawlers it names, and omitting the argument returns all stances. There is no contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description front-loads a long JSON-RPC snippet for whoami before the actual purpose statement, making it noisy and poorly structured. The core explanation is economical, but the unrelated prefix wastes characters and could mislead an agent scanning the description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-optional-parameter read-only tool, the description covers the prerequisite call, what the tool returns, the no-argument behavior, and an example; the schema fills in the exact stance values. The main gap is not explaining why the whoami call is needed, but this is minor given the tool's simplicity and annotation coverage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the stance property lists all allowed values, so the schema carries most parameter meaning. The description adds the optional no-arg behavior and an example, reinforcing semantics beyond the schema. This earns a small bonus over the baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool produces 'a ready-to-paste robots.txt block for one stance' with reasoning and named crawlers, so the verb and resource are specific. The opening JSON-RPC snippet for whoami is confusing but does not obscure the core purpose. It does not explicitly distinguish from siblings, but the task is distinct enough from the crawler-lookup tools nearby.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a concrete prerequisite by instructing a 'FIRST CALL' to whoami with no arguments, and explains the omit-argument behavior to get all eight stances. It also provides a concrete example stance. It lacks explicit when-to-use vs alternative tools, but for a simple generator the guidance is adequate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the safety profile (readOnlyHint, idempotentHint, non-destructive), so the bar is lower and credit goes to added context. The description discloses what the call returns (matching prefix, source, crawlers), the meaningful semantic that a miss is 'reported as a miss, not as a forgery', a six-hour data refresh window, and that the MCP call wraps the same implementation as a public HTTP endpoint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description squanders its front-loaded position on a full inline JSON-RPC payload for a different tool (whoami) before stating what this tool does. The remaining content is reasonably tight and informative, but the whoami invocation should have been a one-line hint rather than a verbatim call payload.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-required-parameter, read-only check tool with no output schema, the description covers purpose, return semantics, miss handling, data freshness, a worked example, and an alternate invocation path. Nothing an agent needs to call it correctly is missing; only minor extras like rate limits or malformed-IP handling are absent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the schema already documents ip as 'IPv4 or IPv6 address to check'. The description adds only a concrete example value (20.171.206.5) and reaffirms IPv4/IPv6 support already present in the schema, so the baseline 3 for full schema coverage is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The operative sentence 'Test an IPv4 or IPv6 address against every crawler prefix the operators themselves publish' is a specific verb+resource statement that clearly distinguishes this from siblings like lookup_crawler and list_crawlers. However, the description opens with an instruction to call whoami, a different tool, so the very first content an agent reads is not about this tool's own purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The only usage guidance is a workflow hint ('FIRST CALL, needs nothing' → call whoami), a concrete example parameter value, and an alternative HTTP transport. There is no explicit statement of when to use this tool versus the sibling lookup/classification tools, and no when-not-to-use or exclusion criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only and idempotent behavior; the description adds that the call requires no arguments, returns per-row crawler/blocking-cost data, and exposes all valid filter values. However, the embedded whoami JSON-RPC payload is confusing and no pagination/ordering behavior is disclosed, so the added behavioral context is only partial.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The core description is brief, but front-loading a full JSON-RPC payload for whoami before describing list_crawlers is distracting and does not earn its place. The actual tool purpose appears only after the embedded call, making the structure confusing rather than concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only list tool with all optional parameters, the description covers the output shape (crawler identity + blocking cost), zero-argument invocation, and filter value discoverability. Missing details like result ordering, pagination behavior, and how filters combine are not addressed, but the annotations and schema cover much of the safety and parameter context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3 and the schema carries parameter documentation. The description adds a concrete example (category='ai-training', limit=20) and hints that valid filter values are discoverable from the reply, which is helpful but not substantial beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description positions the tool as 'the index, filtered or whole' and specifies that each row identifies a crawler and its blocking cost, which clearly conveys a listing/filtering purpose. It does not explicitly name sibling tools or edge cases, but it is unambiguous enough to separate from a single-record lookup.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says 'FIRST CALL, needs nothing', signaling this is a safe zero-parameter starting point, and notes that the reply lists every valid filter value, so an agent knows to use it for filter discovery. It does not state when to prefer lookup_crawler or whoami instead, so it stops short of a full routing rule.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds useful behaviors beyond the readOnly/idempotent annotations: it discloses that an unknown slug returns near matches and that the record includes blocking cost and verification method. This gives the agent expectations for edge cases and output richness. No annotation contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with a long JSON-RPC block about calling whoami, which is irrelevant to this tool and constitutes wasted space. The remaining field list is dense and informative, but the confusing preamble reduces clarity and structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by listing all major return fields (operator, category, robots stance, user-agent example, verification, IP-range endpoint, blocking cost) and the near-match fallback. It is concise enough for a single-slug lookup, although the purpose of the whoami first call is left unexplained.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents slug with 100% coverage; the description reinforces it with a concrete example (slug='claudebot') and clarifies handling of unknown slugs. It does not redefine the parameter type but adds practical usage context, which is helpful.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool returns the full index record for one crawler and enumerates the fields. This distinguishes it from list_crawlers by focusing on a single crawler. However, the opening 'FIRST CALL, needs nothing' JSON-RPC to whoami introduces noise and could mislead about the tool's purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit guidance on when to choose this tool over siblings like list_crawlers or classify_user_agent. It implies singular lookup context but does not state exclusions or alternatives. The 'FIRST CALL' instruction suggests a workflow without explaining why, so an agent gets little routing help.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description adds useful behavioral context about what state is reported: freshness, failures, and changelog. No contradiction exists between the description and annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is brief, but it front-loads a full JSON-RPC example for `whoami`, which is a different tool and irrelevant to understanding `whats_changed`. This makes the opening confusing and poorly focused. The actual purpose appears after that clutter, and the useful since-example is at the end.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with one optional parameter, strong annotations, and no output schema, the description adequately covers what the tool returns and when to invoke it. It lists the key pieces of information reported and gives a usage example. The irrelevant `whoami` example hurts clarity but does not leave a critical gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides 100% coverage by describing `since` as an ISO-8601 date/timestamp and stating to omit it for current state. The description adds only an example value (`since='2026-08-01'`) and the note that nothing is needed, which is helpful but not substantial beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states what the tool reports: index build time, last fetch times for operator sources, changed/failing sources, and a dated changelog. The phrase 'The cron check before regenerating a config from this data' further defines its role. It does not explicitly distinguish itself from the sibling `changes_since`, but the description gives enough functional specificity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description says this is the cron check to run before regenerating a config from this data, which is a concrete use case. It also says 'FIRST CALL, needs nothing,' indicating it can be invoked without parameters. However, it does not mention when not to use it or how it compares to alternatives like `changes_since`.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint and idempotentHint, and the description goes well beyond them by disclosing that the tool touches no third party, fetches nothing, is deterministic, requires no API key/account/consent, and derives every fact from request headers or an already-published file. There is no contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is far longer than needed and repeatedly restates the same output fields ('user-agent, address, class, seen before') and the same reassurances ('free, read-only, no API key, no account, no consent prompt'). It opens with a full JSON-RPC body instead of a concise summary, making it harder to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-argument introspection tool, nothing is missing: it explains what the tool returns, where the data comes from, that no side effects occur, and how it relates to the server's classifiers. An output schema exists, and the description is consistent with it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With zero parameters and 100% schema coverage, there is almost nothing to explain. The description still adds useful confirmation by saying 'Takes no arguments,' 'no argument exists,' and giving an explicit arguments={} example, so the agent will not invent parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as a no-argument, read-only introspection call: it 'describes YOU, the caller' and returns the caller's user-agent, address, classification, and seen-before status. It stops short of explicitly distinguishing itself from sibling tools classify_user_agent and is_verified_crawler_ip, though it references them as internal building blocks.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The appropriate context is implied: use this when you want information about the current caller rather than classifying a supplied user-agent or IP, and the description stresses 'Takes no arguments' and 'nothing for you to invent.' However, it never explicitly says when to prefer this over the sibling tools or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.