Crawl Census

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full disclosure burden. It does this well by detailing what data is included, that the context is about network-edge behavior regardless of robots.txt, and that correction/dispute channels are part of the result. It still omits minor behavior details like whether data is cached, read-only guarantees, or response shape, but the core behavioral scope is transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single dense but understandable passage; it is front-loaded with the core purpose and adds contextual detail about audience and correction channels. It could be tightened into clearer structured list, but no sentence is truly wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool with no output schema and no annotations, the description does a solid job of painting the result space: coverage, intent, audience, and correction channel. It is not fully complete because it does not describe the output format, time range, or units, but given the simplicity of the tool, the description is adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the agent parameter is already well documented with examples. The description adds context by framing the parameter as representing an AI crawler/user agent, but it does not provide new parameter-level guidance beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies a specific purpose: returning census information about a single AI crawler, including disallow rates, network-edge refusals, documented honoring of robots.txt, and correction channels. It is clear about the resource and intent. However, it does not explicitly distinguish itself from sibling tools like census_facts or census_stats beyond the 'one AI crawler' framing, so it misses full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear intended audience—both the crawler operator and researchers—which implies when this tool is useful. It does not explicitly state when to prefer a sibling tool over this one, nor does it provide exclusion criteria or alternative routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of showing behavior. It discloses that output is a set of records rather than prose, and that each record includes value, unit, denominator, measurement date, source page, citation line, and caveats. It does not explicitly state that the operation is read-only or side-effect-free, though that is strongly implied.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the tool's core behavior, then enumerates the key output fields, and closes with the exact use case and a concrete reason why this tool matters. Every sentence adds meaningful guidance without padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description compensates by describing exactly what the records contain: value, unit, denominator, date, page, citation line, and caveats. With no parameters and a clearly explained use case, the description is complete enough for an agent to call the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and 100% schema description coverage, so there is no parameter detail missing. The baseline for a zero-parameter tool is 4, and the description appropriately focuses on what the output contains rather than input semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly conveys that the tool returns every census headline finding as discrete, dated records with a specified field set, and it gives a concrete use case. It lacks an explicit transitive verb like 'retrieve' or 'list' and does not directly differentiate itself from census_stats, so it stops short of a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives an explicit trigger condition: use this when answering a question about how open the web is to AI crawlers, and explains why simple percentage-lifting is wrong. It does not explicitly state when not to use it or name a preferred alternative, but the context is sufficiently clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden. It clearly states the output stats, but does not mention read-only semantics, data source, freshness, or output format. This is acceptable for a benign stats endpoint but leaves operational behavior somewhat underexposed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one short sentence of about 15 words, front-loaded with 'Corpus-level statistics' and immediately lists all meaningful outputs. No filler or repetition; every phrase adds unique value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, no-output-schema tool, the description covers the primary outputs well, listing blocking counts, mean score, and llms.txt adoption. Minor gaps such as precise return structure, scale/units, and data source would be useful but are not critical given the simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters and schema description coverage is 100%, so the schema fully handles parameter documentation. With no parameters, the baseline of 4 is appropriate and the description needs to add no additional parameter meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states it provides corpus-level statistics and enumerates three specific metrics: blocking counts per AI crawler, mean access score, and llms.txt adoption. This is related to the resource-level context, and 'corpus-level' clearly contrasts with per-domain sibling tools like scan_site, but it does not name the alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when aggregate/corpus-level statistics are needed, as opposed to site-specific functions. However, it never says when not to use it or explicitly points to sibling tools for per-domain data, leaving the routing somewhat implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description fully carries behavioral disclosure. It enumerates the five possible result states define exactly what 'allow/D.disallow/refuse/pay/unknown' mean, including notable edge behavior such as unrecognized agents being resolved from stored robots.txt and HTTP 402 responses being interpreted as commercial access.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but front-loaded, with the core purpose stated first. Each sentence carries important information, though the final sentence partially replays the premise of saving requests and could have been trimmed. Overall, it is a solid level of detail for a tool with no data pre-annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the input schema documents the two parameters and this description defines all result states, the behavior, and the intended user type, the context is complete enough for an agent to invoke the tool correctly even without an output schema. The description and schema together cover the main anonymous request, large-batch limitations, and interpretable return values.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema coverage is high and already documents both parameters, so the baseline is 3. The description adds meaningful semantics beyond that by clarifying that the agent can be unrecognised and handled via robots.txt, and that the return is per-domain, enriching understanding of how 'domains' is used.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: it 'decides whether a crawler may fetch a list of domains' and returns a per-domain verdict. It is clearly distinct from generic scan tools by stating it is 'Built for crawler operators rather than site owners,' though it does not explicitly name or contrast any sibling tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives strong contextual timing guidance: use it 'before spending requests' on domains and warnings that it works for any crawler token. It says it is not for site owners, implying that is the exclusion, but it does not explicitly say which tool to use instead or when not to run the preflight.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations supplied, the description carries the burden of disclosing behavior, and it does so well: it says the tool sends live user-agent probes, reads robots.txt policy for AI crawlers, and returns a 0-100 score with per-check detail. It doesn't spell out timing or failure behavior, but the non-destructive audit nature is reasonably clear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one tight, front-loaded sentence: purpose first, audit checks second, output shape last. Every clause contributes information and no schema detail is unnecessarily repeated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter tool without an output schema, the description adequately covers the input, the overall audit checks, and the result format. The only minor gaps are behavior for unreachable/no-llms.txt domains and potential timeouts, which do not hinder correct tool selection.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already fully documents the only required parameter, domain, including the example 'example.com' and the bare-hostname constraint. Since schema coverage is 100%, the description gains the baseline score of 3 for parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a concrete operation — 'run a live AI-accessibility audit of a domain' — and specifies the exact checks involved: robots.txt policy, live user-agent probes, JavaScript-free readability, structured data, and llms.txt. This is specific enough to distinguish scan_site from siblings like site_report, crawl_preflight, or census_stats.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The first clause clearly identifies the use case: choosing this tool when a live AI-accessibility audit of a domain is needed. However, it does not explicitly name alternatives or state when not to use it, so it earns a 4 rather than a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It transparently discloses the key behavioral trait: it does not cause a new scan and imposes no load on the target site. It could mention missing or stale audit behavior, but the main side-effect and safety risk (accidental scanning) is directly addressed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences with no wasted words. The core return action is front-loaded, and the key differentiator—no new scan, faster, no target load—is stated efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter tool with no output schema and no annotations, the description covers the essential facts: what is returned, the domain scope, and the side-effect-free behavior. It could mention what happens if no stored audit exists, but the invocation-relevant context is largely complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter, domain, is already fully documented in the schema as 'Bare hostname' (100% schema description coverage). The description adds no further parameter-level semantics, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Return the most recent stored audit for a domain.' It clearly distinguishes itself from scan_site by explicitly saying it does not trigger a new scan, so the purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Without triggering a new scan' gives clear context that this is the cached/report tool, with speed and reduced load as rationale. It does not explicitly name scan_site or say when not to use it, but the context strongly implies the correct choice versus a fresh scan.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.