Whetstone

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden. It transparently states the three categories of information returned (tool catalog, tier boundaries, source). For a simple read-only tool, this is sufficient, though it could mention whether it returns structured data or plain text.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that is front-loaded with 'What this service is' and contains no extraneous words. Every part serves a purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (no parameters, no output schema) and the context of sibling tools, the description is complete enough for an agent to understand its purpose. It could optionally note the format of the output, but it's not essential.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and the schema coverage is 100%. According to the rubric, the baseline is 4. The description does not add parameter semantics but none are needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool provides information about the service itself: the tool catalog, tier boundaries, and source location. It uses a specific verb ("about") and resource ("whetstone"), and implicitly distinguishes from sibling tools which are functional (e.g., audit_leakage, bank_health).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not provide explicit guidance on when to use this tool vs alternatives. It implies usage for obtaining service metadata, but lacks 'when to use' or 'when not to use' statements, and no alternatives are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It describes the tool as an 'audit' (suggesting read-only) but does not confirm non-destructiveness or disclose any side effects, permissions, or error conditions. The mention of 'clean exam export' could imply output generation but lacks clarity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise, conveying the tool's purpose and output types in two sentences plus an example. Every sentence is informative, with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no output schema, so the description should compensate. It mentions output components but not structure or types. Parameters are underspecified, and the example is the only usage hint. For a complex auditing tool, more detail is needed for an agent to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, but the description only lists keys without explaining their meaning or usage. The tool description adds minimal value beyond the schema, and the example hints at a key 'leakage' but does not clarify parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it performs an 'exact declared-exposure audit over exam rows' and lists components like row identity, fingerprints, and similarity flags. It is specific but does not explicitly differentiate from siblings like 'bank_health' or 'counterexample_hunt', though the unique purpose is implied.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, nor does it mention prerequisites or when not to use it. The only hint is the example reference, which is insufficient for an agent to decide context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description must bear the full burden. It only mentions diagnostics and gives an example endpoint, but does not disclose whether the tool is read-only, destructive, or any side effects. The behavioral characteristics are under-described.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very concise with one sentence and an example reference. It is front-loaded with purpose, but the example reference is vague and could be clearer. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and no annotations, the description is incomplete. It does not explain the return format, how to interpret the diagnostics, or any details beyond the high-level purpose. A tool with such sparse description leaves the agent guessing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters with 100% coverage, so the description does not need to add parameter information. The description adds value by explaining the tool's output focus, which is sufficient.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it provides 'item-lifecycle diagnostics' and lists specific aspects (discriminators, saturated/flaky items, frontier gaps), making the purpose clear. However, it does not explicitly differentiate from sibling tools like audit_leakage or counterexample_hunt, though the focus on overall health diagnostics is distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives is provided. The description does not specify context or prerequisites, leaving the agent to infer usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must convey behavioral traits. It states the search is 'bounded' and 'rate-limited', and that it returns an 'exact certificate when found'. However, it does not clarify whether the tool is read-only or has side effects, nor does it mention authentication or resource requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long and front-loads the primary purpose immediately. It includes a usage example reference without unnecessary elaboration. Every phrase earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite the tool's complexity (search algorithm, graph counterexample, DSL predicate class), the description lacks details on output format, certificate structure, and parameter behavior. Without an output schema, the description should explain return values but does not. The tool seems partially documented.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, meaning the input schema likely has detailed per-parameter descriptions. The tool description lists the parameter keys but adds no semantic value beyond the schema. With high coverage, baseline is 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as a 'bounded simulated-annealing search for a graph counterexample inside a DSL predicate class' with an exact certificate. This is a specific verb (search) and resource (graph counterexample in DSL predicate class), distinguishing it from sibling tools like 'about_whetstone' or 'audit_leakage' which have different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions 'CPU-bounded and strictly rate-limited' but provides no guidance on when to use this tool versus alternatives. No explicit when-to-use or when-not-to-use criteria, nor any comparison to sibling tools (full list provided).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It does not disclose whether the tool is read-only, modifies state, or requires specific inputs. The behavioral implications of calling this tool are unclear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core output. However, the second sentence referencing an example could be more integrated. Overall, every sentence serves a purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of a statistical gate evaluation, the description is insufficient. It does not explain the decision logic, return format, or how parameters map to the metrics. For a tool without output schema, this is inadequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds context beyond the schema by naming the metrics (gains, regressions, p-value, per-domain breakdown) that are used, which are not listed in the schema. This helps the agent understand expected input content.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description indicates the tool returns a decision (PASS, HOLD, or BLOCK) based on paired per-item results, but it does not explicitly state the tool's action (e.g., 'evaluates' or 'determines'). It is somewhat vague and does not differentiate from sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

No usage guidelines are provided. There is no indication of when to use this tool versus alternatives like 'inspect_promotion' or 'report_card_start'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must bear full burden. It mentions turning events into checkpoints, rewinds, etc., implying mutation or creation, but does not disclose whether it is safe, destructive, or requires authentication. No side effects or limitations are described.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loading the purpose and then providing an example. It is efficient with no filler, though the example could be considered minor overhead.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations, no output schema, and 0 parameters (though schema allows additional properties), the description omits crucial details like return value, error conditions, and prerequisite constraints. It is incomplete for an AI agent to confidently invoke.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (keys: events, notes), so baseline is 3. The description does not add meaning beyond what the schema already conveys; it does not explain the format or expected values of the keys.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the tool transforms reasoning-emulator control events into checkpoints, rewinds, notes, and a timeline, providing a specific verb and resource. It does not explicitly differentiate from sibling tools, but the sibling names are distinct enough to imply uniqueness.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an example (GET /api/examples key 'replay') but does not specify when to use this tool vs alternatives, nor does it give exclusion criteria or prerequisites. This leaves the agent without guidance on context appropriateness.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description fully discloses session lifecycle, expiry, rate limits, and the fact that it's a promotion-gate mechanism, not a private-bank credential.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences convey all necessary information, though the first sentence is somewhat dense; still efficient and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Without parameters or output schema, the description fully covers what the tool does, expected workflow, and constraints.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters exist, so baseline 4 applies; the description adds value by explaining the tool's purpose beyond the empty schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description specifies the tool starts a disposable report-card session and returns exam items, clearly distinguishing it from siblings like report_card_submit and promotion_gate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states to answer all items then call report_card_submit exactly once, and notes one-shot nature, expiry, and rate limits.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses the destructive nature ('session is destroyed by this call') and explains the grading logic (strict refinement, no answer key). It also mentions the output format. Full authentication or rate limit details are absent, but the key behavior is covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the main action ('submit answers for a report-card session and receive the graded report'). No redundant or extraneous text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 2 parameters (one nested) and no output schema, the description adequately explains the return value components and the grading process. It covers the destructive side effect. However, it lacks error conditions or prerequisites, though these are largely implied.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 50% (only answers has a description). The tool description does not elaborate on session_id beyond implying it comes from starting a session. It also does not clarify the format of answers beyond what the schema already provides. The description should compensate for the missing session_id documentation but does not.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool submits answers for a report-card session and returns a graded report. It mentions specific outputs (per-item verdicts, per-domain totals, SHA-256 commitments) and implicitly distinguishes from sibling report_card_start by referring to a 'report-card session'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage after starting a session with report_card_start (by mentioning 'submit answers for a report-card session') and notes that the session is destroyed. However, it does not explicitly state when to use vs alternatives or provide exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses conservation checks (untouched sections byte-identical, protected tokens preserved) but fails to mention side effects, permission requirements, or error behavior. Some behavioral context is added, but more detail is needed for a mutating tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, the first efficiently states the core functionality. The second points to an example. No wasted words, though the structure could be improved by front-loading key info.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations and no output schema, the description should be more comprehensive. It lacks details on return values, error scenarios, parameter formats, and prerequisites. The tool's complexity (Markdown patch with conservation checks) demands more context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with the schema naming keys 'document', 'reason', 'operations'. The description adds no extra meaning beyond these names. Baseline 3 is appropriate as the schema already documents the keys.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool applies a section-scoped Markdown patch with conservation checks, specifying that untouched sections stay byte-identical and protected tokens are preserved. This provides a specific verb and resource, and distinguishes it from sibling tools like audit_leakage or bank_health which are unrelated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions an example using GET /api/examples key 'safepatch' but does not explicitly state when to use this tool versus alternatives or provide any exclusions. Usage is implied for patching Markdown, but no guidance on when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.