Arquestra Audit

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds genuine behavioral value by disclosing the progressive depth of returns (overview → per-result → full reasons and remediation) and that level only filters within a named example. It says nothing about caching, availability, or failure behavior, so it sits at solid adequacy rather than richness.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded, each tied to a distinct calling mode with zero filler. The telegraphic phrasing 'result by result (level filters by light)' is slightly awkward to parse but costs nothing in length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With four optional parameters and no output schema, the description carries the burden of explaining return shapes, and it does so for all three modes including what the deepest result contains. The lang parameter's effect is only documented in the schema, which is a minor gap for an otherwise complete definition.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning: it reveals that id is meaningless without name (a dependency the schema does not state) and that level scopes to the figures of a named example rather than globally. Only lang is left to the schema entirely.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies the resource precisely — published example audits such as the AI Project Radar — and its three operating modes make clear it is a retrieval tool. It lacks an explicit verb (get/list), but the mode lines leave no doubt about what it returns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit, argument-conditioned usage: no arguments for the overview and list, a name for one example, a name plus id for a single result. The closing line 'They are not the person's data' implicitly separates it from the sibling audit tools, though it never names show_audit or start_audit directly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, and the description is consistent with them, adding real value on top: the file stays in the browser and is never sent to the server, and the measurement is performed client-side by measure.py. It stops short of saying what happens on failure or whether the panel state persists between calls.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action, then the two modes, then the privacy note — a sensible order with no filler. The long enumerated list of profile fields in the middle is dense and slightly run-on, but it is load-bearing because there is no output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, describing the returned profile (column kind, empty cells, distinct values, category counts, ranges, first rows) is exactly the right compensation, and the privacy statement covers the biggest behavioural question for a file-reading tool. Missing only the explicit handoff from start_audit and any error-path behaviour.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both parameters are documented there, including the 'leave it out' semantics of spec, so the schema already carries the load. The description adds the useful consequence of supplying an unconfirmed vs. confirmed spec and implies the columns parameter's checking role only indirectly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete verb and resource ("Opens the audit panel in the chat") and then splits the behaviour into two named modes (no spec vs. a confirmed spec). An agent can picture the action, but the description never names its siblings (start_audit, show_audit, prepare_script), so the distinction from those requires inference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The when-to-use logic is clear: omit spec to open the panel and receive the file, pass a confirmed spec to run the measurement. What is missing is routing against alternatives — the dependency on start_audit appears only in the schema's spec description, not in the tool description, and no exclusion against show_audit is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly/openWorld annotations, the description discloses substantial behavior: the script uses only the standard library, makes no network calls, uses fixed seeds, supports .csv and .json, and emits counts with a fingerprint but no rows. It also documents the error path (returns spec errors instead of the script), which is exactly the kind of context annotations cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The content is dense and nearly waste-free, but the opening clause "Returns measure.py with a confirmed spec written in" is a run-on whose syntax obscures the actual object being returned. Front-loading is attempted but the phrasing gets in the way.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, so the description must define the return value, and it does: either the script text or the spec errors. With a nested object parameter and a fixed-read annotation set, the remaining gap is mostly operational (is the script returned as text or a file, and how it is meant to be executed).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters are already documented, including the optional columns reconciliation parameter. The description adds no additional meaning about spec shape or what happens when column checking is omitted, so it sits at the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource ("Returns measure.py") and explains concretely what the artifact does: a fixed Python script that counts what each check needs and prints counts plus a fingerprint, never rows. That distinguishes it from sibling tooling like show_audit or measure_in_app, though no sibling is named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the spec parameter is described in the schema as coming from start_audit as confirmed by the person, so the ordering is inferable. The description itself never states when to call this versus start_audit or measure_in_app, and there are no exclusions or prerequisites spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered; the description still adds real context by disclosing the fingerprint/spec integrity check, the red-amber-green semantics with reasons, that results are both shown in chat and returned, and that an external arquestra.dev link carrying only codes accompanies them. 'Stores nothing' usefully reinforces the no-side-effect claim. It stops short of describing behavior when verification fails, which is the main behavioral gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the core transformation, then the verification precondition, then the return/link behavior. Every clause carries information, though the final sentence about the arquestra.dev link is densely packed and slightly harder to parse than the rest.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter, fully documented, non-mutating tool with no output schema, the description covers what it consumes, what it verifies, what it produces, and that it persists nothing. The notable omission is failure behavior when the fingerprint or spec check does not match, which an agent would need in order to handle that case.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds genuine meaning: it explains why the output must contain the fingerprint line (to confirm the script is unchanged) and that the spec is the same one given to prepare_script, which is the basis for the comparison. It does not add format or syntax detail beyond the schema, so it does not reach the top of the scale.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The verb and resource are specific: it turns measure.py output into one red/amber/green light per result, with reasons, and returns them. It also states the verification precondition (unchanged script, matching spec), which is concrete enough that an agent can tell this is a render/verify step rather than a measurement step. It never names or contrasts with the siblings example_audits or start_audit, so differentiation is only partial.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Placement in the pipeline is implied by 'output of measure.py' and by the schema's reference to prepare_script, so an agent can infer it runs after measurement. There is no explicit statement of when to prefer this over example_audits or start_audit, nor what to do instead if the fingerprint check fails, leaving the routing decision to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description goes beyond that by disclosing that no rows are received (a meaningful privacy/data-boundary trait) and by enumerating what comes back: a proposed spec with column roles, claims, claim-writing guidance, a check catalog, and confirmation points. Since no output schema exists, that return description carries real weight.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action and inputs, then a compact enumeration of outputs. The parenthetical about type and example value is well placed. It is dense but every clause carries information; the only cost is a slightly run-on second sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only, no-output-schema tool with three parameters, the definition covers inputs, the no-rows constraint, and the full shape of the response, which is enough to invoke it correctly. What is missing is sequencing relative to the sibling audit tools, which matters given the multi-step workflow implied by prepare_script and show_audit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 33%, so the description must compensate, and it largely does: it explains that columns come with 'optionally a type and one example value each' and characterizes figures as 'the results the person wants to publish'. Only dataset_name goes unmentioned in both schema and description, a minor omission for an obviously named field.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource: it 'proposes how to read a data file for an audit' from column names plus desired published results. That is clearly distinct from show_audit or measure_in_app, though it never names those siblings to make the distinction explicit. An agent can tell what it does without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the tool takes column metadata and figures, so it is evidently the entry point before preparing or measuring. There is no explicit when-to-use statement, no prerequisites, and no routing to example_audits, prepare_script, or show_audit. The closing note 'It receives no rows' does implicitly tell the agent not to send data.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.