Forge AI Accountability

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnly, idempotent, closed-world, and non-destructive, so the bar is lower. The description still adds real signal: statelessness, the demo-data-only constraint, and that the output includes the evidence needed to overturn the verdict.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, zero filler, with the core purpose front-loaded and the operational constraints (read-only, stateless, demo data) trailing compactly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description does a fair job describing what comes back, but for a single opaque nested-object input it leaves the agent without enough to supply a valid record. The main completeness gap is input provenance, not output behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is one required parameter, 'record', typed as a free-form object with additionalProperties true and zero schema description coverage. The description says it is a 'demo decision record' but gives no field names, shape, or where to obtain one, so the agent cannot reliably construct the argument.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: explains why a verdict was reached in a Forge Orbital decision record, plus what evidence would change it. This implicitly separates it from siblings like replay_decision_record and verify_decision_record, though it never names them explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is implied only: 'demo decision record' and 'demo data only' narrow the scope, but there is no explicit when-to-use versus replay_decision_record, verify_decision_record, or get_attestation_summary. The agent must infer the routing itself.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive/closed-world, and the description reinforces the non-persistence trait with 'Does not store the record' plus the demo-only data constraint. It also enumerates the summary dimensions (signature status, decision, abstention, human review, evidence count), which tells the agent what to expect without an output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, front-loaded with the action and scope, followed by two short constraint clauses. Every element carries information; nothing is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Enumerating the summary fields compensates for the absent output schema, which is good. But with an untyped nested `record` object and no hint about how to obtain a valid record (e.g., from replay_decision_record or verify_decision_record), an agent can select the tool yet still not know how to supply input.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single `record` parameter is a free-form object with additionalProperties=true and 0% schema description coverage, so the schema conveys nothing about expected shape or required keys. The description never describes what a record must contain or where to obtain one, leaving the only input completely opaque.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb pair (verifies, summarizes) and a specific resource (a Forge Orbital demo decision record), and enumerates the summary fields the caller gets back. It stops short of naming how it differs from the close siblings verify_decision_record and replay_decision_record, so an agent must infer the split.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Demo records only' is a real scoping constraint that rules out production records. However, there is no explicit when-to-use versus verify_decision_record (which presumably validates rather than summarizes) or replay_decision_record, so the selection logic between the three closely related tools is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnlyHint=true, openWorldHint=false and non-destructive, so safety disclosure is handled. The description adds real value beyond that: the result set is fixed at five items and is fabricated demo data, and the entries carry evidence ids — details the annotations cannot convey, especially with no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence plus a short qualifier, front-loaded with the main clause. Every clause carries information: count, domain, consumer tool, included field, and data provenance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-argument read-only list with full annotation coverage, the description supplies the essentials: what comes back, how many, and that the data is synthetic. With no output schema it could sketch the return shape slightly more (names plus ids), but the key field is already named.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Zero parameters, so there is nothing to disambiguate and the baseline applies. The description correctly omits any parameter discussion rather than inventing one.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb (Lists) plus resource (the five sample scenarios) and an explicit scope limitation: the ones run_check accepts. Naming run_check ties it to a sibling tool the agent already has, so its role is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'that run_check accepts' plus 'with their evidence ids' makes the intended use clear: call this to obtain valid scenario inputs for run_check. It stops short of explicitly saying when not to call it (e.g., in non-demo contexts), but the 'Demo data only' note covers that implicitly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world, so the bar is lower; the description still adds real context: 'Reads nothing about the user', 'Demo only, fictional sample data', and the determinism guarantee. It omits what happens on a mismatch (raised error vs. returned diff), which is the one behavioral detail an agent would want.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the action, and the demo-data caveat is prominent at the end. 'Replayability is the point: the same inputs and engine version give the same decision' is slightly restating but carries the core conceptual guarantee.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must convey the return contract; it hints that a match check occurs but not the result shape (match flag, diff, key values). For a 3-param tool with a nested untyped 'record' object, the definition leaves meaningful gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It implicitly touches all three parameters ('the same sample scenario', 'any withdrawn evidence', 'the record'), but never explains the shape or required fields of the free-form 'record' object (additionalProperties: true, no properties defined) or the semantics of withdrawing evidence.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Replays a Forge Orbital demo decision') plus the mechanism ('re-runs the deterministic engine on the same sample scenario'). The verification aspect ('checks that the replay key and the decision match the record') distinguishes it in substance from verify_decision_record, but no sibling is named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is implied: you replay to confirm reproducibility, and the 'Demo only, fictional sample data' clause scopes it. There is no explicit when-to-use versus when-not guidance and no routing to verify_decision_record or explain_verdict, which an agent would plausibly confuse with this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false; the description adds genuine context beyond them by stating no code runs, no external agent calls, no changes or deployment occur, and that synthetic prices are unsigned estimates never approved savings. That clarifies output trustworthiness, a real behavioral trait. Minor gap: no mention of latency or payload-size limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action, then mode guidance, then the important caveat about description-only inputs, then safety disclaimers. Sentences are dense and each adds information; the trailing demo URL is a small non-essential addition but harmless.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, a nested open payload, and vague sibling boundaries, the description covers safety and a few payload modes but not the return shape or required inputs for a real evaluation. An agent could construct the simple sample_mode call but would be guessing for labeled baseline/candidate inputs.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and the single payload parameter is an open object with additionalProperties true, so the description must carry the semantics. It does name three usable payload modes, which is real added value, but leaves the shape of keys, required fields, and the labeled baseline/candidate format undefined.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: run the Forge evaluator and optimizer on synthetic examples or supplied baseline/candidate outputs. It distinguishes itself from siblings like run_check or explain_verdict implicitly by naming evaluation plus cost optimization. However, it never explicitly contrasts itself with the closest sibling, so an agent must infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives concrete modes (sample_mode: improved, regression, changed_path) and states that a description alone yields an integration plan rather than a measured evaluation, which is a useful when-not condition. But it never names alternatives such as run_check or list_scenarios, so the choice among siblings is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, destructiveHint=false and openWorldHint=false, so the safety profile is known; the description nonetheless adds substantive context: nothing is stored, sent or changed, and the tool is explicitly demo-only with sample rules and no connection to the user's systems or live actions. That sandbox framing is behaviorally important and appears nowhere in the structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded with the action, then constraints, then the optional lever. Only the read-only clause overlaps slightly with annotations, and even that adds the concrete 'nothing is stored, sent or changed' phrasing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter read-only tool with no output schema and 0% schema coverage, the description covers purpose, side effects and demo scope adequately. It leaves gaps on how to obtain scenario and evidence ids, and what the returned 'signed decision record' contains or whether verify_decision_record should follow.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must carry the load. It clarifies that scenario_id selects among exactly five fictional scenarios (matching the enum) and that withdraw_evidence takes evidence ids and changes the resulting verdict — useful meaning beyond a bare string array. It still never says where valid evidence ids come from or that scenario ids can be discovered via list_scenarios.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: computes a Forge Orbital verdict for one of five named scenarios and returns it with a signed decision record. The scope (sample scenarios, demo rules) separates it from production-path tools, but no sibling tool is named, so the agent must infer that list_scenarios supplies the ids and explain_verdict explains the result.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by 'demo only' and 'optionally withdraw evidence ids to see how the verdict changes', which tells the agent the second parameter is an exploration lever. There is no explicit when-to-use versus explain_verdict, replay_decision_record, or verify_decision_record, which are the obvious alternatives for a returned decision record.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds behavior beyond the annotations: the service is stateless ('keeps nothing'), verification is tamper-evident ('editing any field makes verification fail'), and the scope is demo-only. Annotations already cover the read-only/non-destructive profile, so this extra context is the valuable part. The one gap is that it never says what a verification result looks like.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with no filler: purpose first, then the calling constraint, then the tamper/edit caveat. Every sentence carries distinct information and nothing is repeated from the schema or annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description should say what verification returns (a boolean, a verdict object, a reason on failure) and it does not, leaving a real gap for a verification tool. It is otherwise complete on input, scope, and safety.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must carry parameter meaning, and it does for the critical case: 'record' is the full signed object, and 'record_id' alone cannot resolve anything. It does not explain what happens if record_id is supplied anyway or whether it is validated or ignored.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and mechanism ('Verifies the Ed25519 signature') against a named resource ('Forge Orbital demo decision record') and names the key id it checks against. It also implicitly separates itself from replay_decision_record and explain_verdict by naming run_check as the producer of the input.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly tells the agent to pass the full signed record object returned by run_check and warns that a record id alone cannot be looked up because the demo is stateless. It lacks a direct comparison to sibling tools like replay_decision_record or explain_verdict, but the input-source guidance is concrete and actionable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.