Replay, verify and audit AI agent decisions with signed evidence. Public demo running on the fictional Acme Mutual insurance environment; a full integration is required to use Forge with real systems.
Scored across 7 tools
Three tools center on verifying/attesting the same signed record: verify_decision_record checks the signature, get_attestation_summary verifies against the pinned key and summarizes attestation (including signature status), and replay_decision_record re-runs and compares keys/decision. Descriptions help, but an agent could easily pick the wrong verification tool; review_agent_workflow also sits somewhat apart from the decision-record lifecycle.
All seven names use uniform snake_case with a verb_noun pattern (run_check, verify_decision_record, list_scenarios, etc.). Verbs vary appropriately by action, but the convention is consistent throughout.
Seven tools is well-scoped for a demo server with a narrow purpose. Each tool covers a distinct operation (list, run, verify, explain, replay, attest, review), and none feels redundant in count.
The decision-record lifecycle is well covered: list scenarios, run a check, verify the signature, explain the verdict, summarize attestation, and replay determinism. Minor gap: no way to inspect raw evidence or rule details beyond the explanation, and review_agent_workflow is an adjacent feature rather than part of the record lifecycle.
Explains, in plain English, why Forge Orbital reached the verdict in a demo decision record and what evidence a reviewer would need to change it. Read-only, stateless, demo data only.
| Name | Required | Description | Default |
|---|---|---|---|
| record | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover readOnly, idempotent, closed-world, and non-destructive, so the bar is lower. The description still adds real signal: statelessness, the demo-data-only constraint, and that the output includes the evidence needed to overturn the verdict.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, zero filler, with the core purpose front-loaded and the operational constraints (read-only, stateless, demo data) trailing compactly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description does a fair job describing what comes back, but for a single opaque nested-object input it leaves the agent without enough to supply a valid record. The main completeness gap is input provenance, not output behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is one required parameter, 'record', typed as a free-form object with additionalProperties true and zero schema description coverage. The description says it is a 'demo decision record' but gives no field names, shape, or where to obtain one, so the agent cannot reliably construct the argument.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: explains why a verdict was reached in a Forge Orbital decision record, plus what evidence would change it. This implicitly separates it from siblings like replay_decision_record and verify_decision_record, though it never names them explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is implied only: 'demo decision record' and 'demo data only' narrow the scope, but there is no explicit when-to-use versus replay_decision_record, verify_decision_record, or get_attestation_summary. The agent must infer the routing itself.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Verifies a Forge Orbital demo decision record against the pinned demo key and summarizes what it attests: signature status, the decision, whether the engine abstained, whether human review is required, and how many evidence items were used. Does not store the record. Demo records only.
| Name | Required | Description | Default |
|---|---|---|---|
| record | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/idempotent/non-destructive/closed-world, and the description reinforces the non-persistence trait with 'Does not store the record' plus the demo-only data constraint. It also enumerates the summary dimensions (signature status, decision, abstention, human review, evidence count), which tells the agent what to expect without an output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the action and scope, followed by two short constraint clauses. Every element carries information; nothing is padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Enumerating the summary fields compensates for the absent output schema, which is good. But with an untyped nested `record` object and no hint about how to obtain a valid record (e.g., from replay_decision_record or verify_decision_record), an agent can select the tool yet still not know how to supply input.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single `record` parameter is a free-form object with additionalProperties=true and 0% schema description coverage, so the schema conveys nothing about expected shape or required keys. The description never describes what a record must contain or where to obtain one, leaving the only input completely opaque.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb pair (verifies, summarizes) and a specific resource (a Forge Orbital demo decision record), and enumerates the summary fields the caller gets back. It stops short of naming how it differs from the close siblings verify_decision_record and replay_decision_record, so an agent must infer the split.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Demo records only' is a real scoping constraint that rules out production records. However, there is no explicit when-to-use versus verify_decision_record (which presumably validates rather than summarizes) or replay_decision_record, so the selection logic between the three closely related tools is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Lists the five sample scenarios for the fictional Acme Mutual Insurance that run_check accepts, with their evidence ids. Demo data only.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish readOnlyHint=true, openWorldHint=false and non-destructive, so safety disclosure is handled. The description adds real value beyond that: the result set is fixed at five items and is fabricated demo data, and the entries carry evidence ids — details the annotations cannot convey, especially with no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence plus a short qualifier, front-loaded with the main clause. Every clause carries information: count, domain, consumer tool, included field, and data provenance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a no-argument read-only list with full annotation coverage, the description supplies the essentials: what comes back, how many, and that the data is synthetic. With no output schema it could sketch the return shape slightly more (names plus ids), but the key field is already named.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Zero parameters, so there is nothing to disambiguate and the baseline applies. The description correctly omits any parameter discussion rather than inventing one.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb (Lists) plus resource (the five sample scenarios) and an explicit scope limitation: the ones run_check accepts. Naming run_check ties it to a sibling tool the agent already has, so its role is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'that run_check accepts' plus 'with their evidence ids' makes the intended use clear: call this to obtain valid scenario inputs for run_check. It stops short of explicitly saying when not to call it (e.g., in non-demo contexts), but the 'Demo data only' note covers that implicitly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Replays a Forge Orbital demo decision: re-runs the deterministic engine on the same sample scenario (and any withdrawn evidence) and checks that the replay key and the decision match the record. Replayability is the point: the same inputs and engine version give the same decision. Reads nothing about the user. Demo only, fictional sample data.
| Name | Required | Description | Default |
|---|---|---|---|
| record | Yes | ||
| scenario_id | Yes | ||
| withdraw_evidence | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive, closed-world, so the bar is lower; the description still adds real context: 'Reads nothing about the user', 'Demo only, fictional sample data', and the determinism guarantee. It omits what happens on a mismatch (raised error vs. returned diff), which is the one behavioral detail an agent would want.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the action, and the demo-data caveat is prominent at the end. 'Replayability is the point: the same inputs and engine version give the same decision' is slightly restating but carries the core conceptual guarantee.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description must convey the return contract; it hints that a match check occurs but not the result shape (match flag, diff, key values). For a 3-param tool with a nested untyped 'record' object, the definition leaves meaningful gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It implicitly touches all three parameters ('the same sample scenario', 'any withdrawn evidence', 'the record'), but never explains the shape or required fields of the free-form 'record' object (additionalProperties: true, no properties defined) or the semantics of withdrawing evidence.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Replays a Forge Orbital demo decision') plus the mechanism ('re-runs the deterministic engine on the same sample scenario'). The verification aspect ('checks that the replay key and the decision match the record') distinguishes it in substance from verify_decision_record, but no sibling is named explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is implied: you replay to confirm reproducibility, and the 'Demo only, fictional sample data' clause scopes it. There is no explicit when-to-use versus when-not guidance and no routing to verify_decision_record or explain_verdict, which an agent would plausibly confuse with this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run the Forge evaluator and optimizer on synthetic examples or supplied labeled baseline/candidate outputs and token usage. Use payload={sample_mode: improved}, regression, or changed_path for examples. A description alone returns an integration plan, not a measured evaluation. No code runs, external agent calls, changes or deployment. Synthetic prices are unsigned estimates, never approved savings. Interactive version: https://demo.forgeorbital.com/sandbox/
| Name | Required | Description | Default |
|---|---|---|---|
| payload | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish read-only, idempotent, closed-world, non-destructive behavior, and the description reinforces this ("No code runs, external agent calls, changes or deployment"). Beyond that it adds genuinely new context: synthetic prices are unsigned estimates, never approved savings. That caveat is valuable but return behavior is still undisclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The core purpose is front-loaded and every sentence carries information about usage, evaluation behavior, or safety caveats. The trailing interactive URL is minor padding but not disruptive.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a nested, unspecified payload with 0% schema coverage and no output schema, the description should do more to explain payload structure. It covers modes and safety but leaves an agent guessing at the full payload shape, so it is adequate but incomplete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% for a single free-form object payload (additionalProperties: true, no description). The description compensates partially by naming sample_mode, regression, and changed_path, but the payload remains largely opaque, so the agent gets only partial meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verbs ("Run the Forge evaluator and optimizer") and names the resources it operates on (synthetic examples or supplied labeled baseline/candidate outputs and token usage). It doesn't explicitly distinguish itself from siblings like run_check or explain_verdict, keeping it a step below a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives concrete invocation guidance (payload={sample_mode: improved}, regression, or changed_path) and a clear conditional: "A description alone returns an integration plan, not a measured evaluation." It stops short of naming when to prefer siblings, so it isn't a full 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Computes a Forge Orbital verdict for one of five fictional Acme Mutual Insurance scenarios and returns it with a signed decision record in the response. Read-only: nothing is stored, sent or changed. Optionally withdraw evidence ids to see how the verdict changes. Demo only: sample rules and sample evidence, not connected to the user's systems, policies or live actions.
| Name | Required | Description | Default |
|---|---|---|---|
| scenario_id | Yes | ||
| withdraw_evidence | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, destructiveHint=false and openWorldHint=false, so the safety profile is known; the description nonetheless adds substantive context: nothing is stored, sent or changed, and the tool is explicitly demo-only with sample rules and no connection to the user's systems or live actions. That sandbox framing is behaviorally important and appears nowhere in the structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four short sentences, front-loaded with the action, then constraints, then the optional lever. Only the read-only clause overlaps slightly with annotations, and even that adds the concrete 'nothing is stored, sent or changed' phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter read-only tool with no output schema and 0% schema coverage, the description covers purpose, side effects and demo scope adequately. It leaves gaps on how to obtain scenario and evidence ids, and what the returned 'signed decision record' contains or whether verify_decision_record should follow.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must carry the load. It clarifies that scenario_id selects among exactly five fictional scenarios (matching the enum) and that withdraw_evidence takes evidence ids and changes the resulting verdict — useful meaning beyond a bare string array. It still never says where valid evidence ids come from or that scenario ids can be discovered via list_scenarios.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: computes a Forge Orbital verdict for one of five named scenarios and returns it with a signed decision record. The scope (sample scenarios, demo rules) separates it from production-path tools, but no sibling tool is named, so the agent must infer that list_scenarios supplies the ids and explain_verdict explains the result.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by 'demo only' and 'optionally withdraw evidence ids to see how the verdict changes', which tells the agent the second parameter is an exploration lever. There is no explicit when-to-use versus explain_verdict, replay_decision_record, or verify_decision_record, which are the obvious alternatives for a returned decision record.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Verifies the Ed25519 signature of a Forge Orbital demo decision record against the published demo key id. Pass the full signed record object returned by run_check (this demo keeps nothing, so a record id alone cannot be looked up). Editing any field makes verification fail. Demo records only.
| Name | Required | Description | Default |
|---|---|---|---|
| record | No | ||
| record_id | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds behavior beyond the annotations: the service is stateless ('keeps nothing'), verification is tamper-evident ('editing any field makes verification fail'), and the scope is demo-only. Annotations already cover the read-only/non-destructive profile, so this extra context is the valuable part. The one gap is that it never says what a verification result looks like.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences with no filler: purpose first, then the calling constraint, then the tamper/edit caveat. Every sentence carries distinct information and nothing is repeated from the schema or annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should say what verification returns (a boolean, a verdict object, a reason on failure) and it does not, leaving a real gap for a verification tool. It is otherwise complete on input, scope, and safety.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must carry parameter meaning, and it does for the critical case: 'record' is the full signed object, and 'record_id' alone cannot resolve anything. It does not explain what happens if record_id is supplied anyway or whether it is validated or ignored.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and mechanism ('Verifies the Ed25519 signature') against a named resource ('Forge Orbital demo decision record') and names the key id it checks against. It also implicitly separates itself from replay_decision_record and explain_verdict by naming run_check as the producer of the input.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells the agent to pass the full signed record object returned by run_check and warns that a record id alone cannot be looked up because the demo is stateless. It lacks a direct comparison to sibling tools like replay_decision_record or explain_verdict, but the input-source guidance is concrete and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
explain_verdictget_attestation_summarylist_scenariosreplay_decision_recordreview_agent_workflowrun_checkverify_decision_record
Add one secure layer between your agents and this server.