Server Quality Checklist

Tool Scores

• get_transaction_recordA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, idempotent, and non-destructive behavior. The description adds substantial extra behavioral detail: ordering by 'predecessor reference rather than by a counter', the concept of a 'broken' chain, and the specific error code 'A202-EVIDENCE-CHAIN-GAP'. This goes well beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, but the first is a long, dense list of contents. Every clause adds value, but the structure could be slightly improved by splitting into bullet points. It is generally efficient and information-dense.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description compensates well by enumerating what the record contains (events, state, keys, object), how ordering works, and how broken chains are reported. It is comprehensive for a read-only tool, though it could mention the exact return format (e.g., JSON structure) to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers the only parameter (transaction_id) with 100% coverage, including the prefix constraint 'txn_'. The description adds no extra parameter-specific semantics, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Return this party's own copy of the hash-chained transaction record'. It uses a specific verb ('Return') and identifies the resource, and the unique phrasing 'own copy' and 'hash-chained' differentiates it from sibling tools like verify_evidence or create_agreement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use: 'Call it at any point to see where a transaction stands before deciding what to do next.' This gives clear context, though it does not explicitly name alternatives or state when not to use, which would be required for a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• issue_mandateA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes far beyond the annotations (which only say readOnlyHint=false, etc.). It discloses that a spending_limit becomes two constraints 'both visible in the audit record', that the mandate is checked before signing so refusals never receive a signature, and lists specific refusal codes. It also states the return value ('Returns the mandate identifier and the signed document'). These are significant behavioral details not derivable from annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than typical but well-structured: purpose and usage are front-loaded, followed by key behavioral notes, refusal list, and return value. Every sentence adds necessary information (e.g., audit record visibility, pre-signing checks, refusal codes), so it is appropriately sized for the tool's complexity. It is not minimal, but it is efficient and free of fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 15 parameters, 8 required, nested objects, and no output schema, the description is remarkably complete. It covers purpose, usage, side effects (audit record), failure modes (specific refusal codes), and return value. It also explains the global role of the mandate ('Every other tool names the mandate an agent acts under'), providing sufficient context for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3. The description adds extra meaning for spending_limit, explaining it becomes both an amount ceiling and a currency constraint with audit trail visibility, and clarifies the scope requirement that a scope must carry transaction_ids or categories. These enrich the parameter understanding beyond the schema's own descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states exactly what the tool does: 'Issue and sign a commercial mandate' and defines the document's content ('which organisation an agent represents, which acts it may take, under which limits, for which transaction or category, and for how long'). It clearly distinguishes from siblings like verify_mandate by saying 'Call this once per acting agent before anything else' and noting that every other tool names the mandate an agent acts under.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs when to use the tool: 'Call this once per acting agent before anything else' and explains its role as a prerequisite for other tools. It doesn't explicitly name alternatives or exclusion conditions, but the 'once per acting agent before anything else' directive provides clear usage context. The refusal conditions also imply when a mandate would not be issued, which indirectly guides usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• record_obligationA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description richly discloses state transitions (in_performance, acceptance_pending, settlement_pending), failure modes with error codes (A202-OBLIGATION-ASSERTION-UNEVIDENCED, A202-OBLIGATION-RESPONSE-UNAUTHORIZED), and constraints on evidence and decisions. This goes far beyond the minimal annotations (all hints false) and provides significant operational detail. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with an intro and three clearly labeled act blocks, following a consistent pattern of required fields then outcome/state transition. It is long but every sentence conveys necessary conditional information; however, it could arguably be tightened without losing value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and high complexity, the description covers all essential selection and invocation aspects: prerequisites, per-act requirements, error handling, and resulting states. A minor gap is not stating the return value or how to reference the resulting obligation/assertion record, which would be expected when no output schema exists.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema already documents all parameters at 100% coverage, the description adds crucial conditional logic: which fields are required per act, how the decision's proposed_action must align, and how accepted_quantity below owed interacts with remainder_obligation_id. That interplay is not visible from schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence 'Record one act in the obligation exchange' clearly states the verb and resource. It then breaks down three distinct acts (issue, assert, respond) with specific purposes, distinguishing itself from sibling tools like create_agreement or verify_mandate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit preconditions: each act must carry a decision obtained from verify_mandate, and identifies required fields per act. It also explains when each act applies (issue derives from agreement, assert follows performance, respond accepts/rejects). However, it does not explicitly mention when not to use this tool or contrast with sibling alternatives like issue_approval, so it stops short of full alternatives coverage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_agreementA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the full write path: 'It writes the whole formation in one call... two events, agreement.direct and agreement.committed, each countersigned by the other party.' It also details refusal conditions (A202-* codes) and notes that a non-matching decision 'refuses the whole call and records nothing.' These are meaningful behavioral details that go beyond the provided annotations (readOnlyHint=false, etc.), which are minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately dense for the tool's complexity. It is organized logically: purpose, scenario, verification steps, write behavior, follow-up, and refusals. Every sentence contributes necessary information without filler. It is longer than average, but the complexity of the tool justifies the length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description comprehensively covers the workflow, verification requirements, state transitions, refusal reasons, and next-step guidance. The only notable gap is that it does not explicitly describe the return value of the tool; since there is no output schema, a brief note on what the call returns (e.g., the transaction record or confirmation) would complete the picture.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema covers all parameters, the description adds essential semantic context for buyer_decision and supplier_decision by specifying the exact act documents to verify, the role-dependent action_type values (offer.submit vs. offer.accept), and the 60-second validity window. It also explains that session_id is minted by the offeror if absent. This substantially enriches the meaning of the parameters beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Form an agreement between two parties who already know each other, with no marketplace and no negotiation room.' This specifies the exact action and resource, and distinguishes it from marketplace-based flows or other transaction tools. It is not a tautology and clearly states the scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Call this once per transaction' and clarifies the appropriate scenario (known counterparties, no marketplace, no negotiation). It also provides a clear follow-up action: 'Follow with record_obligation act=issue,' which orients the agent on the next step. This is strong when-to-use guidance with an explicit alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• issue_approvalA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint=false, destructiveHint=false), the description discloses the signing key ownership, single hash/tx binding, expiration, and the A202-APPROVAL-HASH-MISMATCH behavior. This adds substantial behavioral context that annotations do not provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is several sentences long but each sentence earns its place: purpose, sequencing, signing details, hash binding, expiration, deny behavior, and return value. It is front-loaded and logically structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite 8 parameters, nested objects, and no output schema, the description states the return fields (approval identifier, action hash, expiration) and provides the full invocation sequence and behavioral constraints. The schema fully documents parameters, so the description covers all necessary context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds semantic meaning beyond the schema by linking action_hash to the verify_mandate result, explaining that conditions become part of the signed approval, and clarifying that a rejected decision releases nothing. This enrichment earns a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action: recording a named principal's approval of one exact act, bound to the action_hash verify_mandate reported. It clearly identifies the resource and distinguishes this from sibling tools by referencing the verify_mandate flow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to call only after verify_mandate answered require_approval, then to call verify_mandate again with approval_id. It also states a key boundary: an approval never releases a deny, providing clear when-to-use versus when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_evidenceA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds critical behavioral detail beyond the read-only/idempotent annotations: every check returns verified, failed, or not_checkable; there is no overall boolean because it would mask non-checkable results; a signature with an unheld key is not checkable and never verified; missing rules_version makes guarded transitions not_checkable rather than verified. These are non-obvious semantics the annotations do not convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place: first sentence defines the procedure, second gives usage context, third maps input modes to usage, and the final two sentences disclose edge-case behaviors. Perfectly front-loaded and free of fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description explains the per-check outcomes, the intentional omission of an overall boolean, and the conditional behavior of guarded transitions based on rules_version. For a complex seven-step verification tool, this provides a complete mental model without needing to inspect additional structured fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already covers all four parameters, so baseline is 3. The description adds value by clarifying the selection semantics: transaction_id verifies 'this server's record of one transaction,' object_ids verifies 'part of it,' and objects verifies 'a bundle somebody handed you.' It also explains the rules_version parameter's effect on guarded transitions. This goes beyond the schema's literal field definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Run the seven-step verification procedure over a set of objects,' then enumerates the seven steps (content hashes, signatures, purpose, version chains, event-chain continuity, guarded transitions, determination follow-through, and non-checkable items). This clearly distinguishes it from sibling verify_mandate, which has a different object of verification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs when to use the tool: 'Use it on your own record before relying on it, and on anything a counterparty sends you.' It also maps input modes to scenarios: transaction_id for the server's record, object_ids for partial records, and objects for counterparty-supplied bundles. This gives clear, actionable guidance on selecting the right input.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_mandateA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, but the description adds substantial behavioral context: no network call is made, status handling rules (older than 60 seconds counts as no result), deny is never approvable, and the exact checks performed. This goes well beyond the annotations with no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every sentence earns its place, covering purpose, checks, return values, approval flow, and status constraints in a logical structure. It is front-loaded with the core purpose and contains no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 params, nested objects, no output schema), the description fully covers what the tool does, what it returns (decision, reason codes, per-constraint outcomes, action_hash), and the approval workflow. It even notes the 'unavailability is not permission' edge case, making it complete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description adds meaningful context: it explains the relationship between mandate and mandate_id, defines the required status object semantics, clarifies that proposed_action paths address from $, and describes the action_hash/approval_id flow. This significantly enriches the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Check a mandate and decide whether an act is permitted under it.' It clearly distinguishes itself from sibling tools by focusing on mandate verification and decision, and explicitly says 'Call this before every act' and covers both own and counterparty mandates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: 'Call this before every act.' Also explains the require_approval flow with issue_approval and re-invocation, and describes how to handle status endpoints. It clearly delineates the alternative paths (mandate vs mandate_id) without needing to mention every sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.