Server Quality Checklist

Tool Scores

• cedulon_spendA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses both possible outcomes—an allow path returning a signed COSE receipt JSON and a deny path returning the fail-closed reason. This goes beyond the annotations, which only indicate hints like readOnly false and idempotent false. It doesn't explain the policy itself, but it clearly communicates the behavioral contract.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two short sentences, front-loaded with the primary action, and each sentence adds distinct value. There is no filler, redundancy, or unnecessary background.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description explains return variants, it does not cover policy prerequisites, parameter formats, or the meaning of nonce, payee, and currency. With no output scheme and only 40% schema description coverage, these omissions make it difficult for an agent to invoke the tool correctly the first time.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 40%, yet the description adds no parameter-level meaning for nonce, payee, or currency, and does not even mention 'amount' or 'tool'. With most parameters undocumented in both schema and description, the agent has little guidance on how to set valid inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('spend'), the resource ('mock rail'), and the policy-gating nature of the operation. It also distinguishes the tool from siblings like cedulon_verify_receipt and cedulon_audit by describing the spend-specific outcome.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is used when a policy-gated spend should be attempted, and sibling names suggest the other tools serve different purposes. However, it does not explicitly state when to use this tool over alternatives, nor does it provide exclusions or condition-based routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cedulon_statusA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare readOnlyHint=true and openWorldHint=false, covering the safety profile. The description adds useful context about the specific data fields exposed, but does not disclose any further behavioral details such as response format, freshness, or failure modes. This is comparable to a straightforward status read where the annotations carry the main burden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that front-loads the most important information: it enumerates exactly what the status tool exposes. Every word earns its place and there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless status endpoint with read-only annotations, the description is largely sufficient: it names the key result fields. There is no output schema to supplement the return values, but the listed fields are concrete enough for an agent to understand what this tool offers in the context of its siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters and the schema is empty, so there are no parameter semantics to document. With 0 params, the baseline of 4 applies, and the description does not need to compensate for any parameter coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The title 'Server status' plus the description's list of returned data ('Server version, policy summary, receipt count, and chain head hash') clearly identifies this as a read-only status tool. It is distinct from the sibling tools (spend, audit, verify_receipt, export_ledger), though it lacks an explicit verb such as 'returns' or 'gets'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to choose this tool versus its siblings such as cedulon_audit or cedulon_export_ledger. There is no stated condition, exclusion, or mention of alternatives; usage is only weakly implied by the word 'status' in the title.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cedulon_auditA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With readOnlyHint=true and openWorldHint=false, the annotations already cover the safety profile and closed-world assumption. The description adds the outcome contract ('balanced or findings') and names the comparison sources, but it does not explain what findings contain, how failures are expressed, or any other behavioral caveats. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler: the purpose is front-loaded ('Reconcile...') and the outcome is stated in the second sentence. Every phrase earns its place, and the description is appropriately sized for an audit tool whose parameters are documented in the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 7-parameter nested-object tool with no output schema, the description is a bit thin: it gives the high-level result ('balanced or findings') but not the shape of findings or any guidance on how trust/manifest inputs affect the audit. The rich schema compensates for parameter coverage, but the return contract remains under-specified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the description does not need to repeat parameter details. The description itself adds no parameter-level semantics, but the schema already explains trust, manifest, payee/issuer/witness/manifest trust, and extraSettlements in enough detail. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Reconcile') and identifies the exact resources involved: the in-process receipt chain and checkpoint are compared against the rail extract. It also says the result is either 'balanced or findings,' which distinguishes this audit/reconciliation tool from siblings like spend, verify_receipt, export_ledger, and status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for reconciliation and audit-style checks, so an agent can infer it is not for spending, single-receipt verification, exporting, or status. However, it never explicitly says when to use this tool instead of a sibling, nor does it mention any conditions or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cedulon_verify_receiptA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only declare readOnlyHint, so the description carries the burden of behavioral caveats. It adds the important warning that omitting expectIssuerKeyPem makes verification self-referential and 'any key satisfies' it, which prevents an agent from over-trusting a nominally verified receipt.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two dense sentences with no filler. It front-loads the primary purpose and then delivers the single most important usage caveat, making every word earn its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 optional parameters, no required fields, no output schema, and no guidance on which parameter combinations are valid, the description is not complete enough for reliable invocation. It explains the issuer-key pitfall but leaves the receipt/countersignature input representations and the verification result unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 43%, and the free-text description explains only expectIssuerKeyPem behaviorally. The relationship between coseHex, receipt, counterCoseHex, publicKeyPem, and payeePublicKeyPem is left unstated, so an agent cannot confidently choose among the seven optional input modes from the description alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with the verb 'Verify' and names the specific resource: a spend receipt COSE_Sign1 plus the optional payee countersignature. This clearly separates it from spend/audit/export/status siblings and makes the tool's operation unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides concrete conditional guidance: supply expectIssuerKeyPem when you want to check against a key you already hold, and omit it when you accept the receipt's self-carried key. It does not name alternative tools, but the parameter-level when/how instructions are clear enough for correct invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cedulon_export_ledgerA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=false, covering the safety and world-scope behavior. The description adds the useful detail that output matches npm run demo:export, but it does not disclose any other behavioral traits such as output size, asynchronous behavior, or prerequisites, so it only partially contributes beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant words. It clearly communicates the operation, the target data, and the output format without padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool, the description is mostly sufficient: it names the data kinds and the output shape. It relies on familiarity with 'npm run demo:export' and leaves terms like 'checkpoint' and 'rail extract' undefined, but in a project context this is likely enough for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters and coverage is effectively 100%, so no parameter documentation is needed. The description's mention of the three export categories adds context about what the tool operates on, but since there are no configurable inputs, there is little parameter semantics to add.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Export') and the exact resources ('receipts, checkpoint, and rail extract'), making the tool's function immediately clear. It also distinguishes itself from siblings by being the export operation, as opposed to spend, audit, verify, or status actions. The reference to the demo:export JSON shape further pins down intent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: any time an export of receipts, checkpoint, or rail extract data is needed in the demo:export shape. However, it does not explicitly mention alternatives or state when not to use it, leaving routing to siblings to be inferred rather than spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.