Server Quality Checklist

Tool Scores

• list_cardsA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the transparency burden. It explicitly details the non-reconciliation behavior ('never reconciled with'), that it shows disagreement rather than picking a side, and that counts are always reported in full regardless of limit. This is meaningful behavioral disclosure beyond a generic 'list' operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately concise: three sentences, with the core comparison behavior front-loaded before the filter note. The dash-embedded caveat and URL-heavy phrasing add some density, but every sentence contributes necessary context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description supplies a conceptual view of returns: two labelled numbers plus row listing with optional filters. It does not detail the row structure or exact output format, but the combination of schema and description gives enough for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with both axis and limit fully described in the input schema. The description adds only the phrase 'recent rows' and confirms both are optional filters, which is marginal added value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's function: it reports two counts from two surfaces (the signed-card index's declared n_cards vs its actual row count, and the card store endpoint's self-reported count), and lists rows with optional axis/limit filters. It doesn't explicitly differentiate itself from siblings by name, but the specific resources and comparison behavior make the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states when to use the tool: when you want to inspect these counts and see a disagreement, or list recent rows via optional filters. However, it offers no guidance on when not to use it or how it compares to sibling tools like get_axis, verify_card, or board_totals, leaving the choice to the agent's inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• board_totalsA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, and it exceeds expectations: it reveals the live-fetch nature, the slot-vs-measurement meaning, the never-summed/never-swapped constraint, both as_of timestamps, the trust posture ('We measure, never certify'), and the UNREACHABLE failure state with 'no cached number is ever presented as live'. This is genuinely transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, all load-bearing: the source resource, the data semantics with the integrity constraints, and the failure behavior. It is dense rather than padded, though it could be slightly trimmed without losing meaning. Front-loads the resource and purpose before the constraints.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description covers everything an agent needs: what is returned, the semantics of each number, the as_of timestamps, and the UNREACHABLE failure state. Nothing material is missing for correct invocation; a concrete return-shape example would be the only marginal addition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing to document and the baseline of 4 applies. Rather than parameters, the description invests in documenting return semantics — labelled counts, kinds, and as_of dates — which is where the agent needs guidance given there is no output schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: 'Live GSPC board totals from https://councilof.ai/api/gspc... Returns the slot count and the measured count'. The distinction between slot and measurement together with 'never summed and never swapped' differentiates it clearly from siblings like get_axis, verify_card, and list_cards. An agent can tell this is the board-aggregate tool without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description details what the tool returns and its integrity semantics, so when to use it is implied — it is the tool for live board totals. However, it gives no explicit routing against the named siblings and no when-not-to-use conditions. Behavior context is strong, but explicit usage exclusions are absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_axisA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description takes on the disclosure burden and does it well: it defines the unmeasured-axis case as a first-class answer rather than an error or zero, and adds the 'never a certification' constraint. It does not discuss read-only guarantees, auth, or rate limits, but those are less critical for a simple live board lookup.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences carry the full definition: the first locates the resource and enumerates the reply fields; the second resolves a likely misinterpretation around unmeasured axes. Every word contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description enumerates all output fields (n, accuracy, interval, status, family, dates) and clarifies edge semantics, while the schema fully documents the input and error fallback. For a one-parameter read-only lookup with no output schema, nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers the sole parameter at 100%, including an example, case-insensitivity, and the unknown-name fallback. The description adds no parameter-specific details beyond the field list, so per the coverage rule it stays at baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('get') and resource ('one axis row from the live GSPC board'), lists the returned fields, and explicitly distinguishes the result from a certification. This is enough for an agent to tell it apart from list_cards and verify_card.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Clarifies the target is a single row on the public board, which implies use for per-axis lookups, and the schema notes unknown names return the board's actual list. However, it never explicitly names sibling tools or states when another tool should be used instead, so it stops short of explicit routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_cardA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and meets it: it discloses the security model (a self-carried key is not trusted), the pinned key source, and the three verdicts including the deliberately distinct UNCHECKABLE state. It also explains why 'could not check' is not 'forged'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is information-dense but every sentence earns its place: procedure, security caveat, verdict semantics, accepted inputs, and explicit boundary. Key details are front-loaded before the edge-case nuance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description states all three verdicts and the reason behavior, plus accepted input forms and failure semantics (UNCHECKABLE). For a single-parameter verification tool with this complexity, nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds domain meaning beyond the schema: the card is a signed gspc.measurement-card, may arrive as an object, JSON string, or councilof.ai / csoai.org URL, and certifications are out of scope. It reinforces the accepted forms and adds context the schema alone lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with a specific verb and resource: 'Verify a signed gspc.measurement-card' and specifies the exact procedure: recompute the id from canonical body bytes and check the Ed25519 signature against the pinned did:web key. This clearly distinguishes it from siblings like get_axis, list_cards, and board_totals.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is explicit — verify a signed card under the published rule — and the description gives a clear exclusion ('Never a certification'). It does not name sibling tools as alternatives, so it stops short of full alternate-route guidance, but an agent can infer when this tool applies.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.