Server Quality Checklist

Tool Scores

• statsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, covering the safety profile. The description adds value by enumerating the metrics returned, but it does not disclose additional behavioral details such as output structure, potential latency, or any dependence on prior operations. With annotations present, this is adequate but not exceptional.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the key information with a compact comma-separated list. Every word contributes meaning, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless, read-only statistics tool with safe annotations, the description covers the main dimensions an agent would need: what the tool reports. The absence of an output schema means the description bears some responsibility for explaining return values, and the high-level list is sufficient for basic invocation, though it doesn't detail types or structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, and the schema confirms this with 100% coverage. Per the calibration rule, a zero-parameter tool receives a baseline of 4 since there are no parameter semantics to clarify. The description appropriately focuses on output rather than inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as providing catalogue statistics through a specific list of outputs: counts, success rate, duplicates prevented, embedding status, and common context tags. It distinguishes itself from the sibling tools (all action-oriented) by describing a read-only reporting function, though it lacks an explicit imperative verb like 'returns' or 'provides'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage is implied by the tool name and description: call when you need aggregate statistics about the catalogue. However, there is no explicit guidance on when to prefer this tool over siblings, nor any mention of alternatives or exclusions, leaving the agent to infer the appropriate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• consolidateA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds 'deterministic', 'Read-only; nothing is changed', and tells the agent to emit one rule record per cluster, which is useful context on top of the readOnlyHint and idempotentHint annotations. The word 'write' is slightly ambiguous (could be read as mutation), though the following 'Read-only' mitigates it; no direct contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences with no filler and purpose front-loaded. The only minor flaw is the ambiguous 'write' phrasing, which costs a point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple optional-parameter tool with no output schema, the description gives enough to understand what the call returns conceptually (a rule record per cluster) and its read-only safety. It could be clearer about the exact return payload or cluster format, but it's largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes both parameters, including defaults, ranges, and meaning; the description contributes nothing about min_size or threshold, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action ('deterministic clustering of episodes') and a concrete output ('ONE record with kind='rule'') that generalises them. This clearly separates it from sibling memory tools like recall or transfer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage when multiple episodes represent the same underlying lesson and need generalisation into a rule, but it never explicitly states when to prefer consolidate over siblings or when not to use it. There are no exclusion conditions or alternative tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• amendA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover the safety profile (non-read-only, idempotent, not destructive). The description adds the critical partial-update behavior—only supplied fields are changed—which goes beyond what readOnlyHint/idempotentHint convey. It does not mention return behavior or failure modes, but those are not essential given the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences deliver the core behavior, scope, and an illustrative example with no filler. The critical constraint 'Only supplied fields change' is placed prominently at the end of the description, making it easy for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a patch operation with a rich nested schema, the description plus schema covers the essentials: what to patch, that it is an existing record, and that the update is partial. With no output schema, return-value explanation is unnecessary. The only notable gap is lack of explicit sibling routing, but that is more a usage-guidelines concern.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is roughly 50%: the patch object has 'Fields to replace', but id has no description. The tool description reinforces the partial-replacement meaning and gives a few examples, but it does not enumerate or explain all patch subfields. Overall the schema names and enums are reasonably self-explanatory, so this is adequate but not exceptional.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Patch') and resource ('fields of an existing record'), immediately distinguishing it from create/read/delete operations. The parenthetical examples ('better fix, extra avoid items, corrected context') clarify the kind of amendments intended, and 'Only supplied fields change' pins down the exact scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The word 'existing' implies this is for updates rather than new records, and 'only supplied fields change' clarifies partial overrides. However, no explicit guidance is given about when to prefer this tool over siblings like record, reinforce, or forget, and no alternative conditions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forgetA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true and readOnlyHint=false. The description adds the crucial behavioral trait of permanence ('Permanently remove'), which is not fully captured by the annotations, and specifies what is destroyed (the wrong or obsolete record). No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no filler. The core action ('Permanently remove') is front-loaded, and the condition ('wrong or obsolete') is placed right after, making it instantly scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple delete operation with one required parameter, no output schema, and destructiveness already annotated, the description is nearly complete. The only gap is the lack of explicit parameter explanation, but the overall context is sufficient for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for its single 'id' parameter, and the description does not mention the parameter at all. The tool's purpose implies the id identifies the record to remove, but the description does not explicitly state how the parameter maps to the operation, so it fails to compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Permanently remove a record', which clearly maps to the title 'Delete an experience'. It distinguishes from siblings implicitly—none of the other sibling tools (record, recall, amend, etc.) perform deletion, so the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear condition for use: the record is 'wrong or obsolete'. This tells an agent when to invoke forget versus other actions, though it does not explicitly name alternatives or exclusions. The context is clear and sufficient for most cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recordA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds meaningful behavioral context beyond the annotations: duplicates are merged automatically, the response indicates whether the record was created, merged, or linked to a different fix, and secrets must never be included. This is valuable operational detail that the schema and annotations do not provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: it states the core purpose first, then deduplication behavior, then a privacy rule. Every sentence earns its place and nothing is redundant with the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 10 parameters and no output schema, the description does a good job covering purpose, when to use it, deduplication behavior, and response semantics. It could be more complete by hinting at how records relate to sibling operations like reinforce or consolidate, but the schema carries the parameter details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 90%, so the schema already documents nearly all parameter meanings. The description adds little parameter-level detail, only a stylistic instruction to keep records terse, which does not materially enhance understanding of the 10 parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action and resource: 'Store what happened after solving (or failing to solve) a non-trivial problem.' This makes the tool's purpose obvious and separates it from read/delete/update siblings like recall, forget, and amend, though it does not explicitly name any alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear conditions for use: record after solving or failing to solve a non-trivial problem. It does not explicitly discuss when not to use the tool or point to alternatives, but the when-to-use context is strong enough to guide an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recallA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover read-only and idempotent behavior, so the description adds value by disclosing the return shape ('ranked hits with fix, avoid-list and confidence') and the matching mechanism ('Exact error text in signals enables deterministic matching; semantic + lexical search catches near matches'). This is meaningful behavioral context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the core purpose, and every sentence contributes: purpose, when to call, return content, input strategy, and read-only nature. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only search tool with no output schema, the description adequately covers return expectations, matching behavior, and invocation timing. The input schema handles parameter details, and annotations cover side-effect safety, making the description complete for an agent to select and call the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description compensates by explaining the role of the 'signals' parameter in achieving deterministic matching and near-match search, which adds semantic meaning not present in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Check whether this problem (or a similar one) has been solved before.' This clearly distinguishes it from sibling tools like record, forget, and amend, and the 'Read-only' tag further separates it from mutation tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The instruction 'Call BEFORE investigating' provides explicit timing for when to use the tool, and the description of deterministic vs. near-match signals gives practical input guidance. It does not explicitly name alternatives or exclusion conditions, but the usage context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinforceA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only say non-readonly/non-idempotent/non-destructive; the description adds the meaningful side effects: it 'Updates the record's confidence, which drives future ranking' and appends failed notes to the record's 'avoid-list'. This tells the agent the tool mutates ranking state and memory, beyond what annotations convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four short sentences, front-loaded with the triggering context ('After applying a fix from `recall`') before mechanics. No filler; the 'Feedback loop' label and side-effect/avoid-list details all earn their place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter mutation with no output schema, the description covers purpose, when to call, side effects, and parameter behavior. Nothing critical is missing; return-value details are the only gap and are not essential for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema documents `id` and `note`, but `worked` has no description. The tool description fills the gap by defining it as 'whether it worked' and clarifies `note`'s conditional use ('If it failed... avoid-list'). With 67% schema coverage and the description covering the remaining param plus the conditional behavior, it adds real semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: 'report whether [a recalled fix] worked' and names its role as a 'Feedback loop' after `recall`. This clearly distinguishes it from siblings like `recall` (which retrieves fixes) and `amend` (which edits records); an agent can tell what `reinforce` does without opening neighboring definitions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly frames when to call: 'After applying a fix from `recall`, report whether it worked.' It also gives conditional guidance for the failure case ('If it failed, pass a short note...'). It doesn't enumerate exclusions vs alternatives, but the trigger condition is unambiguous enough for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transferA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond the annotations, detailing the JSONL format, path resolution rules, idempotent merge behavior, conflict resolution (newer wins, duplicates merged), and the important caveat that embeddings are not transferred and are recomputed. This gives the agent substantial behavioral transparency beyond the simple readOnly/destructive/idempotent hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the single word 'Portability' establishes intent immediately, followed by three dense, purposeful sentences covering modes, path behavior, merge semantics, and embedding caveats. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and moderate parameter count, the description covers all necessary operational details: mode selection, input/output destinations, idempotency, conflict handling, and the embedding recomputation caveat. An agent has enough information to call the tool correctly and predict its side effects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 67%, with path and jsonl already described in the schema. The description adds significant semantic value by explaining the mode enum values in context, clarifying how path is resolved relative to the transfer directory, and defining idempotent merge semantics for import. This meaningfully supplements the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb-resource pairing: export or import the catalogue, with specific modes for each direction. The opening 'Portability' frames it as a bulk data movement operation, which distinguishes it from the sibling memory operations like record, recall, or forget.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly explains when to use export vs import and how each mode behaves. It does not explicitly name alternatives or say 'use this instead of X', but the portability framing combined with the detailed mode semantics gives a clear context for choosing this tool over per-record siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.