Server Quality Checklist

Tool Scores

• kb_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes what the tool returns in both modes (notes by type, stale counts, per-path freshness). No annotations exist, so description carries burden; it is transparent about the query nature, though it does not explicitly state it is read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each covering a distinct mode (no paths vs. with paths). No wasted words; highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description clearly specifies what is returned for both invocation modes (overview counts, per-path freshness). For a tool with one optional parameter, this is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides a description for the single parameter 'paths'. The tool description adds context by explaining the behavior difference and the concept of 'anchored notes' and 'freshness', adding value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool provides a notebook overview with counts by type and stale/superseded status, and describes an alternative mode for specific paths. Distinguishes from siblings like 'kb_search' (likely searches content) by focusing on status and anchoring.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to pass the 'paths' parameter ('a quick is anything known here? check') and implies using without paths for a full overview. Does not explicitly mention when not to use or alternatives among siblings, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kb_searchA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes freshness stamp ([fresh], [evidence changed]) and that absence notes are re-checked live. Results are inlined in full. No annotations provided, so description carries full burden; sufficiently transparent for this read-only search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured: first sentence states purpose, then usage guidance, then result details. Every sentence adds value. Slightly verbose but remains clear and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations or output schema, the description covers purpose, usage, result format (freshness stamp), and fallback behavior. It also guides on re-verification. Complete for a search tool with one parameter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter 'query' with schema coverage 100%. Description adds value by providing examples ('symptoms work: "logout loop after refresh"') and mentioning the 'over-supply rule' from find, which guides the agent to provide all salient tokens.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it searches the repo's NOTEBOOK for durable notes from past agents, specifies search terms (plain task words, symptoms, symbol names, file names), and distinguishes from sibling 'find' by recommending use before find.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to try this BEFORE find when the task might have been seen before, explains that hits can answer or point to files, and describes fallback behavior ('No hits or an empty notebook → fall through to find, no tax'). Also mentions re-verification with kb_write if evidence changed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kb_repairA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral burden. It does well: explicitly states the tool 'Returns a worklist, never a change', describes the empty-notebook sentinel ('Nothing to repair here.'), and clarifies that notes written before the fields were required are 'correct but unreachable' — disclosing the tool's non-destructive diagnostic nature. It doesn't over-claim error/rate behavior, but the safety profile (read-only worklist) is clearly conveyed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three well-organized sentences: purpose, what-it-returns/next-steps, and usage trigger. Every sentence earns its place with concrete, actionable detail. The key information (returns worklist, never a change; fix with kb_write passing id; merge semantics; empty-notebook sentinel) is front-loaded and complete.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter diagnostic tool with no output schema, the description is complete. It covers what is found, why notes are unfindable, what the tool does NOT do (doesn't repair), how to fix each issue (kb_write + id + merge), the clean-notebook output, and when to run it. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters and schema coverage is 100%, so there are no parameters to document. Per the rubric, 0 params = baseline 4. The description adds no param-specific detail because none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource ('List the notebook notes') with a clear scope ('WRITTEN BUT UNFINDABLE'), and gives concrete examples of what makes a note unfindable (missing identityAliases, anchor symbols, verified paths). It clearly distinguishes this from siblings by framing it as a diagnostic/worklist tool rather than a search or write tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to run it ('when the user asks to repair, fix, or clean up the notebook') and contrasts with kb_write ('Fix each one with kb_write'). It also warns that repairing is the user's (agent's) work, requiring judgement about code, and clarifies the tool returns a worklist never a change. The merge semantics are stated so the agent knows replays don't lose data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden — and it delivers: symbol reordering/truncation beyond 20 symbols, the exhaustive 'do not grep' semantics of importers under match, and the 'Edited together is a HABIT, not a dependency' caveat with an explicit 'do not go looking for one' instruction. It also discloses absence conditions (no git history, weak pairing), rendering scope ('Rendered in EVERY view'), and the in-tool grep fallback for non-declared symbols. No annotation contradiction is possible since no annotations exit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well front-loaded with a one-sentence purpose ('Drill into a known file. Returns these sections as compact text:') and a scannable bullet structure for the five sections. The length is largely justified by dense anti-error guidance, but it loses a point for redundancy: 'Rendered in EVERY view, including a narrowed one' appears twice, and parameter prose overlaps the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with no annotations and no output schema, this is complete: every section type is defined, non-obvious absence conditions are explained, all four parameters are documented, and explicit stop/continue heuristics are given. An agent could select, invoke, and interpret results correctly without any additional lookup.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3; the description adds value beyond the schema by explaining when each parameter matters — match for god-files with concrete exaples ('match: "auth"', '/^handle/'), and symbol as a replacement for the read-at-off-set hunt, including its override behavior and grep fallback. Some duplication with the schema's own parameter text remains, so it stops short of 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a concrete verb+resource ('Drill into a known file') and enumerates the exact output sections (Symbols, Imports, Importers, Edited together, Related), thoroughly scoping what the tool does. It also differentiates it from siblings and the Read alternative by positioning it as the follow-up to find and as 'the right tool for who uses this file / who calls this symbol'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit when-to-use ('Use this AFTER find surfaces a candidate file', 'Prefer this over Read when you need shape, neighbors, or usage') and when-not-to ('Reach for Read only for implementation details inside a method body'). It even names a stopping rule — after 5+ gs calls for one question, go back to find with a sharper glob — an unusually precise guard against enumeration.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kb_repair_aliasesA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and delivers richly: explains the pagination shrink behavior, the aliasesVerified mechanic that stops reappearance, the resurfacing of hidden aliases on the next fold, and the exact return string 'No file/flow notes to reconcile.' Also warns about stale nextOffset silently skipping entries. Exceptionally transparent about behavioral edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is thorough but densely packed with genuinely useful operational detail. It front-loads the core purpose first, then branches into reconciliation workflow and pagination caveats. Each sentence earns its place, though it's long enough that a lighter-weight summary of the critical warnings could improve scannability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema and no annotations, this description compensates fully. It covers the return value ('No file/flow notes to reconcile.'), pagination behavior, workflow dependencies (kb_write, aliasesVerified flag), and edge-case pitfalls. For a complex reconciliation tool with no structured metadata support, this is exceptionally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaningful value beyond the schema: it explicitly warns about the offset pitfall ('do NOT increment this across calls once you start marking notes aliasesVerified') and clarifies that offset is for browsing only, while reconciled notes auto-drop. This is useful semantic depth beyond what the schema description states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states precisely what the tool does: lists file/flow notes whose identityAliases have gone stale, explicitly distinguishes it from kb_repair (no aliases at all vs. stale aliases). The verb 'List' plus the specific resource (file/flow notes with stale identityAliases) is specific and immediately differentiated from the sibling kb_repair.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use vs. when-not-to: 'a DIFFERENT problem from kb_repair' naming the exact alternative. Also gives explicit operational guidance: don't increment offset while reconciling, call again with offset 0, never retract mechanically, end every kb_write with aliasesVerified:true. This is exemplary usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• findA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully describes behavior: how it greps body and declared names, ranking by distinct-term coverage, output format (inline preview vs bare paths), partitioning of prose/stylesheets, and naming insensitivity. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is thorough and well-structured with sections (HOW IT WORKS, OUTPUT, NAMING, AFTER THE RESULT). It is slightly verbose but every sentence adds value, and the key points are front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of the tool and lack of output schema, the description is exceptionally complete, covering input, mechanism, output format, naming conventions, synonyms, and post-result instructions. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter, but the description adds significant value beyond the schema by explaining effective query construction (pass all salient identifiers, more tokens = sharper ranking) and the underlying search mechanism.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Locate the files relevant to a task.' It uses a specific verb and resource, and explicitly distinguishes from siblings by advising 'Reach for find BEFORE Read/Grep/Glob.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage guidance: pass all salient identifiers, not one keyword; explains when to use (before Read/Grep/Glob) and when not to (do not grep spelling variants of absent tokens). Also includes post-result actions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kb_lookupA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description fully discloses behavior: returns facets, cross-file flows, lessons/absences with freshness stamps, explains drift detection, and notes the optional symbol narrowing. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured and front-loaded, but slightly verbose. Every sentence adds value, but could be tightened. Still concise enough for the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description provides comprehensive context: return types, freshness stamps, drift warnings, and usage scenarios. Missing output format is acceptable as it's implied.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of params with descriptions, but the description adds meaning: 'Address-keyed, not concept-keyed' for path, and 'narrow to one top-level symbol' for symbol, plus context on exhaustive lookup.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads notebook knowledge at a specific file path, using 'Read everything the NOTEBOOK knows at ONE exact address'. It also distinguishes itself from kb_search by specifying exact lookup vs fuzzy search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance: 'Reach for this the moment you have DECIDED on a file and are about to modify it.' It contrasts with kb_search and explains how to interpret results (clean result = positive signal, re-verify drifted stamps).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kb_writeA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It thoroughly discloses key behaviors: the two-phase reuse gate returning `{status:"candidates"}` instead of writing, non-atomic multi-spec writes (malformed notes reported together but good ones written), a coverage line at the end, flow-evidence checks with session, and the git/publishing boundary. It also explains the consequences of missing required fields ('written but unreachable') and the stability/replacement semantics of identityAliases vs incidentAliases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured with clear paragraphs for purpose, reuse gate, spec shape, required fields, and git behavior. It is front-loaded with the core purpose. However, there is redundancy: the 'REQUIRED for the note to be findable' paragraph is nearly duplicated in the spec parameter description, and incidentAliases semantics appear twice. This prevents a perfect score while still being appropriately detailed for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description must explain return behavior and does so thoroughly: candidate responses, coverage lines, malformed note handling, and the flow-evidence warning. It covers the two distinct call modes (`spec` vs `specs`), the reuse gate interaction, required fields for findability, and non-commit behavior. This is comprehensive for a tool with six parameters and nested object inputs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Though schema coverage is 100%, the description adds substantial semantic context beyond the schema. It explains the relationship between `spec`/`specs`, the phase-2 meanings of `into` and `is_new` (merging vs declaring new), the purpose of `session` for flow-evidence checks, and `agent` for the coverage line. It also details the `spec` shape and required fields per type, enriching the bare schema definitions significantly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Save or correct NOTEBOOK notes after finishing real work here', using a specific verb (save/correct) and resource (NOTEBOOK notes), and clearly distinguishes this write tool from read/search/repair siblings like kb_search, kb_lookup, and kb_repair. It also explicitly states it handles fixes and retractions in addition to initial writes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'after finishing real work here — you have the files in context, so no future agent is better placed to record what you learned.' It also explains when to use `specs` vs `spec`, describes the two-phase reuse gate process, and clearly states what the tool does NOT do ('never commits to git — publishing notes is a human-only step'), providing sufficient context and exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.