Server Quality Checklist

Tool Scores

• statusB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It lists the output fields, which gives a good sense of what the tool returns, but it does not explicitly state that it is read-only, what happens for invalid repo names, or any performance considerations. The absence of side-effect disclosure is less critical for a status tool, but still a gap that prevents a higher score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that front-loads the core purpose ('Per-repo health') and then enumerates the specific metrics. Every word contributes meaning, with no redundancy or filler. It is extremely concise while retaining all essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no nested objects, no output schema), the description provides a solid list of return fields, giving the agent a clear idea of what to expect. However, it omits potential error behavior (e.g., repo not enrolled) and does not clarify whether results are real-time or cached. For a basic status tool, this is nearly complete, but the missing failure-mode context prevents a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter 'repo', which is described as 'name of an enrolled repo'. The tool description does not add much beyond this, but the phrase 'Per-repo' reinforces the parameter's purpose. Since the schema already handles the parameter meaning, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as providing per-repo health metrics, listing specific data points (last indexed, staleness, parse errors, coverage %, snapshot count, latest stored hash, file cap). While it lacks an explicit verb like 'get' or 'show', the resource and scope are clear. It distinguishes from sibling tools like 'health' by focusing on repository status, though not explicitly naming alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There are no explicit context cues, exclusions, or references to sibling tools. The description implies it is for checking repo health but does not explain when one might prefer 'health' or 'structure' instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• healthA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description lists the output fields (schema version, integrity check, enrolled repos, db path), giving some transparency about return content. However, it does not disclose whether the integrity check is expensive, whether the tool is read-only, or any side effects, and no annotations are present to fill that gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded phrase 'Store-wide health' followed by a colon and a list of relevant items. Every word contributes, no filler, and it is easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema and no annotations, the description sufficiently communicates what the tool does and what information it provides. It could be more complete by addressing potential overlap with siblings, but the tool is simple enough that this is minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the input schema is trivially complete. The description adds value by enumerating the health data returned, which is the only semantic content needed for this parameterless tool. Baseline 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's function as a store-wide health check, listing specific items like schema version, integrity check, enrolled repos, and db path. This is specific enough to distinguish from siblings like structure or diff, though it doesn't explicitly contrast with 'status' which could be similar.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as 'status' or 'locate'. The description implies it is for overall health, but there is no explicit mention of scenarios or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hashA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral traits. It adds meaningful context beyond a simple 'computes hash' by emphasizing determinism and cross-machine reproducibility. It does not mention side effects, but hashing is inherently non-destructive, and the 'current code' phrasing implies a read-only operation on the repo's existing state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the core purpose in the first sentence and a valuable behavioral trait in the second. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description adequately explains the return value ('root hash + file count') and a key behavioral guarantee (determinism). It could mention the hash format or error handling for unenrolled repos, but the description is largely complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% because the single parameter 'repo' is fully described as 'name of an enrolled repo'. The description adds minor extra context by saying 'current code', which clarifies that the hash is computed from the repo's present state, but it does not significantly enhance the parameter's meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool computes a 'deterministic root hash + file count' of 'an enrolled repo's current code', using a specific verb (compute/generate) and resource (repo code). It distinguishes itself from siblings like structure, diff, and status by focusing on a hash fingerprint.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The tool's purpose is narrow and the deterministic property ('Same code → identical hash across machines') implies a usage case for verifying consistency across machines. However, there is no explicit guidance on when to prefer this over siblings (e.g., diff or status), and no exclusions or alternative tool mentions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• diffA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the operational behaviors for every mode without hiding assumptions or side effects. There is no mention of side effects or permissions, but for a read-only diff operation, the behavior is transparent enough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, followed by a precise enumeration of modes. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all parameter combinations and default behavior, and with no output schema, return details are not required. It is slightly thin on the definition of 'structural drift' and possible error conditions, but overall it gives a complete mental model for using the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds meaningful semantic grouping: it explains that a and b are used together, `since` uses a label against live code, and `session` pins the reference snapshot. This goes beyond the schema's literal parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Structural drift for an enrolled repo' and enumerates three distinct comparison modes (snapshot-snapshot, snapshot-live, latest snapshot-live). This signals a specific verb+resource and differentiates it from sibling tools like 'structure' or 'status'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context by defining when each mode applies: with a+b for snapshot diffs, with `since` for snapshot-vs-live, and default for latest-vs-live. It doesn't name alternative tools, but the decision logic is clear and tied to argument combinations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• locateA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It reveals deterministic parsing from the live AST, zero-match definitiveness, capped results, pagination via offset, and default filtering to functions+classes. These details far exceed minimal expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences pack the core purpose, usage, and rationale without redundancy. The description is front-loaded with the primary action and organizes supporting details logically, earning its length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description defines the return format (file:line + body hash), covers cap and pagination behavior, and explains the zero-match result as definitive. Combined with the rich schema, the tool is fully comprehensible without further context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so a baseline of 3 is appropriate. However, the description adds a key behavioral nuance not in the schema: a named symbol lookup searches every kind. This supplemental meaning justifies a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('locate') and resource ('symbols in an enrolled repo') with an explicit output mapping (name → file:line + body hash). It clearly distinguishes itself from sibling tools like structure, diff, and hash by focusing on deterministic definition lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool ('verify a symbol exists before claiming it does') and contrasts it with grep, explaining why grep is unreliable. However, it does not explicitly name sibling tools or provide when-not-to-use exclusions, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• structureA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and excels: it discloses determinism, current-code snapshot, cost considerations for detail levels, and even flags that the `doc` field is unverified author intent. This is exemplary transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The single-paragraph description is dense yet readable, with every clause adding concrete value—no filler or repetition. It front-loads the core purpose before diving into parameter trade-offs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and rich enum parameter, the description covers all necessary operational context: what is returned per detail level, how to use paths, cost implications, and a caveat about data reliability. Without an output schema, this description fully compensates.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds substantial value beyond the schema by explaining each detail level's content, token costs, and when to avoid hashes. It also clarifies the `doc` field's caveat, which is critical for correct interpretation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the deterministic structure (files + symbols) of an enrolled repo's current code, with a specific use case: grounding claims about functions/types/exports. It distinguishes itself from sibling tools by focusing on structural inspection rather than diffs, hashes, or status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is given for when to use the tool ('to ground claims about what functions/types/exports exist') and how to scope it with paths and detail levels. It does not explicitly mention alternatives, but the strong purpose statement effectively differentiates it from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.