Server Quality Checklist

Tool Scores

• statsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does not explicitly state that this is a safe read-only operation, nor does it describe potential side effects, performance characteristics, or required permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the core purpose and avoids any filler. Every word contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a one-parameter tool with no output schema, but it lacks specific details about what 'file and line statistics' includes (e.g., counts, code vs. blank lines) and the return format. It is not misleading but leaves room for ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context beyond the schema by clarifying that 'root' is optional and that omitting it applies to all trees. This supplements the schema's brief 'Optional project root' description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's purpose as providing per-language file and line statistics, with a scope of an indexed tree or all trees. It distinguishes from siblings by the unique 'statistics' resource, though it lacks an explicit action verb.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given about when to use this tool versus alternatives. The mention of 'indexed tree' implies a prerequisite that data must be indexed, but it does not elaborate on conditions or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rootsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It states a read-only list operation but does not disclose return format, potential volume, ordering, authentication requirements, or any other side effects. The minimal sentence offers little beyond the literal action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single eight-word sentence that states the action directly, with no filler. It is appropriately sized and front-loaded, scoring high for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description conveys the core purpose but leaves the term 'indexed root' undefined and does not specify the output structure. However, for such a simple operation, this is a minor gap, so the description is mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty and there are zero parameters, so the description has nothing to add semantically. Per the rubric, 0 parameters merits a baseline of 4, and the description appropriately focuses on the operation itself rather than parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' with a clear resource 'every indexed root in the global database.' It distinguishes from sibling tools like search, index, and stats by focusing on enumeration of roots rather than querying or modifying them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling tools (search, index, stats). There is no mention of alternatives, prerequisites, or exclusions, leaving the agent to infer usage from the tool name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• indexA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. The incremental note ('only changed files are re-read') is a useful behavioral disclosure. However, it doesn't describe side effects or prerequisites, which keeps it at a minimum viable level.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, no filler, information-dense.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple indexing tool with well-documented parameters, the description provides essential purpose and a behavioral caveat. No output schema is present, so the outcome is implied by 'searched later', which suffices.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline of 3. Description adds no extra parameter details, just restates the primary action.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'Index' with resource 'a directory' and outcome 'so it can be searched later'. Distinguishes from sibling tools 'search', 'stats', 'roots' by the action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'so it can be searched later' implies a preparatory use case. Does not explicitly name alternatives or exclusions, but the purpose alone differentiates it from search.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that matching is literal (not regex/fuzzy) and describes the return format, providing useful behavioral detail. It does not discuss edge cases like case sensitivity, but those are not critical for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no fluff, front-loading the core purpose and then adding return-value detail. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with a fully descriptive schema, the description covers the essential behavioral and output aspects. It does not need to explain return values (no output schema exists, but the description already does) or additional edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with all parameters already described clearly. The description reinforces the 'literal substring' aspect for the query parameter but does not add significant meaning beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches previously indexed files for a literal substring and specifies the output (matching lines with file path and line number). This distinctively differentiates it from siblings like index, stats, and roots.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'previously indexed files' implies the tool should be used after indexing has occurred, giving clear context. It does not explicitly name alternatives or exclusions, but the context is unambiguous enough for a simple search tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.