Server Quality Checklist

Tool Scores

• search_identifiersA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses some behavioral traits: search is case-insensitive, query formatting requirements, and that results include file, line number, and context. However, it doesn't mention performance characteristics, error handling, or other operational details like rate limits or authentication needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and well-structured with clear sections for purpose, usage notes, arguments, and returns. Each sentence adds value, though the formatting could be slightly more front-loaded by moving the 'Args' explanation closer to the beginning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (6 parameters, no annotations, but with output schema), the description is reasonably complete. It explains the purpose, parameters, and return format. The presence of an output schema means the description doesn't need to detail return values, which it correctly avoids. Some behavioral context could be added, but overall it's adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates well by explaining all 6 parameters in the 'Args' section, adding meaning beyond the bare schema. It clarifies that 'project_root' must be an absolute path and explains what each boolean parameter controls. The description provides essential semantic context that the schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for identifiers in code files and returns matching identifiers with file, line number, and context. It specifies the resource (identifiers in code files) and verb (search), distinguishing it from the sibling 'repo_map' tool which likely provides a different functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage guidance about how to format the search query (identifier name without special characters, case-insensitive), but it doesn't specify when to use this tool versus alternatives or mention the sibling 'repo_map' tool. There's no explicit guidance on use cases or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• repo_mapA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: it generates a map with ranking boosts for different file types, includes related files, respects a token limit, can exclude unranked files, supports cache refreshing, and returns a structured dictionary with map and report. It doesn't mention permissions, rate limits, or error handling beyond the returns statement, but covers core operational traits well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, but becomes verbose with extensive parameter documentation that might be better suited to the schema. While informative, the length could be optimized; some parameter details (like defaults) are redundant if the schema is well-structured. Every sentence adds value, but the structure mixes high-level purpose with low-level param specs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (10 parameters, no annotations, output schema provided), the description is highly complete. It explains the tool's purpose, parameter semantics, ranking logic, and return structure. The output schema existence means the description doesn't need to detail return values, and it adequately covers behavioral aspects. No significant gaps remain for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage and 10 parameters, the description compensates fully by providing detailed semantic explanations for each parameter. It clarifies the purpose, constraints (e.g., 'must be an absolute path!' for project_root), ranking hierarchies (e.g., chat_files get 'highest ranking'), defaults, and optionality. This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a repository map for the specified files, providing a list of function prototypes and variables for files as well as relevant related files.' It specifies the verb ('generate'), resource ('repository map'), and scope ('for the specified files'). However, it doesn't explicitly distinguish this from its sibling tool 'search_identifiers', which appears to be a different operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through parameter explanations (e.g., 'files that are currently in the chat context' for chat_files), but doesn't provide explicit guidance on when to use this tool versus alternatives like 'search_identifiers'. It mentions ranking boosts for different file types, which hints at prioritization logic, but lacks clear when/when-not directives or named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.