Server Quality Checklist

Tool Scores

• get_feature_detailC
  2.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden of disclosing behavioral traits. It reveals that matching is done against detected routes, controllers, services, etc., giving some insight into the search scope. However, it does not discuss edge cases (e.g., no match, multiple matches) or the nature of the returned context, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at two sentences, with no wasted words. It front-loads the main action and adds relevant matching details. However, it could be slightly more structured by explicitly linking parameters to their roles.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has two required parameters, an output schema, and sibling tools, the description lacks crucial information about the 'path' parameter and usage context. It does not set expectations for the return value or when to prefer this tool over alternatives, making it incomplete for effective selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and two undocumented parameters, the description adds minimal semantic value. It only implies that 'name' is the feature name, leaving 'path' entirely unexplained. This is insufficient for correct parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as retrieving detailed code context for a feature by name. It specifies that matching occurs against routes, controllers, services, etc., distinguishing it from sibling tools like search_symbols or find_features. However, the 'path' parameter is not explained, causing slight ambiguity in the full scope of the tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit guidance on when to use this tool versus its siblings. While the purpose implies use when needing detailed context for a named feature, there is no mention of alternative scenarios or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_featuresC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must disclose behavioral traits. It does not mention whether the operation is read-only, destructive, or any side effects. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded purpose with a clarifying second sentence on filtering. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Lacks usage guidance, parameter details beyond examples, and behavioral traits. While output schema exists, the description does not help with context like path format or filter case sensitivity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. The description adds meaning for type_filter by providing examples, but the required path parameter lacks semantic explanation beyond its name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool detects features, API routes, controllers, etc. in the codebase, with optional type filtering. This distinguishes it from siblings like search_symbols but lacks explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like analyze_structure or get_feature_detail. The description is purely declarative without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_symbolsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions regex usage and optional filtering but lacks details on performance, error handling, or whether the operation is read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core function. No wasted words; efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and an output schema (implied), the description is too minimal. It lacks guidance on regex syntax, prerequisites, and behavior when no matches are found.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description adds meaning by stating pattern is regex and file_ext filters. However, the 'path' parameter is not explained, and the scope 'across the codebase' is ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for symbols/patterns using regex across the codebase and optionally filters by file extension. This distinguishes it from sibling tools like analyze_structure or find_features. However, it could be more precise about what constitutes a 'symbol/pattern'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The description only mentions optional file extension filtering but does not compare to sibling tools or provide context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_structureA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, description carries full burden. It lists outputs but lacks details on performance, scope, or any destructive potential. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no redundancy, front-loaded with purpose and outputs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given complexity of codebase analysis and presence of output schema, description covers key aspects. Could add more about supported languages or limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% coverage (no parameter descriptions). Description clarifies 'path' as codebase root but does not explain 'depth' parameter. Partial improvement over schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool gets a high-level overview including specific outputs (languages, file count, entry points, directory tree). It distinguishes from siblings that focus on features, details, project types, or symbols.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied usage for initial codebase exploration, but no explicit guidance on when to use vs alternatives (e.g., find_features) or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_project_typesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It indicates a read-like operation ('list') but does not disclose whether any side effects exist, authentication requirements, or rate limits. While the tool is likely safe, the description is minimal and does not add significant transparency beyond the action itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that conveys the core purpose without any unnecessary words. It is concise and front-loaded, making it easy for an agent to quickly understand the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (zero parameters, output schema present), the description is largely complete. However, it could briefly mention the output format or that it returns a list of types/frameworks. Still, it is sufficient for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema description coverage is 100% (none to cover). The description adds meaning by specifying what is listed ('project types and frameworks'), which is valuable context beyond the empty schema. A baseline of 4 is appropriate given no parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: listing all project types and frameworks. It uses a specific verb ('List') and resource ('project types and frameworks'), and distinguishes itself from sibling tools like analyze_structure or find_features by focusing on enumeration rather than analysis or search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, context, or situations where siblings might be preferred. The agent receives no help in decision-making beyond the basic action.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.