Server Quality Checklist

Tool Scores

• relationsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It discloses default depth and limit, and mentions clamping from the schema. It describes the nature of results (neighbourhood with signatures). But it doesn't mention side effects, permissions, or data handling. The output schema covers return structure, so the description adds moderate context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with purpose, then parameter guidance, then usage hint. No wasted words; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists and the description covers input, defaults, and a disambiguation example, it is complete for a tool of moderate complexity. The description provides all necessary context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50%, with depth and limit already documented. The description compensates by adding meaning for 'symbol' (accepts node id or name) and 'file' (disambiguation for matching names). This provides practical guidance beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'A symbol's neighbourhood in one call: who calls it, what it calls, what implements/subclasses it, and non-call references — each with its signature', which is a specific verb+resource. It distinguishes from siblings by claiming 'THE tool for impact analysis and 'what implements X'', clearly differentiating from info and search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear when-to-use guidance (impact analysis, 'what implements X') and explains when to use the file parameter for disambiguation when names match. However, it lacks explicit exclusions or comparisons to sibling tools like info or search.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• infoA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It describes the read-only nature ('Read a specific target'), details return content per target type, explains default limit clamping to 200, and mentions windowing with offset/limit. It does not explicitly state that the tool is non-destructive or require permissions, but the behavioral details are sufficient for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately concise, packing many details into a few sentences. While all sentences contribute value, the text is somewhat dense and could be better structured (e.g., bullet points) for quick scanning. Still, it avoids redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema to cover return values, the description adequately explains the tool's behavior for both symbols and files, including default mode, limit clamping, and disambiguation. It lacks handling of not-found cases or errors, but covers the main use cases comprehensively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With only 20% schema coverage (only limit has a description), the description adds significant meaning to all parameters: target (symbol vs file path), mode (outline vs source), file (disambiguation), offset/limit (windowing defaults). It clarifies defaults and behavior beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Read a specific target', specifying that it takes a SYMBOL (node id or name) or a FILE path and returns structural info or source content. It distinguishes itself from generic file reading by advocating use of mode='source' instead of opening the file directly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use each mode: outline for cheap structural overview, source for actual content with line numbers and import information. It also explains when to use the file parameter to disambiguate symbol names. However, it does not compare this tool to its siblings (relations, search), which is a minor gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description fully covers behavior: returns graph nodes with signatures, not dead text lines; text_matches for non-symbol hits; literal matching; note field indicates no exact matches; exhaustive mode returns all file paths without signatures; test file exclusion logic.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but packed with valuable, non-redundant information. It is front-loaded with the main purpose and each sentence adds necessary detail. Slightly verbose but acceptable given the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the high complexity (4 parameters, multiple behaviors) and the presence of an output schema, the description covers all relevant aspects: parameter details, edge cases (note field, exhaustive, test files), and redirection to sibling tools. It is fully adequate for correct tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 25% (only limit described), but the description compensates by detailing all parameters: limit (default 25, clamped to 200), query (search term), path_glob (scoping with examples and exclusion), exhaustive (boolean for full file listing). Adds significant meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool finds code by name, content, or meaning, and it is the primary search entry point, distinguishing it from the sibling tools 'info' and 'relations' which provide details and relationships.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides extensive usage guidance: use it like grep or search for symbols, how content hits fold, how to scope with path_glob (with examples), test file exclusion behavior, and the exhaustive option. It also implies when to use siblings after obtaining a node.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.