Server Quality Checklist

Tool Scores

• find_callersA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It reveals important matching constraints ('directly call' and 'exact fully-qualified'), but it does not explicitly state read-only behavior, return format, or edge-case handling like missing symbols or re-exports.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that immediately communicates the tool's purpose with no filler. It is well-structured and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter tool without an output schema, the description covers the essential input criteria and matching behavior. It does not explicitly state the return value or format, but this is a minor gap given the tool's low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully documents the single 'symbol' parameter with type, requiredness, and an example, so the description adds no new parameter semantics; it merely restates that the symbol must be exact and fully-qualified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Find') and clearly identifies the resource: functions/methods that directly call an exact fully-qualified TypeScript symbol. This distinguishes it from sibling tools like impact_of_change and explain_context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for locating direct callers, but it does not provide explicit when-to-use or when-not-to-use guidance. It also does not contrast this tool with transitive-caller or impact-analysis alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• impact_of_changeA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does convey the key behavior of transitive traversal through 'transitive blast radius' and implies a read-only operation via 'Find'. However, it does not disclose what the output will contain, whether the symbol itself is included, or any potential performance/cost implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. It begins with the active verb 'Find' and immediately communicates the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool, the description covers the core purpose but lacks usage guidance and return-value details. Since there is no output schema, the agent is left to guess what the response will contain. The description is adequate but leaves some context gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value by specifying 'exact fully-qualified TypeScript symbol', which clarifies the required format beyond the schema's 'Fully-qualified symbol'. The 'exact' and 'TypeScript' qualifiers help the agent understand the input more precisely.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb 'Find' and clearly defines the scope: transitive blast radius of changing a TypeScript symbol. It distinguishes from sibling tools by emphasizing the transitive reach, which is broader than just direct callers (as in find_callers) or general context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'transitive blast radius' implies this is for assessing change impact, but the description does not explicitly state when to use it versus alternatives like find_callers or explain_context. No direct comparison or exclusions are mentioned, so usage guidance is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_contextA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden and does a good job explaining the behavior: it finds the most relevant symbol, retrieves structured graph data, and instructs the agent to use that data to answer the user's question. It implies a read-only analysis operation through verbs like 'find' and 'return', though it doesn't explicitly state 'read-only'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core workflow, and has no filler or redundant content. Every clause adds meaningful information about inputs, outputs, or how to use the returned data.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description adequately enumerates the key return items (callers, callees, call-site evidence, two-hop impact) and the self-answering instruction. It doesn't mention limitations or edge cases, but for a single-parameter tool with simple inputs, this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes the only parameter 'question' with an example, and the description adds no extra semantic detail beyond restating it's a natural-language codebase question. With 100% schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: take a natural-language question, find the most relevant symbol, and return callers, callees, call-site evidence, and two-hop change impact. This specific output set distinguishes it from sibling tools find_callers and impact_of_change, which are narrower.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use it: when a natural-language question about the codebase needs a structured graph data answer. It does not explicitly mention sibling alternatives or exclusions, but the usage context is sufficiently clear for an agent to select it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.