Server Quality Checklist

Tool Scores

• find_calleesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only states what the tool finds but not whether the search is direct or transitive, what the output format is, or whether it reads from the graph without modification. This leaves significant ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The entire description is a single, focused sentence with no unnecessary words. It is appropriately front-loaded and easily parseable by an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should at least hint at the return structure. It does not. Although the params are well-documented, the tool lacks a full behavioral contract, making it adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with each parameter having a description, including the requirement that symbol_id is an exact symbol id from find_definition. The description adds no additional parameter information, so the schema itself does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function (find callees) with a specific verb and resource. It distinguishes from siblings like find_callers, which would be the inverse operation. The phrasing 'from within a given symbol' directly conveys the scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like find_callers or get_file_dependencies. The description does not mention that symbol_id should come from find_definition, although the schema does. No exclusions or comparisons provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_file_dependenciesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavior. It does state the one-hop, bidirectional nature, but it omits important behavioral context such as prerequisites (e.g., a previously saved graph), error behavior when the file is not found, or return format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. It efficiently conveys the tool's purpose and the one-hop constraint, earning the maximum score for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately conveys the core return value (incoming and outgoing direct imports), but it is missing important context about the graph_path parameter and prerequisites. Since there is no output schema, a bit more detail about result structure would improve completeness, though the tool is relatively simple.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both file_path and graph_path. The description adds no extra parameter meaning beyond referring to a 'file', which earns the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Show') and identifies the exact resource ('a file's direct import relationships'), including both directions (what it imports and what imports it). The 'one hop' qualifier clearly differentiates this from transitive dependency tools like impact_analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus its siblings (e.g., find_callers, impact_analysis). The 'directly (one hop)' comment implies scope but does not explicitly name alternatives or exclusion conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_graph_summaryA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits itself. It does not state that the operation is read-only, whether an existing graph is required, or how it behaves if the graph file is missing. The description only lists return content, not side effects or prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately communicates the tool's output with specific examples. It is well-structured and avoids any unnecessary words, making it highly concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one optional parameter, but the description lacks important context such as prerequisites (e.g., graph must already exist), output format, and explicit usage scenarios. The schema and sibling tool names provide some context, making it minimally complete but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides full coverage for graph_path with its description and default value, so the tool description does not need to elaborate. The description adds no parameter semantics beyond the schema, which is expected given 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns summary statistics including file/symbol/edge counts and most-called/depended-on items. This distinguishes it from sibling tools like find_definition and find_callers, which focus on specific queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for high-level overview of the codebase graph, but it does not explicitly state when to use it over the sibling tools, nor does it mention prerequisites like running index_codebase first. No alternatives or exclusions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_definitionA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool returns a file and line, and that it operates by symbol name. However, it does not cover behavior for missing symbols, whether the graph must be indexed, or how multiple matches are handled.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that front-loads the main purpose and includes the key output details. No filler or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool, the description states the essential return value (file and line) and the schema covers the parameters. It lacks explicit return format or edge-case handling, but the tool's simplicity makes this adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers 100% of the parameters, including an example with dotted paths. The description only adds the symbol types (function, method, class) and does not provide additional parameter semantics beyond what the schema gives.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds the file and line for a function, method, or class definition by name. This verb+resource phrasing distinguishes it from sibling tools like find_callers and find_callees, which focus on call relationships.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when a definition location is needed, but it does not explicitly mention alternatives or when to prefer this over siblings like find_callers or impact_analysis. There is no exclusion or comparison context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_callersA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It mentions the use of the real TypeScript type checker and correct cross-file handling, which is helpful, but it omits details about return format, error cases, or performance costs, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, both earning their place. The primary action is front-loaded, and the second sentence provides a necessary usage hint without redundancy. No filler words or circular phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description adequately conveys the outcome (list of callers) and the prerequisite workflow. Given the moderate complexity, it is sufficiently complete for an agent to invoke correctly, though it could mention the exact return structure without harming the score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear parameter descriptions, so the baseline is 3. The description adds practical context by emphasizing that symbol_id must come from find_definition, which reinforces the schema and aids correct invocation, but it does not add new meaning beyond that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action with a specific verb and resource: 'Find every function/method that calls a given symbol.' It also distinguishes the tool from the sibling find_callees (which finds the opposite direction) and the cross-file correctness adds meaningful context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use find_definition first to obtain the exact symbol id, providing a clear prerequisite. It implies superiority over text-matching searches but does not list when-not-to-use alternatives. The guidance is practical and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• impact_analysisA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It explains the traversal semantics (direct or indirect, depth-limited) and the intent, but doesn't disclose whether the operation is read-only, requires a pre-built graph, or any edge-case behavior. This leaves some gaps in behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the action, and the second sentence adds practical context. No verbosity or unrelated details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description effectively communicates the purpose and main use case for a graph traversal tool. However, it omits the dependency on an existing graph file and doesn't describe the output structure, which are notable for an agent expecting to use the result in an edit workflow. Given the schema is rich, the description is mostly complete but has those gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage of all three parameters with descriptions, so the description adds little beyond what the schema already provides. The description's mention of 'depth limit' aligns with max_depth, and 'given file' with file_path, but no new parameter-specific semantics are introduced.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Find') and resource ('file') and explicitly states the transitive, indirect nature with depth limit. It distinguishes itself from sibling tools like find_callers/find_callees by focusing on transitive dependencies, and ties to a concrete use case ('what could break if I change this file').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly states the primary use case for an AI agent about to edit code, giving explicit when-to-use context. However, it does not explicitly name alternatives or when not to use it, which limits full differentiation from sibling tools like find_callers or get_file_dependencies.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• index_codebaseA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It discloses a key behavioral trait: using the real TypeScript type checker rather than regex matching, which indicates accuracy. However, it does not mention side effects like writing a graph file (though implied by graph_path) or potential performance implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the core purpose and method. The usage guideline is placed at the end, but the entire text is efficient with no filler words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only 2 params and no output schema, the description covers the essential purpose, methodology, and usage order. It doesn't explain the resulting graph's format or return behavior, but the schema's graph_path parameter already hints at the output file, and query tools are expected to consume it. Adequate for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% — both root_dir and graph_path have descriptions in the input schema. The tool description adds no additional semantic meaning beyond what the schema already provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool parses a TypeScript/JavaScript codebase into call and import graphs, with a specific verb ('Parse') and resource (codebase). It distinguishes itself from siblings by positioning it as the prerequisite step before query tools, which are all about querying existing graphs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Run this first before using any query tool,' providing clear usage context and ordering. It implies when not to use it (for answering queries) but does not name alternative tools explicitly; the sibling list makes that implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.