Server Quality Checklist

Tool Scores

• debug_dump_graphA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior. It claims to 'Dump the entire graph structure' but the schema reveals a default limit of 50 nodes (limit_nodes), contradicting the 'entire' claim. Additionally, it does not note potential performance issues or truncation behavior for large graphs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the DEBUG prefix and a clear action. Every word earns its place, and it effectively communicates the purpose without unnecessary filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description mentions return formats, which is helpful given no output schema. However, it fails to explain the default node limit that contradicts the 'entire graph' phrase, which is critical for understanding potential truncation. This omission makes it incomplete for a tool with a configurable limit.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions cover 100% of the parameters with detailed explanations (e.g., format, show_edges, limit_nodes). The description adds minimal semantic value beyond repeating the output formats ('Mermaid diagram, JSON, or Graphviz DOT format'), so the baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Dump the entire graph structure for visualization.' The verb 'dump' and resource 'graph structure' are specific, and the DEBUG prefix distinguishes it from sibling tools like find_symbol or get_context, which focus on focused code navigation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context with 'Useful for debugging and understanding the graph structure,' indicating when to use this tool. However, it does not explicitly exclude any alternatives or mention sibling tools, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• expand_sourceA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden for behavioral disclosure. It reveals the method ('using token-level source maps') but does not state whether the operation is read-only, what happens on missing symbols, error handling, or return format. This is a significant gap for a tool with no annotation safety net.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with zero waste. The first sentence states what and how; the second gives usage guidance. Every phrase earns its place, and the structure is front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the tool is relatively simple and the description adequately explains its purpose and usage. The return value (full source code) is implied by the action. However, it could be slightly more complete by explicitly stating the output type or error behavior, but for the complexity level it is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for all four parameters, so the baseline is 3. The description does not add any parameter-specific meaning beyond the schema; it only mentions 'symbol' generically. The schema itself clearly explains parameter usage (e.g., 'Use either symbol_id or name, not both.'), but the description contributes nothing extra.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Expand a symbol's compact signature to its full original source code using token-level source maps.' It uses a specific verb ('expand') and resource ('symbol's compact signature'), and distinguishes it from siblings like find_symbol and get_context by focusing on source expansion rather than lookup or context retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit conditions for use: 'Use this when the signature is not enough and you need the implementation.' This gives clear context, but it does not mention alternatives or when not to use it, missing the 'when-not' or alternative tool guidance found in higher scores.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_symbolA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses cost ('cheaper'), output ('locations and compact signatures'), and optional parameters. It does not detail matching semantics (case sensitivity, exact match), result limits, or side effects, though it appears to be read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each serving a distinct purpose: purpose, output/benefit, and options. No filler or redundancy. It is front-loaded with the main action and efficiently conveys all necessary context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only lookup tool with 3 well-documented parameters, the description gives a solid overview of expected output (locations and signatures) and performance characteristics. Without an output schema, it does not specify result structure details, but it is sufficient for common use cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and each parameter already has a clear description. The description adds reference to 'filter by language or include call sites' which maps to the parameters, but does not provide additional semantic detail beyond the schema, warranting the baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Find') on a specific resource ('function, class, or method definitions') by name. It clearly distinguishes itself from sibling tools like find_callers by focusing on definitions and locations rather than call sites.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it: 'far cheaper than reading files' suggests using this for symbol lookups instead of reading entire files. However, it does not explicitly mention alternatives or when-not-to-use scenarios, which keeps it from a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_callersA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It does disclose a key behavioral trait (call-graph resolution rather than text search) and implies read-only operation via 'find'. However, it omits potential prerequisites (e.g., indexing) and details about output format or limitations, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first front-loads the core purpose and method, the second adds a practical use case. There is no redundancy or filler, every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description covers purpose, method, and use case. It lacks explicit prerequisites or output structure, but these are minimal for this tool and can be reasonably inferred from the call-graph context and sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and the parameter description already explains what 'name' is. The tool description adds the crucial context that resolution is via the call graph, enriching the parameter semantics beyond the schema's generic explanation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'find' with a clear resource ('functions/methods that call a given function') and explicitly distinguishes itself from text search by stating it is 'resolved through the call graph (not text search)'. This differentiates it from sibling tools like find_symbol and makes the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides a clear use case ('Useful for impact analysis before changing a function') and implies when not to use it by saying 'not text search'. However, it does not explicitly name alternative tools or state exclusions, so it lacks the full explicitness of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• index_projectA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses key behaviors: scanning file types, extracting definitions, resolving relationships, automatic indexing on first query, and the ability to force a full rebuild. It stops short of detailing side effects like performance impact or overwriting behavior, but the core behavior is well covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, and every clause adds value. It efficiently conveys scope, automatic behavior, and usage options without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given only two optional parameters and no output schema, the description adequately covers what the tool does, when to use it, and what to expect (stats and context-footprint reduction). It could mention error conditions or return format, but for this simplicity level, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and both parameters have descriptions that fully explain their meaning and defaults. The tool description mentions 'force=true' and path indirectly, but adds no new semantic information beyond what the schema already provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('index') and resource ('project to build the code graph'). It also specifies file types scanned and what relationships are resolved, making it distinct from sibling tools like find_symbol or get_context which query the graph.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clarifies that indexing happens automatically on first query, implying the tool is usually unnecessary unless a full rebuild is needed or stats are desired. It provides clear context for when to use it with force=true, though it doesn't explicitly mention alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_contextA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It explains the behavior in detail: returns symbol plus connected nodes within a depth, and relationships. It does not disclose edge cases like duplicate symbol_id/name or error handling, but for a read tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler. The first sentence states the action and scope, the second gives a practical usage rationale. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only graph traversal tool without an output schema, the description conveys the return concept (nodes + relationships) and depth semantics. It omits details about response formats or limits, but the schema's format parameter and maximum depth cover some of that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% coverage, so the baseline is 3. The description adds meaning beyond schema by enumerating node types (callers, callees, imports, containers) and framing depth as neighborhood expansion, which enriches the schema's per-parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Get' and names the resource 'contextual neighborhood around a symbol', specifying included node types (callers, callees, imports, containers) and depth. This clearly distinguishes it from siblings like find_callers (specific relationship) or find_symbol (single symbol).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'compact alternative to reading whole files for context' gives clear usage context and implies when to prefer this tool. However, it does not explicitly mention alternative tools or when not to use it, so it's a half-step above basic guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.