Server Quality Checklist

Tool Scores

• find_referencesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses performance ('Fast operation (1-3s)') and mechanism ('efficiently searches the indexed code graph'), which is helpful. However, it does not discuss potential limitations, whether the search includes comments or test files, or any side effects (though likely none). This is adequate but not deeper transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with clear labels (PURPOSE, USAGE, PERFORMANCE, WORKFLOW, TIP) and is front-loaded with the main purpose. It includes some redundancy (the TIP repeats the impact analysis point from USAGE) and uses emojis, but each section adds value. Overall, it is well-organized and not overly long for the information conveyed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description is quite complete. It covers purpose, usage, performance, workflow, and a practical tip. It does not describe the exact return format, but that is less critical without an output schema. The description gives enough context for an agent to use the tool correctly in a typical workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with a description for 'symbol' ('Symbol name to find references for (exact or partial match)'). The tool description itself does not add additional meaning about the parameter beyond what the schema already provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Find all references to a symbol throughout the codebase' and elaborates with 'Discovers everywhere a symbol is used, imported, or referenced.' This is a specific verb + resource, but it does not explicitly differentiate itself from sibling tools like find_callers or find_callees, which also deal with symbol relationships.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'Use after find_definition to understand how a symbol is used' and 'Essential for impact analysis before refactoring.' It gives a workflow (find_definition → find_references → ...) and a use case, but does not mention alternatives or when not to use this tool, stopping short of the highest rating.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_calleesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It adds performance context ('Fast operation (1-2s) - leverages indexed call relationships'), but it doesn't clarify whether results include only direct callees or transitive calls, nor does it describe the return format. These limitations matter for a dependency-mapping tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with labeled sections (PURPOSE, USAGE, PERFORMANCE, WORKFLOW, TIP), each adding meaningful context. It is longer than a minimal two-sentence description but not bloated; every section contributes practical guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter, no output schema), and the description covers purpose, usage, workflow, and performance. It lacks an explicit mention of return values and direct-vs-transitive scope, but the workflow and tip provide enough context for an agent to select and invoke the tool correctly in common scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the only parameter ('function') is already clearly described in the schema. The description doesn't add significant parameter-level detail beyond what the schema provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Find all functions called by the specified function.' It clearly distinguishes from siblings like find_callers, find_definition, and dependency_analysis by focusing on outgoing dependencies of a single function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The USAGE section explicitly says when to use it: to understand function complexity, identify extraction opportunities, or analyze dependency chains. The WORKFLOW provides a suggested sequence (find_definition → find_callees → complexity_analysis), giving clear practical context, though it doesn't explicitly state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_callersA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses performance ('Fast operation 1-2s') and the underlying mechanism ('pre-built call graph'), which adds useful context. However, it does not explicitly state that the operation is read-only or describe error handling or edge cases, leaving some transparency gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with labels (PURPOSE, USAGE, PERFORMANCE, WORKFLOW, TIP) and front-loaded with the core action. It is slightly longer than necessary, with some repetition of the purpose in the TIP, but every section contributes meaningful context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description covers purpose, usage, performance, and workflow, giving an agent enough context to decide when and how to use it. It does not explicitly describe the return format or behavior when no callers are found, but these are inferable from the stated purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a descriptive parameter entry ('Function name to find callers for (supports method names with class context)'). The description adds no additional parameter details beyond the schema, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Find all functions that call the specified function' with a specific verb and resource. It further explains it identifies the call hierarchy, which distinguishes it from sibling tools like find_callees, and is well-scoped.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear usage context: 'Use when analyzing function dependencies, planning refactoring, or understanding code flow patterns.' It also includes a workflow showing where find_callers fits relative to find_definition and find_callees, but it does not explicitly mention alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_definitionA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It adds valuable behavior details: performance (sub-second), reliance on a cached code graph, and partial-name matching with an example ('MyClass' will find 'MyClassImpl'). It does not cover error cases or return format, but for a simple read-only lookup the disclosed behaviors are useful and non-obvious.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with labeled sections (PURPOSE, USAGE, PERFORMANCE, WORKFLOW, TIP) and front-loaded with the core purpose. It is slightly verbose due to emojis and some redundancy between the first line and the PURPOSE section, but every section adds useful context. Not perfectly concise, but effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema and no annotations, the description is remarkably complete. It covers the tool's purpose, usage context, performance, matching behavior, and workflow integration. All key aspects an agent needs to select and invoke the tool are present, making it a fully self-contained description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds extra meaning to the 'symbol' parameter by explaining partial matching and advising 'Use exact names for precision.' This goes beyond the schema's simple 'supports partial matching' and gives the agent a clearer understanding of the parameter's behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: locating the definition of a symbol (function, class, variable, method). It uses specific verbs like 'find' and 'locates', and explicitly mentions the resource (symbol definitions). It distinguishes itself from sibling tools like find_references and find_callers by focusing on the original definition location rather than usage patterns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'Use after analyze_codebase when you need to understand where a specific symbol is implemented.' It also outlines a workflow (analyze_codebase → find_definition → examine context → find_references/find_callers) and mentions what the tool is not for ('find_references/find_callers for usage patterns'). It lacks explicit 'when not to use' statements but the workflow and alternative mentions implicitly guide the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• project_statisticsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It adds useful behavioral context: performance expectations (1-3s), data source ('pre-calculated metrics from the code graph'), and interpretation of health score (below 7/10 indicates issues). This goes beyond basic read-only implication, though it doesn't explicitly state side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with clear sections (purpose, usage, performance, workflow, tip), making it scannable. It's slightly verbose for a zero-parameter tool, but every section adds value—especially the workflow and tip. It earns a 4 rather than 5 due to length, but it's well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description is remarkably complete: it explains the tool's purpose, high-level metrics, performance characteristics, integration into a workflow, and how to interpret the health score. It covers all necessary context for an agent to decide when to use this tool and what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has zero parameters, so the baseline is 4. The description adds meaning by listing the metrics the tool calculates (file counts, language distribution, complexity trends, health score), which helps the agent understand what the tool outputs even without an output schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb+resource: 'Get comprehensive project statistics and health metrics,' then details what it provides (file counts, language distribution, complexity trends, health score). This distinguishes it from sibling tools like find_definition or complexity_analysis, which are more focused.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit use cases: 'project assessment, progress tracking, generating project reports.' Includes a workflow suggestion (analyze_codebase → project_statistics → drill down) indicating when to use it relative to other tools. Does not name alternatives explicitly but gives clear contextual guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_codebaseA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the operation is expensive (10-60s), that results are cached, and that progress is shown. It explains that it builds the code graph that powers subsequent operations. However, it does not explicitly clarify whether the tool modifies the filesystem or is purely read-only, though this is implied by 'analysis.' This is adequate but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with labeled sections (PURPOSE, USAGE, PERFORMANCE, WORKFLOW, TIP) and each section adds value. While it includes decorative emojis, they do not dilute the content. The length is appropriate for a tool of this complexity, though it could be slightly tighter without losing clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides a complete picture for invocation: it explains what the tool does, when to run it, performance characteristics, caching behavior, and the workflow to follow. It even lists the types of outputs (metrics, structure overview). With no output schema, this describes return content sufficiently. It does not mention potential failure cases or permissions, but for a read-only analysis tool, this level of context is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully documents the single parameter (rebuild_graph with description), achieving 100% coverage. The tool description adds extra context by providing a condition for use: 'Use rebuild_graph=true only if code has changed significantly since last analysis.' This goes beyond the schema baseline, adding meaningful guidance for parameter selection.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Perform comprehensive codebase analysis with metrics and structure overview.' It specifies concrete outputs (file counts, language distribution, complexity metrics, architectural overview) and differentiates itself from sibling analysis tools by explicitly positioning itself as the foundational first step that builds the code graph.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: 'Run this FIRST before using other analysis tools' and shows the workflow (analyze_codebase → specific analysis tools). It also advises when to use the rebuild_graph parameter ('only if code has changed significantly'), which provides clear context for when to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• dependency_analysisA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It discloses performance (3-10s), that it builds a dependency graph, and offers a tip about what to look for. It doesn't detail return format or side effects, but for a read-only analysis tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with labeled sections (PURPOSE, USAGE, PERFORMANCE, WORKFLOW, TIP). Each section adds unique value without redundancy, making it both compact and highly informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema and no annotations, the description is remarkably complete. It covers purpose, usage scenarios, performance expectations, workflow integration, and even analysis tips, fully enabling an agent to decide when and how to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty schema, so the description doesn't need to explain parameter details. It confirms that the tool analyzes the entire codebase implicitly, which is appropriate given the lack of inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: analyzing module dependencies, mapping interdependencies, identifying circular dependencies, and revealing architectural patterns. It distinguishes itself from sibling tools like find_references or find_callers by focusing on the overall dependency graph rather than specific references.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit use cases (architectural analysis, identifying tightly coupled modules, planning restructuring) and places the tool in a workflow (analyze_codebase → dependency_analysis → complexity_analysis). However, it does not explicitly mention alternatives or when-not-to-use conditions, so it's not a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_usage_guideA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It adds value by noting the tool is lightweight and provides strategic guidance without analysis overhead, clarifying it doesn't perform heavy analysis. However, it doesn't explicitly state the absence of side effects or mention authentication needs, which is less critical for a read-only guidance tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description uses a structured format with short labeled sections (PURPOSE, USAGE, PERFORMANCE, WORKFLOW) that convey key information efficiently. While slightly longer than the simplest examples, every section adds relevant context and the content is front-loaded with the main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description comprehensively covers purpose, usage timing, performance, and workflow. It also positions itself within the sibling tool set as the entry point for guidance, making it sufficiently complete for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, making schema description coverage trivially 100%. The description avoids parameter details appropriately, and a baseline of 4 is warranted for zero-parameter tools. No additional parameter meaning is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as providing comprehensive guidance on using code analysis tools, focusing on instructions, best practices, and workflow recommendations. This distinct purpose differentiates it from sibling analysis tools that perform actual code analysis rather than offering meta-guidance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states to call this FIRST when needing guidance on tool selection, sequencing, or best practices, and recommends using it before complex analysis tasks. This provides clear when-to-use guidance and implicitly contrasts with the analysis siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• complexity_analysisA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It includes performance characteristics ('Moderate operation (5-15s)'), scope ('across the entire codebase'), and output nature ('priority rankings'). It doesn't explicitly state it is read-only, but as an analysis tool this is implied, and the added performance/scope context goes beyond a basic statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with short labeled sections (PURPOSE, USAGE, PERFORMANCE, WORKFLOW, TIP) and a front-loaded summary sentence. Every sentence contributes actionable information, with no wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having only one parameter and no output schema, the description covers all necessary aspects: purpose, usage context, performance, workflow integration with siblings, and parameter tips. It is complete enough for an agent to select and invoke the tool correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already includes full descriptions of the single 'threshold' parameter with ranges. The description's TIP adds strategic guidance on how to choose thresholds (e.g., 'Start with threshold=15 for critical issues, lower to 10 for comprehensive analysis'), which provides practical value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: calculates cyclomatic complexity, identifies code smells, and suggests refactoring opportunities with priority rankings. The PURPOSE section is specific and distinguishes it from siblings like analyze_codebase by emphasizing deeper complexity analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The USAGE section explicitly says 'Use after basic analysis' and provides a workflow (analyze_codebase → complexity_analysis → examine high-complexity functions → find_callers/find_callees), giving clear positioning relative to sibling tools. It also states it is 'Essential for code quality assessment,' making the when-to-use unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.