Server Quality Checklist

Tool Scores

• list_repo_dependenciesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It accurately declares 'Read-only graph traversal with no side effects' and explains empty results. It does not mention auth requirements or rate limits, but for a read-only tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (introduction, usage, behaviour, parameter guidance). It is concise at 4-5 sentences with no fluff. Each sentence adds value, making it easily digestible.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description clearly states the return format: 'Returns a JSON array of repository names.' It covers the tool's behavior, parameters, and usage context comprehensively for the complexity of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context beyond schema: repo_name must match indexing, max_depth defaults/limits, reverse toggle behavior. This extra guidance helps the agent use parameters correctly, justifying a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Read-only cross-repository dependency graph lookup' and explicitly answers the two key questions ('which repos does this repo depend on?' and 'which repos depend on this repo?'). It distinguishes itself from sibling tools like explore_file and find_callers by focusing on repository-level dependencies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use BEFORE cross-repo analysis' and 'Use reverse mode for impact analysis before breaking changes in shared libraries.' It does not explicitly state when not to use the tool, but the context is sufficiently clear for an agent to decide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_callersA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so description carries full burden. It clearly states read-only with no side effects, describes return format (Markdown grouped by relationship type), and explains special behaviors for JVM code (overridden/overrides) and disambiguation for multiple entities. Highly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured with clear sections (usage, behaviour/return, parameter guidance, language support). Each sentence adds value. Could be slightly shorter, but clarity and comprehensiveness offset verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, so description must explain return values, which it does in detail: Markdown grouping by relationship type, exact file paths and line numbers, and special handling for JVM and multiple entities. Covers all essential aspects for a dependency lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so schema documents parameters. However, description adds value by explaining entity_name supports exact names or signature fragments and repo_name is optional but recommended for filtering. This extra guidance justifies a score above baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'read-only reverse dependency lookup' and explains it finds all code that references, calls, extends, or implements a specific entity. It distinguishes from search tools by providing exact dependency tracking. The verb 'find' and resource 'callers' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use (impact analysis before refactoring, detect dead code) and when not to use (not for semantic feature discovery, use 'search_hybrid_context' instead). Also provides critical guidance about including signature fragments for common method names to avoid irrelevant results.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_repositoriesA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Declares read-only, no side effects, describes return format (Markdown table), empty state behavior ('No repositories found.'), and mentions support for all languages and build systems.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-organized into sections with clear headings. Every sentence is informative, though somewhat verbose. Could be slightly tightened but remains effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given only one optional parameter, no output schema, and no annotations, the description fully covers purpose, usage, behavior, parameter details, and return format, leaving no gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by clarifying case-insensitive substring match and providing examples of filter usage, elevating it above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a read-only listing of indexed repositories with optional name filtering. It distinguishes itself from siblings by specifying it is for discovering codebases, not for searching code entities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to use this tool FIRST, provides when-not-to-use (e.g., 'Do NOT use this tool to search for code entities'), and lists specific sibling alternatives for different tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explore_fileA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description fully covers behavioral traits. It declares the operation as read-only with no side effects, describes the return format (Markdown outline with line numbers), and details path handling behavior including normalization and ambiguity resolution.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively long but well-structured with clear sections (Usage, Behaviour & Return, Path handling, Parameter guidance). While every sentence adds value, slight redundancy exists (e.g., repetition of file_path details). Overall efficient for the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains the return type (Markdown outline with entity groupings and line numbers). It covers all necessary aspects: purpose, usage context, behavioral traits, parameter details, edge cases (ambiguous paths), and supported languages. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the description adds significant value beyond the schema. For file_path, it explains preferred relative paths, absolute path acceptance, and normalization. For repo_name, it clarifies when to include it and the behavior when omitted, which is critical for correct usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it's a 'Read-only file anatomy inspection' tool for listing classes, methods, and properties within a specific source file. It distinguishes itself from siblings like 'search_hybrid_context' by explicitly noting it is not for searching across multiple files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use: 'Use AFTER identifying an interesting file via search_hybrid_context to understand its available methods, or before modifying a file.' Also states what not to use for: 'Do NOT use this for searching across multiple files.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_hybrid_contextA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description fully carries burden. It declares 'Read-only', 'No side effects', describes the dual query mechanism, and notes prerequisites (active server with databases). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (header, prerequisites, behavior, usage, parameters, supported languages). Every sentence adds value. Front-loaded with key purpose. No unnecessary repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, so description explains return format: Markdown with file paths, line numbers, code snippets, dependencies. Supports multiple languages. Addresses all aspects needed for a complex search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters (100% coverage). Description adds valuable guidance: query should be 2-5 words, max_results recommendations, repo_name inclusion to avoid cross-repo pollution.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs read-only semantic and structural code search combining vector embeddings and graph analysis. It specifies what it returns and distinguishes from siblings by explicitly mentioning it's for initial discovery.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'FIRST step when exploring unfamiliar code or discovering architectural patterns.' Also provides when not to use and names the alternative tool 'find_callers'. Includes prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.