Server Quality Checklist

Tool Scores

• search_symbolsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description lists return fields and gives an example, but lacks additional behavioral context such as performance or case-sensitivity (though case-sensitivity is in the schema). With no annotations, the burden is on the description, which is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences plus an example, all front-loaded with purpose and usage, no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description adequately covers what the tool does and what it returns, but could mention that only exported symbols are searched.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter is well-documented in the schema (100% coverage), and the description reinforces its usage with an example, but does not add significant new meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches for exported symbol names across the repository, distinguishing it from sibling tools that deal with file skeletons, indexes, or stats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides guidance on when to use (to find where symbols are defined) and includes an example, but does not explicitly mention when not to use or compare to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_repo_indexA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses approximate cost (10-20 tokens per file) and behavior (returns file paths and symbols). No side effects or contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, front-loaded with purpose, then usage guidance, then cost. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one optional param, no output schema), the description fully covers purpose, usage, cost, and relationship to sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the parameter. Description does not add extra meaning beyond the schema's description of include_stats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'returns a compact table-of-contents' with specific verb and resource ('myorg/myrepo'), listing files and symbols. Distinguishes from sibling tools like get_file_skeleton and get_folder_skeleton.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to call this tool FIRST before others, and provides specific downstream usage (input to get_folder_skeleton or get_file_skeleton). Lacks explicit when-not-to-use but strong on when-to-use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_repo_statsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It correctly indicates a read-only operation returning stats, with no side effects. It does not explicitly state read-only, but the behavior is straightforward.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundancy. The first sentence states the action, the second gives usage context. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only aggregate tool with 0 parameters and no output schema, the description is complete. It specifies what is returned (total count and breakdown by language) and the target repository.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, and the schema coverage is 100%. According to guidelines, 0 parameters yields a baseline of 4. The description does not need to add parameter info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns aggregate statistics for a specific repository, including total file count and breakdown by language. It distinguishes from siblings like get_file_skeleton (file structure) and search_symbols (search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this to understand the tech stack and scale of the codebase at a glance.' This provides clear when-to-use guidance, though it does not explicitly mention when not to use or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_file_skeletonA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes what the tool returns (exports, imports, class members, doc comments) and implies it is a read operation. No side effects are mentioned, but the tool is inherently safe.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main purpose. Every sentence provides necessary information without fluff. Very efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (one parameter, no output schema, no nesting), the description is complete enough. It might lack detail on the exact format of the AST skeleton, but that is acceptable for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% coverage for the single 'file' parameter with a clear example. The description adds value by specifying to use the exact path from get_repo_index output, which helps the agent understand how to construct the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the AST skeleton for a specific source file, listing exports, imports, class members, and doc comments. It distinguishes from siblings by naming get_folder_skeleton and get_repo_index.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool (after narrowing down from get_repo_index for a single file) and when not to (for multiple related files, prefer get_folder_skeleton for one round trip).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_folder_skeletonA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses cost (~50 tokens per file), what skeletons include (exports, function signatures, types, doc comments), and that it's cheaper than raw source code.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, front-loaded with purpose, followed by usage, content description, and a cost tip. No unnecessary words; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, description adequately explains return content. Cost information and sibling differentiation make it complete for an effective selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Single parameter 'folder' is well-described in schema (100% coverage). Description adds example values and an important note ('Do NOT include a trailing slash'), enhancing understanding beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns compact AST skeletons for source files under a folder path. It explicitly distinguishes from siblings like get_file_skeleton (single file) and get_repo_index (discover paths).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: 'Use this to understand a module or feature area without reading raw source files.' Also includes a tip to call get_repo_index first, showing when to use this tool vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.