Server Quality Checklist

Tool Scores

• get_semantic_contextB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. While it states the core function (return content and dependency tree), it does not mention potential performance impacts, required permissions, file system access, or any side effects. For a tool that may perform heavy analysis (dependency traversal), this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with a front-loaded verb ('Returns'). Every word is meaningful and there is no redundancy. It is efficiently structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description should fully explain what is returned. 'Rich context' is vague; it does not specify the format of the dependency tree or whether it includes full file content. For a tool that returns complex data, this leaves significant ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description adds minimal value beyond the schema. It does not explain the practical implications of the depth parameter or how outline_only reduces tokens. The baseline of 3 is appropriate as the description does not enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns file content and its local dependency tree, which is a specific verb+resource pairing. This distinguishes it from sibling tools like get_changed_files (returns list of changed files) and get_project_tree (returns directory structure). The purpose is unambiguous and well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for code understanding but does not explicitly state when to use this tool versus alternatives. There is no mention of when not to use it or which sibling tool to choose instead. The usage context is implied but not directly guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codebaseA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states basic functionality (searching and returning matches). It does not disclose whether the tool is read-only, performance implications, or any side effects—though as a search tool, this may be less critical. It provides the core behavior but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the action and output. Every word is informative and there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema and the moderate complexity (5 parameters, no enums), the description is minimally adequate. It covers the main purpose but does not explain the structure of results or any advanced behavior like result ordering or error handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add any meaning beyond the schema; it summarizes the tool's purpose but does not elaborate on parameter semantics or usage nuances.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as searching the codebase for text or regex patterns, returning specific information (file paths and line numbers). It distinguishes itself from sibling tools like 'get_changed_files' and 'get_project_tree' which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives, nor does it mention when not to use it. The agent receives no context about prioritization or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_project_treeA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description mentions that the tree view respects .gitignore rules, which is a behavioral detail. However, with no annotations provided, the description carries the full burden; it does not disclose result format, recursion behavior (though max_depth parameter suggests depth control), performance implications, or whether it includes file contents. Some transparency but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 12 words, no redundant information. It front-loads the core purpose (returns a tree view) and appends the key rule (respects .gitignore). Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters, no output schema, and simple functionality, the description is somewhat complete but lacks explanation of the return format (e.g., JSON, list) or behavior with large directories. It covers the main constraint (.gitignore) but not other aspects like depth or hidden files (though parameters handle those). More detail would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the input schema already describes each parameter (path, max_depth, show_hidden) with adequate descriptions. The tool description adds no additional meaning beyond what the schema provides, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns a structured tree view of the project repository while respecting .gitignore rules. This differentiates it from sibling tools like get_changed_files (diffs), get_project_index (index), get_semantic_context (semantic), and search_codebase (search). The verb 'returns' and resource 'project repository' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for browsing the project's directory structure, but it does not explicitly state when to use it versus alternatives or provide any exclusions or prerequisites. No guidance on when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_changed_filesA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description carries full burden. It adds helpful context like default ref, optional diff with size limit (50 KB), and grouping by status, but does not address performance, git environment requirements, or edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences conveying purpose, behavior, and usage. No redundant or filler words. Very efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without output schema, description could specify the exact return structure (e.g., list of file paths with status). Also missing prerequisites like needing a git repo. Adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all 3 parameters (100% coverage), baseline 3. Description adds value by stating the default for base_ref and the size limit for include_diff, which are not in schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it returns a list of files changed since a git ref, grouped by status, with optional diff. This distinguishes it from sibling tools focused on project structure and search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description includes 'Use this to understand what changed in a branch or commit,' providing clear context. However, it does not explicitly state when not to use it or mention alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_project_indexA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but the description discloses the tool returns only exported symbols with signatures, no implementation bodies, and is compressed (~500 tokens). Lacks mention of any side effects or permissions, but adequate for a read-only extraction tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary purpose, no extraneous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description fully explains the return value and provides usage context and efficiency estimate, making it complete for this tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  100% schema description coverage means the schema already documents both parameters; the description adds no additional parameter-level detail, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a compressed semantic map of the entire project including exported signatures without implementations, and distinguishes from siblings by focusing on overview efficiency and token size.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends calling this first when exploring an unfamiliar project and notes it's ideal for a single call overview, but does not mention when not to use it or contrast with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.