Server Quality Checklist

Tool Scores

• analyze_directoryA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must cover behavior. It only states purpose and parameters, not side effects, performance implications, or whether it reads file contents. Minimal behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences plus a relevant example. No fluff, front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, and description does not specify return format (e.g., list of files with summaries). The phrase 'understand what each file/module does' is vague. Missing critical detail for agent to interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters (100% coverage). Description adds an example with typical values, providing practical context beyond the schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool maps repository structure and understands files/modules. It distinguishes from siblings like deep_research or quick_query by focusing on project organization.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Preferred when questions ask about project organization or what's in this directory' and provides an example. No explicit alternative guidance, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_chunkA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It implies a read-only operation ('retrieve') and provides an example, but does not explicitly state that it is non-destructive, any side effects, rate limits, or error conditions. The example helps but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences and an example, front-loaded with purpose, followed by usage guidance. Every element is relevant and concise without filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with two params and no output schema, the description explains what it does and when to use it, but does not describe the response format or error handling. It is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds an example showing how cacheKey and chunkIndex relate, but the schema descriptions already cover their meaning. The example adds marginal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('retrieve'), the resource ('continuation of a large response'), and the context (when 'chunks' metadata is present). It distinguishes from siblings like analyze_directory and deep_research by focusing on paginated retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use the tool: 'Use when a previous tool response included 'chunks' metadata indicating more content available.' It does not mention when not to use it or alternatives, but the guidance is clear and context-specific.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_pathsA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It states 'Verify file paths exist and are accessible' but does not describe what the tool returns (e.g., boolean, list of invalid paths), error handling, or whether it is read-only. This leaves significant behavioral ambiguity for an AI agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences plus a relevant example. It front-loads the core purpose and usage, with no unnecessary words. Every sentence contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema) and lack of annotations, the description covers purpose and usage adequately but omits critical behavioral details like return value or side effects. It is minimally viable but incomplete for an agent to confidently invoke.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (parameter 'paths' described as 'Array of paths to validate'). The description adds value by clarifying that validation checks existence and accessibility, and provides an example input. This goes beyond the schema's minimal description, meriting a score above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose: 'Verify file paths exist and are accessible before analysis.' It uses a specific verb ('Verify') and resource ('file paths'), and distinguishes itself from siblings like 'analyze_directory' by focusing on validation rather than analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear when-to-use guidance: 'Use when uncertain about path correctness or troubleshooting PATH_NOT_ALLOWED errors.' It lacks explicit when-not-to-use or alternative comparisons, but the stated scenarios are sufficiently specific for an agent to decide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• quick_queryA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description lacks detailed behavioral traits such as rate limits, side effects, or limitations of the large context window.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences plus an example, no redundant information, and the key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequately covers purpose, usage guidance, and parameter semantics given the absence of an output schema; could include more behavioral details but sufficient for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all parameters; the description adds an example showing parameter usage, enhancing semantic understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool analyzes code/files quickly using Gemini's large context window, and distinguishes itself by specifying it's preferred for specific file or repository code questions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to prefer this tool (questions mentioning specific files or requiring repository code), providing useful context for selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deep_researchA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It explains the tool performs deep reasoning across multiple files but does not mention side effects, auth needs, or performance implications. The example gives some insight into usage but lacks comprehensive behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of two sentences and an example. The purpose is front-loaded, and every element serves a clear function with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and the fact that there is no output schema, the description covers the main aspects: what it does, when to use it, and how to use it via the example. It could be improved by mentioning any limitations or typical use cases, but it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all parameters. The description adds value by providing a concrete example that demonstrates how to use the parameters, especially the prompt and focus/citationMode options, which goes beyond the schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs comprehensive codebase analysis with deep reasoning, and it distinguishes itself from siblings by specifying it is preferred for complex architectural questions or multi-file investigation. The example further solidifies its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('complex architectural questions or multi-file investigation'), providing clear context. However, it does not mention when not to use it or suggest alternatives, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• health_checkA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description carries burden. It notes diagnostics inclusion but lacks info on error behavior or expected output format. Adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two clear sentences plus an example, no fluff. Front-loaded with the main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-param tool with no output schema, the description covers the main use case. Could mention return format but not necessary for basic functionality.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%. Description adds an example JSON with includeDiagnostics: true, which provides practical guidance beyond the schema's field description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool verifies server status and Gemini CLI configuration, using strong verbs 'Verify' and 'troubleshooting'. It distinguishes from siblings like analyze_directory and deep_research which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use for troubleshooting connection issues or confirming setup. While no when-not is given, the context is clear and the tool name reinforces usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.