Server Quality Checklist

Tool Scores

• searchA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint. The description adds useful behavioral context beyond the annotations: Gemini may issue multiple searches, synthesizes a 500-1500 word summary, includes source URLs, and output is not raw search results. This meaningfully enriches the agent's understanding.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear opening, a 'Use for' list, and behavioral notes. It is appropriately sized for the tool's complexity, though phrases like 'Google Search grounded research' and 'synthesizes a comprehensive answer' are slightly redundant, preventing a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Since there is no output schema, the description effectively covers the return behavior: a synthesized summary of 500-1500 words with source URLs and citations. It also explains the multi-search behavior and the role of maxResponseLength, fully equipping an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes all 5 parameters with 100% coverage, including the meaning of 'query' and 'maxResponseLength.' The description only adds that maxResponseLength adjusts the summary length, which is already in the schema. Thus it adds little beyond the structured data, warranting the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Gemini searches the web using google_web_search and synthesizes a comprehensive answer.' It also provides concrete use cases (current events, API references, verifying facts) that distinguish it from siblings like 'query' and 'structured.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an explicit 'Use for' list that tells the agent when this tool is appropriate, such as current events, documentation lookups, and verifying facts. However, it does not explicitly state when not to use it or mention alternative sibling tools, so it falls short of a perfect 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch-chunkA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, idempotent, non-destructive. The description adds valuable context: the 10-minute in-memory cache expiry, which is a critical behavioral constraint. It also clarifies that large responses may return only the first chunk plus a cacheKey, explaining the tool's place in the interaction flow. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences that front-load the core purpose, then provide context and usage. Every word earns its place; no fluff or redundancy. Perfectly sized for the task.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description fully covers what an agent needs: what it does, when to use it (with a cacheKey), how to use it (1-based chunkIndex), and the expiry constraint. No output schema exists, but the return value is implied to be the next chunk, which is acceptable for this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds minimal extra meaning: it explains that cacheKey comes from the initial chunked response and that chunkIndex is 1-based with 2 as the next segment, which slightly reinforces the schema. However, it does not substantially go beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Retrieve') and resource ('a cached chunk from a previously chunked response'). It distinguishes itself from sibling tools (ping, query, search, structured) by being the follow-up mechanism for fetching remaining chunks, which is explicit and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage context: use when a large query/search response returns a cacheKey and you need subsequent segments before the 10-minute expiry. It could be stronger by explicitly stating not to use without a cacheKey, but the scenario is clear and self-contained.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pingA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the tool as read-only, idempotent, and non-destructive. The description adds valuable behavioral context: it is fast (~1s) and makes no model call. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise, starting with 'Health check.' and providing key details in two sentences. Every sentence adds value without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description is complete: it states the purpose, what it verifies, what it reports (CLI version, auth status, models, server version), and performance characteristics. Nothing missing for an agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty. The description does not need to explain parameters; the baseline for zero-parameter tools is 4, and no additional information is necessary.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a health check that verifies CLI installation and authentication, and reports version/status details. This distinguishes it from sibling tools like query and search, which are data-retrieval operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for health verification (e.g., before other operations) but does not explicitly state when to use it versus alternatives, nor does it mention exclusions or prerequisites. Context is clear but usage guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• structuredA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint true, destructiveHint false), the description discloses the agentic design: Gemini runs inside workingDirectory with read_file and grep tools, and the response is validated against the schema with isError set on failure. This adds significant behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences, front-loaded with the core purpose. Each sentence contributes useful information without redundancy—purpose, agentic context, and validation behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage, validation, and behavioral context, making it mostly self-contained. It does not describe return value structure, but with no output schema and strong annotations, this is an acceptable gap for a structured-output tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds context about agentic file reading and validation but does not elaborate on parameter formats beyond what the schema already states, so it adds only marginal semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'generate a JSON response conforming to a provided JSON Schema.' It clearly distinguishes from siblings by targeting structured output for data extraction/classification, leaving no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use for data extraction, classification, or any task needing machine-parseable output,' providing actionable usage guidance. It does not name alternative tools, but the use cases are clear enough to guide an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queryA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint=false, openWorldHint=true, etc.), the description richly discloses behaviors: fresh CLI spawn per invocation (~15–20s startup), file-hint semantics, read-beyond-hints, gitignore restrictions, pre/post-spawn git snapshot safety, structured edit parsing never chunked, and _meta.appliedWrites detection. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Although lengthy, the description is logically organized with sections for capabilities, file handling, change mode, model tips, and startup overhead. Each sentence contributes essential information, and key facts are front-loaded. No filler or redundant prose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 7 parameters and no output schema, the description covers operational behavior, constraints, prerequisites, and response metadata (e.g., _meta.edits, _meta.appliedWrites). It provides enough detail for an agent to use the tool correctly without requiring additional external knowledge.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds meaningful context: files are hints not inlined, images trigger --yolo mode, model guidance (flash vs pro), timeout floor due to CLI startup, and changeMode's structured edit safety. This goes beyond the baseline schema descriptions and helps the agent set parameters effectively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies this as an agentic query tool where Gemini runs with read_file, grep, list_directory, and glob tools in the workingDirectory. It lists specific capabilities (code analysis, image understanding, general knowledge, text transformation, change mode) that distinguish it from simpler or sibling tools, even without naming them explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit use cases and prerequisites: requires a git working directory, text-only in change mode, gitignored files unreadable in text-query mode, and model selection tips (flash for speed, pro for depth). While it doesn't explicitly compare to sibling tools like search or structured, it implies when this agentic tool is appropriate and notes limitations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.