Server Quality Checklist

Tool Scores

• metricsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, consistent with description. Description adds detail on returned metrics (success rates, latency, top errors, breakdown) but does not disclose rate limits, data freshness, or other behavioral traits. Acceptable given annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise single sentence with front-loaded verb and resource. No wasted words, efficiently conveys purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequately covers purpose and output for a simple read-only tool. Lists specific metric categories and breakdown dimensions. No output schema requires explanation of return structure, which is sufficiently hinted. Slight gap: no mention of whether data is historical or real-time.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema fully describes the sole parameter (windowMinutes) with type, default, and description. Tool description adds no additional parameter semantics. Baseline 3 at 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'Get' and resource 'observability metrics', listing concrete metric types (success rates, latency, top errors, breakdown by tool and target). Clearly distinguishes from siblings like recent_errors (specific errors) and circuit_status (circuit breaker state).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus siblings. Description only states what it does, leaving the agent to infer when to prefer it over recent_errors or circuit_status.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• circuit_statusA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, indicating no side effects. The description adds value by explicitly listing returned data (circuit state, failure count, cooldown), providing behavioral context beyond the annotation. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, front-loaded sentence with no wasted words. Every element (action, inputs, outputs) is present and immediately scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only check with one optional parameter, the description is adequate. However, it omits usage guidelines and differentiation from sibling tools, which would be needed for full completeness in a multi-tool context. Output schema not required.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description for the single parameter 'target'. The tool description does not add additional semantic meaning beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb-resource pair: 'Check the health status of a target.' Specifies return fields (circuit state, failure count, cooldown), distinguishing it from sibling tools like circuit_reset (write) and metrics (broader health).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like circuit_reset or metrics. The parameter hint 'Omit to see all targets' is the only usage direction, but no exclusions or context for choosing among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wrap_callA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries full burden. It mentions retry and circuit breaker but lacks details on number of retries, backoff strategy, circuit breaker thresholds, or return behavior. This leaves gaps for an agent to predict exact behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short, front-loaded sentences convey the tool's purpose, behavior, and usage without waste. Each sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 parameters, no annotations, and no output schema, the description is insufficient. It fails to explain return values, retry count, circuit breaker details, or what happens on success/failure. More detail is needed for full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description only adds general context ('pass a target name') without explaining parameter meaning beyond the schema. It doesn't compensate for the lack of detail in the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's core function: wrapping a call with retry, circuit breaker, and metrics. It distinguishes from siblings like circuit_reset and metrics by showing this tool actually executes the unreliable call.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to 'use this to wrap any unreliable external call,' giving clear context. However, it does not specify when not to use it or compare with alternatives, which would provide stronger guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recent_errorsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this as readOnlyHint=true. The description adds that it returns 'full error details and retry logs', which is valuable context beyond the annotation but does not elaborate on behavior like pagination or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, concise sentence with no filler. Every word contributes to the understanding of the tool's purpose and output.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with one optional parameter and no output schema, the description adequately conveys what the tool returns (error details and retry logs). However, it could hint at the structure of the output or mention that errors are from a specific timeframe.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter 'limit'. The description adds explicit mention of the default value (20), which is not in the schema, providing extra meaning beyond parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' and clearly identifies the resource as 'most recent failed calls' with 'full error details and retry logs'. It distinguishes itself from siblings like 'metrics' or 'circuit_status' by focusing on error specifics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving recent failed calls, but provides no explicit guidance on when to use this tool versus alternatives like 'metrics' (error counts) or 'circuit_status' (breaker state). No when-not-to-use or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• circuit_resetA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=false and idempotentHint=true. Description adds 'back to closed (healthy)' and usage timing, but no further behavioral details beyond what annotations imply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no extraneous words. Purpose and usage stated upfront and efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple reset tool with one optional parameter and no output, the description is complete: it explains the action, the corrected state, and proper usage timing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline 3. Description does not add extra semantic meaning for the parameter beyond the schema's own description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Reset' and resource 'circuit breaker back to closed (healthy)', which is distinct from siblings like circuit_status (read) or metrics (read metrics).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use after fixing the underlying issue', providing clear context for when to use. Does not explicitly state when not to use, but the guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.