Server Quality Checklist

Tool Scores

• woodpecker_diagnose_root_causeA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. While it explains what the tool returns and that it is deterministic, it fails to mention that the tool takes no input parameters (schema has zero properties). This omission may confuse an AI agent about how the tool is invoked or what context it requires. The description claims it 'localizes' root cause but does not state the implicit input or state it operates on.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, informative paragraph that front-loads the main purpose. It is concise yet covers key outcomes. Minor improvement could be breaking into bullet points, but overall it is well-structured and not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description does explain return values partially. However, it lacks critical context: how the tool determines which services to analyze without parameters, and how it fits with sibling tools. The completeness suffers because the input mechanism is undefined, making it unclear for an AI agent to know when to call this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the schema provides 100% coverage by default. According to the rules, a baseline of 4 is appropriate when there are no parameters. The description does not need to add parameter semantics, but it lacks an explanation of how the tool operates without explicit input.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to deterministically localize the root cause by identifying the deepest failing service. It specifies what it returns (root cause, causal chain, blast radius, etc.) and distinguishes its deterministic nature from typical inference. This makes the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context of use is clear: it is for diagnosing root cause when services are failing. However, it does not explicitly state when to use this tool versus its siblings (e.g., woodpecker_get_service_health for health checks) or when it would be inappropriate. The description implies its use for root cause but lacks explicit exclusionary guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• woodpecker_get_blast_radiusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It explains the core operation but does not disclose any side effects, limitations, authorization needs, or whether it is read-only. It is safe to assume read-only, but not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, first defines the tool's function, second explains the two directions. No redundant words, efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of transitive dependency closure, the description is mostly complete. However, it lacks details about the return format (e.g., list of service names). With no output schema, this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description compensates. It adds meaning for the 'direction' parameter (upstream vs downstream) beyond the schema's title and default. However, 'service' is not elaborated beyond its name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it computes transitive dependency closure over DEPENDS_ON edges, distinguishes upstream (blast radius) and downstream (deeper root cause) directions. It is specific and differentiates from sibling tools like woodpecker_diagnose_root_cause which is a different operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context on when to use each direction but does not explicitly tell when to use this tool versus alternatives like woodpecker_get_topology or woodpecker_diagnose_root_cause. The implicit guidance is through the blast radius concept.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• woodpecker_get_service_healthA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the output fields (status, container state, restarts, error rate, etc.), suggesting a read-only health check. It does not mention side effects or hidden behaviors, but the information given is sufficient for basic transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently lists key output fields. It is front-loaded with the purpose and avoids extraneous detail. However, it could be slightly more structured for readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (1 parameter, no output schema, no annotations), the description adequately covers the tool's function and output. It lists the fields returned, which compensates for the lack of output schema. The missing param semantics reduce completeness slightly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'service' is a string with 0% schema description coverage. The description does not explain what format or values 'service' expects (e.g., service name or ID), leaving the agent to guess. This is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides a 'detailed health snapshot for one service' and enumerates specific fields (status, container state/health, restarts, etc.), which distinguishes it from sibling tools like 'woodpecker_detect_blind_spots' or 'woodpecker_diagnose_root_cause'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for obtaining health details of a single service, but it does not provide explicit when-to-use or when-not-to-use guidance relative to siblings. The context is clear enough for a simple tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• woodpecker_detect_blind_spotsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral burden. It explains what the tool does (lists blind spots) and adds context that these are not outages, avoiding misinterpretation. It could be improved by noting if it only lists current state or historical data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the action 'List observability blind spots' and then provides clarifying context. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description is largely complete. It defines what blind spots are and the condition (no live scrape target). However, it does not specify output format or how results are ordered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100%. Per calibration, zero-parameter tools get a baseline of 4. The description does not need to add parameter info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool lists observability blind spots, defined as healthy services without Prometheus scrape targets. It uses a specific verb-resource combination and distinguishes from sibling tools like diagnose_root_cause or get_service_health.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by clarifying that blind spots are not outages and should not trigger pages, but does not explicitly say when to use this tool versus alternatives. No direct mention of sibling tools or when-not-to-use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• woodpecker_get_topologyA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the output content (graph of services, statuses, dependencies), the status enum, and the meaning of monitoring='MISSING'. This provides good behavioral insight without hiding important details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first clearly states the function and output, the second adds usage guidance and additional detail on status values. No extraneous words; every sentence earns its place. Well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has zero parameters, no annotations, and no output schema, the description is thorough. It explains what is returned (graph with status and dependencies), key fields (status enum, monitoring flag), and when to use it. This is complete for an agent to understand and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the schema description coverage is 100% by default. Per guidelines, baseline for 0 params is 4. The description does not need to add parameter semantics, and it does not mention any parameters (correctly).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool returns the materialized service dependency graph with each service's status and dependencies. It lists status enum values and mentions the monitoring='MISSING' flag, providing specific detail beyond a generic statement. This clearly distinguishes it from siblings by positioning it as the first call for causal structure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises 'Call first to establish the causal structure before diagnosing,' which gives clear when-to-use guidance relative to other diagnostic tools. It does not explicitly mention when not to use or name alternatives, but the context with sibling tools implies the flow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.