Server Quality Checklist

Tool Scores

• degraded_sitesA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of disclosing behavioral traits. It does not mention side effects, read-only nature, authentication requirements, or potential errors. The phrase 'not reporting fully healthy' is also somewhat ambiguous regarding what 'reporting' means.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no unnecessary words. It front-loads the action and includes the key filtering condition without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple list operation with no parameters and an output schema indicated. It could mention output fields or ordering, but the existence of an output schema reduces the need for the description to explain return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. No parameter descriptions are needed since there are no parameters to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific verb ('List') and resource ('sites') with a clear filtering condition ('at least one layer not reporting fully healthy'). It distinguishes itself from sibling tools like list_sites, which presumably lists all sites, by focusing on degraded ones.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing sites with unhealthy layers, but it does not explicitly state when to use this over alternatives like site_health or root_cause. Guidance is present only by inference from the wording.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• site_healthA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral burden. It discloses that the operation is a read ('Get'), enumerates the status layers, and adds the exact-match input constraint. However, it doesn't describe error behavior for mismatched site names, the output/return format, or any side effects — though for a low-risk read operation the risk profile is modest. Adds useful context but has gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero waste. The primary purpose is front-loaded in the first sentence, and the operational constraint follows in the second. Every word earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with no output schema and no annotations, the description covers the essentials: what the tool does and what the parameter requires. It could add the return format of the health status values, but the core calling information is complete and an agent can invoke this tool correctly with what's provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, and it does: the second sentence clarifies that the 'site' parameter is a name that must exactly match one from list_sites. This adds meaningful semantic information beyond the bare schema, which only types it as a required string. The description fully compensates for the coverage gap for the sole parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('layered health status') and enumerates exactly which layers are covered (circuit, power, core, access). This distinguishes it from siblings like list_sites (which lists sites) and degraded_sites (which finds problem sites), though it doesn't name them explicitly. Clear purpose with minor room for explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The second sentence gives a concrete, actionable constraint: 'site must exactly match a name returned by list_sites.' This tells the agent to call list_sites first and use an exact name value. It implies usage context well but stops short of stating when not to use this tool or naming alternatives as the preferred choice for other scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sitesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. The verb 'list' clearly indicates a read-only operation, and there is no mention of side effects. While not explicitly stating 'read-only', the semantics are unambiguous enough for a safe interaction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no redundant words. It delivers the essential information efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of parameters and the simple listing action, the description is sufficient for an agent to invoke the tool correctly. No additional context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so schema coverage is 100%. The description adds no parameter-specific information because there is nothing to describe. This meets the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list) and the resource (sites) with specificity (every, known to this diagnostics server). It distinguishes itself from sibling tools like site_health or degraded_sites by being the comprehensive listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (as the general listing of all sites) but does not explicitly name alternatives or conditions for choosing this tool over the siblings. An agent can infer usage from the wording, but it is not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• root_causeA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the walk order, the logic (first unhealthy layer), and the reporting behavior. It does not mention side effects (likely read-only) or error cases, but for a trace tool this is a reasonable level of transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each serving a purpose: purpose, algorithm/reasoning, and input constraint. Front-loaded with the primary action, no unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, algorithm, and input constraint, but does not specify the exact return format (e.g., whether it returns the layer name, a report object) or the behavior when all layers are healthy. Given there is no output schema, this is a minor gap for an agent invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so by adding a crucial constraint: 'site must exactly match a name returned by list_sites.' This gives meaning to the single parameter beyond the bare schema, making the input unambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action: 'Trace the most likely root-cause layer for a site's issues.' Describes the algorithm (walks circuit, power, core, access in dependency order) and the output (first unhealthy layer). This clearly differentiates it from siblings like site_health or degraded_sites, which focus on status or listing rather than root-cause analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context for when it is appropriate ('an upstream issue typically explains downstream symptoms') and a prerequisite ('site must exactly match a name returned by list_sites'). However, it does not explicitly mention alternative tools or when NOT to use this one, leaving some ambiguity for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redundancy_gapsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of explaining behavior. It makes clear that the tool lists/flags redundancy gaps without indicating side effects or mutations, and it explains the rationale behind including healthy layers.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, with the primary purpose front-loaded and the second sentence adding a valuable clarifying nuance. No unnecessary words or details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity, no parameters, and existing output schema, the description is complete enough for an agent to understand what the tool does and why it matters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so the baseline is 4. There is no parameter information needed beyond the empty schema, and the description focuses entirely on the tool's purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and the specific resource ('every (site, layer) pair currently lacking redundancy'), which distinguishes it from sibling tools like list_sites and site_health.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool, especially the nuance that it flags non-redundant layers even when status is healthy. It does not explicitly mention alternatives or when not to use it, but the use case is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.