Server Quality Checklist

Tool Scores

• enforce_deniesD
  1.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden of behavioral disclosure. It only states what is counted without explaining whether this is a read-only operation, how the 'window' is defined, or what 'Enforced' means in contrast to preview.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short phrase, but it is under-specified rather than concise. It omits critical context needed to invoke the tool correctly, so the brevity is not a strength.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having only one parameter and an output schema, the description is too sparse to provide a complete picture of the tool's behavior. It does not clarify key terms like 'window' or 'Enforced', making it inadequate for confident tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter (since_hours) with 0% description coverage, and the description does not mention it at all. The meaning and effect of the parameter are completely unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description mentions the resource (Cloud Armor rule priority) and metric (enforced DENY counts), but lacks a clear verb and the phrase 'over the window' is ambiguous. It distinguishes from sibling 'preview_denies' via the word 'Enforced', but not explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like preview_denies or health_check. There is no mention of appropriate use cases, exclusions, or conditions for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• daily_briefB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. The words 'summary', 'check', and 'preview' strongly imply a read-only, non-destructive operation, which is useful behavioral context. However, it does not explicitly state side effects, authentication requirements, or return format. The description adds context beyond the name but lacks a clear safety declaration.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is concise, front-loaded, and free of fluff. It efficiently lists the three components of the brief, making every word earn its place. This is an excellent example of brevity without sacrificing meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has an output schema and only one optional parameter, the description covers the core purpose well. The main gap is the undocumented 'since_hours' parameter, but the schema's default and the simple nature of the tool partially mitigate this. The description is complete enough for a low-complexity tool with an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes one parameter, 'since_hours', with a default of 26, but the description does not mention it at all. Schema description coverage is 0%, so the description must compensate, but it fails to explain what 'since_hours' represents or how it affects the brief. The parameter name suggests a time window, but this is not linked to the 'morning patrol' context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides a 'Morning patrol summary' covering three specific components: enforced DENYs by rule, home-region false-positive check, and preview DENYs. This distinguishes it from sibling tools like enforce_denies, preview_denies, and home_region_denies, which are individual operations. The verb is implied ('summary') rather than explicit, but the scope and content are unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Morning patrol summary' implies use for a routine daily check, and 'in one call' suggests it consolidates multiple separate operations. However, there is no explicit guidance on when to use this tool versus the sibling alternatives, nor when not to use it. The context is clear but not explicit about exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• home_region_deniesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the transparency burden. It discloses a prerequisite (CLOUDARMOR_HOME_REGION) and implies a read-only review operation, but it does not explicitly state that no changes are made or describe the output shape beyond the output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the primary action, and the second sentence provides valuable context. There is no filler or repetition, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, has an output schema, and the description covers purpose and usage context well. However, it omits any explanation of the since_hours parameter and the concept of 'known-normal', which are relevant to effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter (since_hours) with no description, and the description never mentions it. Since schema coverage is 0%, the description should compensate, but it doesn't, leaving the parameter's role and default behavior undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states exactly what the tool does: it lists enforced DENYs whose source IP geolocates to the home region, framing it as a false-positive lens. This is a specific verb-resource combination that clearly distinguishes it from siblings like enforce_denies or preview_denies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage context: use this when reviewing home-region users being denied on a priority not marked known-normal. It does not explicitly name alternatives or when-not-to-use, but the context is sufficient for most agents.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_deniesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. The term 'dry-run' clearly communicates non-destructive behavior, and the false-positive evaluation context adds behavioral insight. It could explicitly state no changes are made, but the meaning is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that front-load the core purpose. The second sentence adds context without fluff. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one optional param, output schema present), the description covers the purpose and use case well. The main gap is the lack of explicit mapping from 'window' to since_hours, and some domain terms like 'rule priority' are undefined, but overall it is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description does not explain the since_hours parameter or its relationship to the 'window'. The parameter name is self-explanatory, but the description adds no parameter-specific meaning, so it does not compensate for the lack of schema description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with specific verb 'Preview' and resource 'DENY counts per rule priority over the window'. It distinguishes itself from the sibling enforce_denies by explicitly labeling itself as a dry-run and mentioning promotion to enforce.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a clear use case: preview rules to evaluate false positives before promoting to enforce. However, it does not explicitly mention alternatives or state when not to use, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• health_checkA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description fully explains the behavior by defining the three possible statuses (healthy, degraded, error) and their conditions. It also discloses that the response has a fixed shape, which is valuable for callers.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two focused sentences, front-loaded with the tool's purpose, and every sentence adds behavioral value. It is concise without being under-specified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter health check with no output schema, the description fully covers purpose, behavior, and return shape. It gives a complete picture for an agent to invoke and interpret the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  This tool has zero parameters, so the schema already provides complete coverage. The description adds no parameter details, but none are needed; the baseline of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states this is a service health check that reports version, config presence, and a minimal API probe. It identifies the exact resource and action, and is distinctly different from sibling tools like enforce_denies or daily_brief.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context by mentioning a 'monitoring caller' and explains why the fixed shape is beneficial (no branching on missing keys). However, it does not explicitly state when not to use this tool or name alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.