Server Quality Checklist

Tool Scores

• cloudsealed_analyze_billing_wasteA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, but the description adds valuable behavior: it explicitly states it does NOT call any cloud provider API, explains the rolling-median and z-score algorithm, describes the error response format, and warns about the masking effect. This goes well beyond annotations and provides deep insight into the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Though longer than average, the description is organized into clear sections (Args, Returns, Error, Examples) with every sentence earning its place. It is front-loaded with the purpose and maintains focus without fluff, making it highly scannable despite its length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is exceptionally complete for a complex tool: it includes the full output JSON schema, error handling, provider-specific input notes, and practical usage examples. It covers all aspects an agent needs to select and invoke the tool correctly, leaving no gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description enumerates all parameters (csv_content, analysis_type, response_format) with their defaults and contextual meaning, including examples like analysis_type='cost-forecast'. This fully compensates for the 0% schema coverage on the top-level 'params' object and adds value beyond the nested schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb and resource: 'Detect cost anomalies in a cloud billing export (AWS/GCP/Azure/generic).' It clearly distinguishes from the sibling tool by explicitly naming cloudsealed_score_architecture_risk as an alternative for architecture/reliability risk, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use examples with concrete questions and how to set csv_content and analysis_type. Clearly states a prerequisite ('caller must already have exported the billing data') and gives a direct 'Don't use when' with the alternative tool, leaving no ambiguity about appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cloudsealed_score_architecture_riskA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint, idempotentHint, destructiveHint), the description discloses that scores come from explicit weighted rules, not a trained model, and provides an auditable rule-by-rule breakdown. It also reveals the underlying HTTP service dependency, configurability via env var, and timeout/error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Though lengthy, every sentence serves a purpose. The structure is front-loaded with the core purpose, then follows with parameters, return schema, error handling, and usage examples. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (nested input schemas, multiple risk dimensions, output format options, error cases), this description is complete. It includes the return schema, error response format, timeout behavior, and concrete usage scenarios. The annotations already cover safety, so no gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must carry full parameter meaning. It thoroughly explains each field within the params object, including types, constraints, and semantics (e.g., historical_metrics improves the scalability-gap score). It adds value beyond the raw schema by explaining input effects and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Score architecture risk from a declared system inventory.' It clearly distinguishes from the sibling tool by explicitly saying not to use it for cost/billing analysis and directing to cloudsealed_analyze_billing_waste.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit 'Use when' examples with concrete queries and an explicit 'Don't use when' alternative. This gives the agent clear decision criteria for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.