Server Quality Checklist

Tool Scores

• get_correlated_logsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the core behavior (fetching from multiple platforms, merging chronologically) but lacks details about authentication requirements, rate limits, error handling, or what the merged output looks like. For a complex multi-platform tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly front-loaded with the core purpose in the first sentence and usage guidance in the second. Every sentence earns its place with zero wasted words, making it highly efficient and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex 8-parameter tool with no annotations and no output schema, the description provides adequate purpose and usage context but lacks critical behavioral details about authentication, error handling, and output format. The high parameter count and multi-platform nature suggest more completeness would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description mentions 'time window and optional trace ID' which aligns with parameters but doesn't add meaningful semantic context beyond what the schema provides. The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('fetch logs from multiple cloud platforms', 'merge them into a single chronological timeline') and the resource ('logs from AWS CloudWatch + Vercel'). It distinguishes itself from siblings by focusing on cross-platform log correlation rather than resource checking, diagnosis, or topology listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: 'to correlate errors across frontend and backend services.' However, it doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools, which would be needed for a score of 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_resource_limitsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively communicates that this is a read-only query operation (implied by 'Query') and adds useful context about the warning threshold behavior. However, it doesn't mention authentication requirements, rate limits, error conditions, or what format the results will be returned in, which are important for a tool interacting with multiple cloud platforms.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states the core purpose, and the second provides usage guidance with concrete examples. Every element serves a clear purpose with zero wasted words, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no annotations, and no output schema, the description provides adequate purpose and usage context but lacks important behavioral details. It doesn't explain what the output looks like, how errors are handled, or authentication requirements. Given the complexity of querying multiple cloud platforms, more complete guidance would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 67% schema description coverage (2 of 3 parameters documented in schema), the description adds significant value by explaining the purpose of the 'warn_threshold' parameter and providing context about what platforms it works with. While it doesn't explicitly mention the 'platforms' or 'aws_region' parameters, it gives enough semantic context about the tool's scope to help understand parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Query quota limits and current usage'), identifies the target resources ('across configured cloud platforms'), and distinguishes this tool from siblings by focusing on proactive monitoring rather than diagnosis or logging. It provides concrete examples of what it monitors ('Lambda concurrency limits, Vercel plan caps').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('to proactively catch...issues before they cause outages'), providing clear context for its purpose. However, it doesn't mention when not to use it or explicitly differentiate it from sibling tools like 'diagnose_service_link' or 'list_cloud_topology', which might also involve cloud resources.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• diagnose_service_linkA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what the tool checks (three specific areas) and what it returns (diagnostic results with actionable messages), which is helpful. However, it doesn't disclose important behavioral traits like whether this is a read-only operation, whether it makes any changes to systems, authentication requirements, rate limits, or error handling. The description adds value but leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly sized and front-loaded. The first sentence establishes the core purpose, the second specifies the three diagnostic checks, and the third describes the return format. Every sentence earns its place with no wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, connectivity diagnostics across multiple cloud platforms) and no annotations or output schema, the description is adequate but incomplete. It explains what the tool does and returns at a high level, but doesn't cover important contextual details like authentication requirements, whether it performs active probes or passive checks, error conditions, or the format of diagnostic results. For a diagnostic tool with no structured output documentation, more completeness would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 83% schema description coverage, the schema already documents most parameters well. The description adds meaningful context by explaining the diagnostic scope (Vercel env vars, AWS Security Groups, API reachability) which helps understand what the parameters enable. It doesn't provide additional parameter-specific details beyond the schema, but the high schema coverage means less burden on the description. The description compensates adequately for the 17% coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb ('diagnose connectivity issues') and resources ('between a source service and a target resource'), distinguishing it from sibling tools like 'check_resource_limits' or 'get_correlated_logs'. It explicitly mentions what the tool checks (Vercel environment variables, AWS Security Group rules, external API reachability) and what it returns (diagnostic results with actionable messages).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for diagnosing connectivity issues between services and resources. It doesn't explicitly state when NOT to use it or name alternatives among sibling tools, but the specificity of its purpose strongly implies it's for connectivity diagnostics rather than resource monitoring or log analysis.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_cloud_topologyA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions scanning 'all configured cloud platforms' and returning a 'unified topology,' which gives some context about scope and output format. However, it doesn't disclose important behavioral aspects like authentication requirements, rate limits, execution time, or what happens if platforms aren't properly configured.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each serve distinct purposes: the first explains what the tool does, and the second provides usage guidance. There's zero wasted language, and the most important information (the scanning action) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (scanning multiple cloud platforms) and lack of both annotations and output schema, the description is somewhat incomplete. While it explains the purpose and usage timing well, it doesn't address authentication needs, error handling, or the structure of the returned topology. For a discovery tool with no output schema, more detail about the return format would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema. The baseline score of 3 is appropriate when the schema does the heavy lifting for parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Scan all configured cloud platforms'), the resource ('active services'), and the output ('unified topology of active services including their endpoints and regions'). It distinguishes this tool from siblings by emphasizing its discovery/scanning purpose rather than diagnostics or log analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('Run this first to understand the infrastructure landscape'), providing clear guidance about its role as an initial discovery step. This differentiates it from sibling tools like check_resource_limits or diagnose_service_link that would be used after understanding the topology.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.