Server Quality Checklist

Tool Scores

• get_instance_healthC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It implies a read-only operation via 'Check' but does not disclose return format, potential errors, or the exact scope of health checks. This lacks the detail needed for an agent to understand side effects or limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence, which is efficient, but it is under-specified. It provides a clear action but lacks any structural breakdown or additional context, making it borderline between appropriate and minimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter read-only tool without an output schema or annotations, the description is thin. It does not clarify what health checks are included, what the response contains, or how this relates to the sibling tools, leaving the agent with significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter, instance_name, with 0% description coverage. The description does not mention the parameter or how to specify the instance, relying solely on the parameter name which is somewhat self-explanatory but not explicitly documented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as checking EC2 state and AWS system/instance health checks, which is a specific verb+resource combination. It distinguishes from sibling tools like get_instance_metrics and get_service_status, though it could be more explicit about the exact health check details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There is no mention of exclusions, prerequisites, or comparison with sibling tools such as get_instance_metrics or get_service_status.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_instance_metricsC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description must carry the full burden of behavioral disclosure. It only states the metric source (CloudWatch) but does not reveal whether this is a read-only operation, how metrics are aggregated, whether results are time-bounded, or any rate limits. The word 'fixed' is ambiguous and not explained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words, front-loading the action and resource. However, the term 'fixed' could be clearer, slightly reducing the overall conciseness benefit due to ambiguity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema or annotations, the description should provide more context about what metrics are returned, how the 'minutes' parameter influences results, and any limitations. The current one-liner is insufficient for an agent to invoke this tool reliably without further guesswork.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has two parameters (instance_name, minutes) with no descriptions, and the tool description does not mention them at all. With schema description coverage at 0%, the agent is left with no clue about what these parameters mean or how they affect the call, making this a severe gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Get) and the resource (EC2 performance and status metrics) with a specific source (CloudWatch). It distinguishes itself from sibling tools that focus on health, errors, service status, or journal, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the sibling tools (e.g., get_instance_health, get_recent_errors). The description does not mention exclusions, alternatives, or conditions that would help an agent choose correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_recent_errorsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior, but it only says 'recent' without defining the time window or clarifying that this is a read-only operation. It does not mention return format, pagination, or how the parameters affect results. The term 'recent' is vague and the description lacks critical behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that is front-loaded with the core action. Every word is informative, with no filler or redundancy. It is appropriately concise for the simple function it describes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the moderate complexity (3 parameters, no output schema, no annotations), the description is far too sparse. It does not explain what constitutes an error, how the parameters interact, or what the response will look like. The agent is left with insufficient context to use the tool reliably.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not explain any of the three parameters (instance_name, minutes, maximum_results). Schema description coverage is 0%, so the description provides no additional meaning beyond the raw schema. The agent cannot infer parameter semantics from the description alone; it must guess or have prior knowledge.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and clearly identifies the resource ('recent CloudWatch application and operating-system errors'). It distinguishes from sibling tools like get_instance_health or get_service_status, which focus on different data (health, status). There is no ambiguity about what this tool retrieves.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the siblings. It does not mention scenarios, exclusions, or alternative tools. The agent is left to infer from the name alone that it is for errors, with no direction on how it differs from get_service_journal or get_instance_metrics.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_service_statusB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It adds useful context ('approved service', 'current state') but omits behavioral details such as read-only nature, return format, permissions, or error behavior. It's minimally adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no extraneous words. It efficiently states the tool's action and target, earning its place despite its brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with no output schema, the description is reasonably complete but leaves gaps: it doesn't explain what 'state' means, what a return value looks like, or what 'approved service' implies. Given sibling tools and no annotations, more context would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no meaning to the parameters. While parameter names (instance_name, service_name) are somewhat self-explanatory, the description does not elaborate on format, allowed values, or how they relate to the 'approved service' concept, leaving the agent to infer entirely from names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('check') and resource ('state of an approved service on an instance'), clearly identifying the tool's purpose. It distinguishes from siblings like get_instance_health (instance-level) and get_recent_errors (errors), though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description implies usage for checking service status but offers no context, exclusions, or comparisons to sibling tools like get_service_journal or get_instance_metrics.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_service_journalA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It adds 'bounded' and 'fixed SSM document' as behavioral context, but does not explain output format, pagination, or prerequisites like SSM agent availability. As a read operation, it is not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence, front-loaded with the main verb and resource. Every word adds meaning, with no unnecessary repetition or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, and the description fails to specify the return format, bounding behavior, or potential prerequisites (e.g., SSM document requirements). It also does not differentiate from sibling tools, leaving the agent with limited operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does not directly explain any of the four parameters; 'bounded' loosely hints at minutes and maximum_results but does not clarify their meaning or the required parameters. This is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and the resource ('bounded nginx systemd journal'), which is specific and distinct from sibling tools that focus on health, metrics, errors, or status. The term 'bounded' also adds scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context implies usage is for retrieving nginx journal logs, but it does not explicitly state when to use this tool versus alternatives, nor does it mention any exclusions. It relies on the tool's name and sibling set for differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.