Server Quality Checklist

Tool Scores

• inspect_task_definitionD
  1.5/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose safety, side effects, and return behavior. It only says 'Inspect a task definition,' which is a restatement. There is no mention of read-only nature, data volume, or any other behavioral characteristic.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only four words, which is extremely under-specified. While it is concise, it lacks any useful structure or front-loaded key information that would help an agent. It is a tautology rather than a meaningful description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although there is an output schema, the description is incomplete for a task that has two required parameters and no annotations. The description fails to explain what the tool does beyond the name, and does not account for the need to specify region and task definition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 2 parameters (region, task_definition) with zero description coverage. The description does not explain what these parameters mean, their format, or how they relate to the inspection. Since schema coverage is 0%, the description must compensate but does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Inspect a task definition' is essentially a tautology of the tool name. It provides no additional detail about what aspects of the task definition are inspected or what the output represents. It does not distinguish from sibling tools like inspect_ecs_service_events or inspect_target_health.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no mention of when to use this tool, what parameters are required, or how it differs from sibling tools like investigate_ecs_service or search_service_logs. The description provides no context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• investigate_ecs_serviceD
  1.8/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It merely says 'investigation' without revealing whether calls are read-only, what data is gathered, any side effects, or anything about the response size or structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence with no wasted words, but it is under-specified to the point of being ineffective. It does not earn its place as a description because it provides minimal actionable information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, an output schema, and no annotations, the description is severely incomplete. It fails to explain what the investigation covers, the role of each parameter, or how the output is structured, making it inadequate for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage for its 5 parameters, and the description does not compensate by explaining any of them. An agent gets no help in understanding what 'lookback_minutes' means or how 'include_logs' behaves.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Run a complete ECS service investigation' starts with a verb and names a resource, but 'investigation' is too generic to convey the specific scope. It hints at comprehensiveness but doesn't distinguish itself from sibling tools like inspect_ecs_service_events or search_service_logs beyond a vague 'complete'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus the sibling tools. The description neither states appropriate contexts nor mentions alternatives, leaving the agent to guess.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_service_logsC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It merely states that logs are searched, without revealing that it searches CloudWatch Logs with configurable lookback, query, or limit parameters, or any details about return behavior or access requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no wasted words. It is appropriately front-loaded, though it sacrifices needed detail for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has five parameters, no annotations, and only a one-sentence description, it is incomplete for effective use. The presence of an output schema covers return values, but the description still lacks essential context about how to construct queries, set time windows, or interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no parameter-level context. The five parameters (log_group, region, limit, query, lookback_minutes) are completely undocumented beyond their schema definitions, which is especially problematic for required parameters like log_group and region.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Search') and the resource ('service logs in CloudWatch Logs'). It is distinguishable from the sibling ECS inspection tools, which focus on services, events, task definitions, and target health rather than log contents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling investigation tools. There is no mention of appropriate scenarios, prerequisites, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_ecs_service_eventsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description alone must convey behavioral traits. 'Inspect' implies a read-only operation but the description does not explicitly state safety, ordering, pagination, or the meaning of 'recent'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The single sentence is concise and front-loaded, containing no filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters (3 required) and no annotations, the description is too sparse. Even with an output schema, it omits parameter guidance and usage context, making it insufficient for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides zero parameter semantics; all four parameters ('cluster', 'service', 'region', 'limit') are left unexplained, leaving the agent to rely solely on property names. 0% schema description coverage means the description fails to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Inspect') and identifies the exact resource ('recent ECS service events'), clearly distinguishing it from sibling tools like 'investigate_ecs_service' and 'inspect_task_definition'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No when-to-use guidance or alternatives are provided. It does not explain in which scenarios one would inspect events versus using 'search_service_logs' or 'investigate_ecs_service'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_target_healthC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose whether the operation is read-only, requires specific AWS permissions, or what output is returned (though an output schema exists). The term 'Inspect' implies a safe read, but this is not explicitly stated, and side effects or rate limits are not addressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, empty-free sentence that directly states the tool's purpose. It could be more informative, but it is appropriately brief for the simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two required parameters and an output schema, but the description lacks usage context and behavioral disclosure. Given no annotations and minimal description, it does not fully prepare an agent for when and how to invoke it, though the output schema covers return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not explain the parameters 'target_group_arn' and 'region'. With 0% schema description coverage, the description needed to compensate, but it only mentions the resource generally. The parameter names are self-explanatory, but no added context (e.g., ARN format or region default) is given.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Inspect') and resource ('target health for one target group'). It distinguishes from sibling tools which focus on ECS services, events, task definitions, and logs, not target health.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling inspection tools. It doesn't specify prerequisites (e.g., need a target group ARN) or typical scenarios (e.g., troubleshooting unhealthy instances). No alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.