Server Quality Checklist

Tool Scores

• get_bedrock_hourly_usage_statsC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. While it mentions the tool returns 'Formatted string representation of hourly usage statistics,' it doesn't disclose important behavioral traits like whether this is a read-only operation, authentication requirements, rate limits, data freshness, or what happens if parameters are invalid. The description is minimal and lacks operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief (4 sentences) and front-loaded with the core purpose. However, the 'Args' and 'Returns' sections are redundant since they don't add value beyond the tool name and basic function. The structure is clear but includes unnecessary boilerplate that doesn't enhance understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 parameters with 0% schema coverage, no annotations, no output schema), the description is inadequate. It doesn't explain what 'hourly usage statistics' includes, how data is aggregated, what format the returned string uses, or provide context about the Bedrock service. For a tool with multiple parameters and no structured documentation, the description should do much more to compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description states 'Parameters specifying the number of days to look back and region,' but the actual input schema shows 4 parameters (days, region, log_group_name, aws_account_id) with 0% schema description coverage. The description fails to mention two parameters entirely (log_group_name and aws_account_id) and provides no meaningful semantic context beyond what's minimally implied by parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get hourly usage statistics with detailed breakdowns.' It specifies the verb ('Get'), resource ('hourly usage statistics'), and scope ('detailed breakdowns'). However, it doesn't explicitly differentiate from sibling tools like 'get_bedrock_daily_usage_stats' or 'get_detailed_breakdown_by_day'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention sibling tools, specify use cases, or provide any context about when this hourly breakdown is preferred over daily statistics or other available tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_bedrock_daily_usage_statsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'detailed breakdowns' and that it returns a 'formatted string representation,' but doesn't disclose important behavioral traits like whether this is a read-only operation, requires specific AWS permissions, has rate limits, or what format the string output takes. For a tool with no annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with three sentences that are front-loaded: the first states the purpose, followed by Args and Returns sections. There's minimal waste, though the parameter explanation could be more informative. The structure is clear but could better integrate parameter details into the flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 nested parameters, no annotations, no output schema), the description is incomplete. It doesn't adequately explain the parameters, behavioral aspects, or output format. For a tool that likely interacts with AWS Bedrock and returns usage statistics, more context on permissions, data scope, and result interpretation is needed to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, meaning parameters are undocumented in the schema. The description only vaguely mentions 'Parameters specifying the number of days to look back and region,' but doesn't explain the actual parameters (days, region, log_group_name, aws_account_id) or their purposes. It fails to compensate for the schema's lack of descriptions, leaving most parameters semantically unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get daily usage statistics with detailed breakdowns.' It specifies the verb ('Get') and resource ('daily usage statistics'), and distinguishes from the hourly sibling tool by emphasizing 'daily' usage. However, it doesn't explicitly differentiate from 'get_detailed_breakdown_by_day' which might provide similar functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention the sibling tools (get_bedrock_hourly_usage_stats, get_detailed_breakdown_by_day, get_ec2_spend_last_day) or explain when daily statistics are preferred over hourly ones or other breakdowns. The only implied context is the need for 'daily' statistics, but no explicit usage rules are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_detailed_breakdown_by_dayC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It mentions the tool retrieves cost data and returns a nested dictionary or error, but lacks critical behavioral details: authentication requirements, rate limits, whether it's read-only/destructive, or how errors manifest. This is inadequate for a tool with potential complexity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise with three sections (purpose, args, returns) and no wasted sentences. However, the return value explanation is somewhat verbose and could be streamlined for better front-loading of key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 0% schema coverage, no output schema, and sibling tools with similar names, the description is incomplete. It doesn't explain the tool's scope (e.g., EC2-specific vs. general AWS costs), error handling details, or how results differ from siblings, leaving significant gaps for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'Parameters specifying the number of days to look back', which partially explains the 'params' object but omits details about 'region' and 'aws_account_id' sub-parameters. This adds some value but doesn't fully bridge the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Retrieve') and resource ('daily spend breakdown') with specific dimensions (region, service, instance type). However, it doesn't explicitly differentiate from sibling tools like 'get_ec2_spend_last_day' which might overlap in purpose, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_ec2_spend_last_day' or 'get_bedrock_daily_usage_stats'. There's no mention of prerequisites, context, or exclusions, leaving the agent to guess based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ec2_spend_last_dayC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions the API used and error handling (returns None on error), but lacks details on permissions, rate limits, cost implications, or what 'last day' means precisely (e.g., UTC day, rolling 24h). This is inadequate for a tool that likely requires AWS credentials and has financial implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by a concise return value note. It's appropriately sized with no wasted words, though the return type detail could be more integrated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 0% schema description coverage, the description is incomplete. It covers the basic purpose and error handling but misses critical context like authentication needs, cost behavior, and sibling tool differentiation, making it only minimally viable for this AWS cost tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it provides no parameter information. The baseline is 3 because the schema fully documents the single 'params' object with its nested fields (days, region, aws_account_id), including defaults and constraints, making the description's lack of param details less critical.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Retrieve' and resource 'EC2 spend for the last day', specifying the AWS Cost Explorer API as the method. It distinguishes from siblings by focusing on EC2 spend rather than Bedrock usage or detailed breakdowns, though it doesn't explicitly contrast them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives is provided. The description doesn't mention sibling tools or suggest scenarios for choosing this tool over others, leaving the agent without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.