Server Quality Checklist

Tool Scores

• query_logs_insightsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavioral traits. It only states the tool supports full query syntax, but fails to mention whether the operation is read-only, any authorization requirements, rate limits, or what happens on error. This is insufficient for an execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences that efficiently convey the core purpose and capability. There is no extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no output schema and five parameters, the description is incomplete. It does not describe the return format, time range handling, or behavior for invalid queries. More context is needed for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all five parameters, so the descriptions already provide meaning. The tool description adds no additional parameter-level insight beyond 'supports the full query syntax' for the query parameter, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes CloudWatch Logs Insights queries for advanced log analysis, using a specific verb and resource. It effectively distinguishes from sibling tools like 'search_logs' by specifying that it supports the full Logs Insights query syntax.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions 'advanced log analysis' but does not give explicit context, prerequisites, or scenarios where this tool is preferred over sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_logsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It only mentions input capabilities, not output format, pagination, rate limits, or any side effects. Minimal behavioral info is provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no unnecessary words. Efficient but could be slightly more structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters and no output schema or annotations, the description lacks details on return format, default behavior, and how it compares to query_logs_insights. Incomplete for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are well-documented. The description adds context about relative time formats and CloudWatch filter patterns, but this is largely redundant with the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches CloudWatch logs using filter patterns and time-based queries. It distinguishes from siblings like list_log_groups and query_logs_insights by specifying the action and resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning supported features, but does not explicitly state when to use this tool vs alternatives like query_logs_insights, or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_log_groupsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It only states the basic operation, omitting details like read-only nature, rate limits, pagination (though 'limit' hints at it), or what happens without a prefix. Important behavioral context is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that communicates the core action and optional filter. It is front-loaded with verb and object, but some additional context could be added without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (two optional parameters, no output schema, no annotations), the description is minimally adequate. However, it lacks information on pagination, return format, or that it lists all groups when no prefix is given. Completeness is sufficient but has gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Parameter schema coverage is 100% with clear descriptions for both 'limit' and 'prefix'. The description adds 'optional prefix filter' but does not provide additional meaning beyond the schema. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the verb 'list', the resource 'CloudWatch log groups', and the optional prefix filter. It distinguishes from sibling tools like get_log_group_info (gets specific group info) and list_log_streams (lists streams within groups), so the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing log groups with an optional filter, but it does not explicitly state when to choose this tool over siblings or mention any prerequisites or alternatives. Usage context is implied but not spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_log_group_infoA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It states 'get detailed information' but does not explicitly declare read-only behavior, side effects, permissions needed, or rate limits. While 'get' implies no mutation, the description lacks explicit behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundancy or fluff. It efficiently conveys the tool's purpose and includes one useful behavioral note about the environment variable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description partially covers return values by mentioning 'retention settings and storage', but it lacks details on other common fields like creation time, stored bytes, or metric filters. For a simple tool, this is adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a description for the parameter, and the description adds value by noting that the logGroup parameter defaults to the CLOUDWATCH_LOG_GROUP environment variable. This provides context beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'get' and identifies the resource as 'CloudWatch log group', mentioning specific details like 'retention settings and storage'. It clearly distinguishes from sibling tools like list_log_groups, which lists groups, and query_logs_insights, which searches logs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives (e.g., list_log_groups for listing, search_logs for searching). The description only implies that you need a specific log group name and mentions an environment variable fallback, but does not provide context for decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_log_streamsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full burden. It mentions filtering behavior and provides an example, but does not disclose pagination, rate limits, or sorting behavior beyond what is in the schema. The description is functional but minimally transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the core action and includes a useful example. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a listing tool with no output schema, the description covers the main purpose and filtering options. It could mention return values or pagination behavior, but given the schema documents all parameters, it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds a concrete example of how prefix and date work together, but does not significantly enhance understanding of the parameters beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'list log streams in a log group' with specific filtering options (prefix, date). A concrete example is given, and the tool is well-distinguished from siblings like list_log_groups and query_logs_insights.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the example ('useful for service-based streams') but does not explicitly state when to use this tool versus siblings like query_logs_insights or search_logs. No when-not-to-use or alternative guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.