Server Quality Checklist

Tool Scores

• get_statusC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action is to 'Get' status, implying a read-only operation, but doesn't specify whether this requires authentication, has rate limits, or details the return format (e.g., pending, completed, error). The description is minimal and misses key behavioral traits for a tool interacting with query executions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words, making it highly concise and front-loaded. It directly states the tool's purpose without unnecessary elaboration, which is efficient for an AI agent to parse and understand.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of query execution tools and the lack of annotations and output schema, the description is incomplete. It doesn't explain what the status includes (e.g., progress indicators, error messages) or how it relates to sibling tools like 'get_result'. For a tool that likely returns dynamic execution state information, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'queryExecutionId' documented as 'The query execution ID'. The description doesn't add any meaning beyond this, such as explaining where to obtain this ID or its format. Given the high schema coverage, a baseline score of 3 is appropriate, as the schema adequately handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'current status of a query execution', making the purpose specific and understandable. However, it doesn't explicitly distinguish this tool from its siblings like 'get_result' or 'run_query', which might also relate to query execution status or results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_result' or 'run_query'. It lacks context about prerequisites, such as needing a query execution ID from a previous operation, and doesn't mention any exclusions or specific scenarios for its use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_saved_queryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool executes a query but doesn't mention what happens during execution (e.g., query runs on Athena, may take time, returns results directly or via reference), potential side effects, authentication needs, rate limits, or error handling. This is inadequate for a tool that likely involves computational resources and timeouts.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. Every part earns its place by specifying the action, resource type, and key identifier, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of executing Athena queries (involving timeouts, row limits, and potential async behavior), no annotations, and no output schema, the description is insufficient. It doesn't explain what the tool returns (e.g., results, status ID, error details), how to handle large results, or integration with sibling tools like 'get_status' for monitoring. This leaves critical gaps for an agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description doesn't add any parameter-specific information beyond implying 'namedQueryId' is required. It doesn't explain relationships between parameters (e.g., how databaseOverride interacts with saved query settings) or provide usage examples, so it meets the baseline but doesn't enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Execute') and resource ('saved (named) Athena query by its query ID'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'run_query' or 'get_result', but the focus on saved/named queries provides some implicit distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'run_query' (for ad-hoc queries) or 'get_result' (for retrieving results). It mentions saved/named queries but doesn't clarify prerequisites (e.g., needing a saved query ID from 'list_saved_queries') or when this is preferred over other execution methods.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_resultA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds value by stating the error condition ('Returns error if query is still running'), which is useful context beyond the input schema. However, it lacks details on permissions, rate limits, response format, or pagination behavior (e.g., how maxRows affects output). For a tool with no annotations, this is a moderate disclosure, scoring a 3.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded: two sentences that directly state the tool's function and a key behavioral constraint. Every sentence earns its place by providing essential information without waste, making it efficient and well-structured for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is somewhat complete but has gaps. It covers the basic purpose and an error condition, but lacks details on return values (since no output schema exists), authentication, or how it integrates with sibling tools. For a result-retrieval tool, this is minimally adequate, scoring a 3.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema fully documents both parameters (queryExecutionId and maxRows). The description does not add any parameter-specific semantics beyond what the schema provides (e.g., it doesn't explain the format of queryExecutionId or how maxRows interacts with query results). According to the rules, with high schema coverage (>80%), the baseline is 3 even with no param info in the description, which applies here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get results for a completed query.' It specifies the verb ('Get') and resource ('results'), and distinguishes it from siblings like 'get_status' (which checks query status) and 'run_query' (which executes queries). However, it doesn't explicitly differentiate from 'list_saved_queries' or 'run_saved_query' in terms of result retrieval, making it a 4 rather than a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance: it should be used only for completed queries ('Returns error if query is still running'), suggesting an alternative might be 'get_status' to check completion first. However, it doesn't explicitly state when to use this tool versus alternatives like 'get_status' for status checks or 'run_query' for execution, nor does it mention prerequisites (e.g., needing a queryExecutionId from a prior run). This is adequate but has gaps, scoring a 3.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_saved_queriesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states what it does, not how it behaves. It doesn't disclose whether this requires specific IAM permissions, how results are paginated, if there are rate limits, or what the return format looks like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with zero waste - every word contributes to understanding the tool's purpose. Front-loaded with the core action ('List all saved queries') followed by clarifying details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only listing tool with no parameters and no output schema, the description is adequate but minimal. It doesn't address behavioral aspects like authentication requirements, result format, or limitations that would help an agent use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, which is correct for this case.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all saved queries') and identifies the target resource ('named Athena queries in your AWS account'). It distinguishes from siblings like run_saved_query (executes) and get_result/status (retrieves query results/metadata).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving available saved queries, but doesn't explicitly state when to use this versus alternatives like run_saved_query or when not to use it. No prerequisites or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_queryA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key behavioral traits: it executes queries, returns full results if completed before timeout, otherwise returns a queryExecutionId. However, it lacks details on permissions, error handling, rate limits, or what 'full results' entails (e.g., format, pagination).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and followed by critical behavioral detail. Every word earns its place with no redundancy or fluff, making it highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 4 parameters, the description is moderately complete. It covers the core action and timeout behavior but misses details like result format, error cases, or integration with sibling tools (e.g., how queryExecutionId relates to get_result). For a query execution tool, more context would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description does not add meaning beyond what the schema provides (e.g., it doesn't explain parameter interactions or usage nuances). Baseline 3 is appropriate as the schema handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Execute a SQL query using AWS Athena') and the resource ('Athena'), distinguishing it from siblings like get_result (which retrieves results), get_status (checks status), list_saved_queries (lists saved queries), and run_saved_query (runs saved queries). It precisely defines what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning timeout behavior, but it does not explicitly state when to use this tool versus alternatives like run_saved_query or get_result. No explicit exclusions or prerequisites are provided, leaving usage context somewhat vague.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.