Server Quality Checklist

Tool Scores

• redshift_connection_statusC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'Returns: Connection status information,' which hints at a read-only operation, but doesn't specify what that information includes (e.g., success/failure, latency, error details) or any behavioral traits like rate limits, authentication needs, or side effects. For a tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two short sentences, front-loading the purpose and then stating the return. There's no wasted text, and it's appropriately sized for a simple tool. However, the second sentence could be more informative, slightly reducing efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, output schema exists), the description is somewhat complete but has gaps. It states the purpose and return, but lacks details on what 'connection status information' entails, which the output schema might cover. With no annotations, it should provide more behavioral context, but the output schema reduces the need to explain return values fully.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so there are no parameters to document. The description doesn't need to add parameter semantics beyond the schema, and it correctly doesn't mention any. Baseline 4 is appropriate for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's purpose as 'Check the Redshift connection status,' which is a clear verb+resource combination. However, it doesn't differentiate this from sibling tools like redshift_query or redshift_list_tables, which might also involve connection checks implicitly. The purpose is clear but lacks sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, such as whether a connection must be established first, or contrast it with siblings like redshift_query that might also test connectivity. There's no explicit when/when-not or alternative tool recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redshift_describe_tableC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool returns a JSON description of columns, which is helpful, but lacks details on error handling (e.g., what happens if the table doesn't exist), performance characteristics, or authentication requirements. This is a significant gap for a tool that interacts with a database.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise and well-structured with clear sections for Args and Returns. Each sentence serves a purpose, though it could be more front-loaded by moving the return statement closer to the purpose. There's no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (database metadata retrieval) and the presence of an output schema, the description is minimally adequate. However, with no annotations and low parameter semantics, it fails to fully address behavioral aspects like error conditions or usage context, leaving gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal semantic context beyond the input schema, which has 0% description coverage. It clarifies that 'schema' defaults to 'public', but doesn't explain what a schema is in Redshift context or provide examples. For a tool with 2 parameters, this is inadequate, though the output schema might mitigate some gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('column definitions for a table'), making it easy to understand what it does. However, it doesn't explicitly differentiate from sibling tools like 'redshift_list_tables' or 'redshift_query', which might also provide table information in different ways.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'redshift_list_tables' (which lists tables) or 'redshift_query' (which might return column info as part of query results). There's no mention of prerequisites, such as needing an active Redshift connection, or typical use cases like schema exploration.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redshift_get_sample_dataB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool returns 'JSON sample data' but doesn't specify behavioral traits such as whether this is a read-only operation, potential rate limits, authentication requirements, error handling, or how it interacts with large tables. This leaves significant gaps in understanding the tool's operational characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and well-structured, with a clear purpose statement followed by bullet points for arguments and returns. Every sentence earns its place by directly contributing to understanding the tool's functionality without unnecessary elaboration or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no annotations) and the presence of an output schema (which handles return values), the description is partially complete. It covers basic purpose and parameters but lacks behavioral context and usage guidelines, making it adequate for simple use but insufficient for robust agent decision-making in more complex scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful semantics beyond the input schema by explaining each parameter's purpose: 'table_name' as the table to sample from, 'limit' as the number of rows (with a default), and 'schema' as the schema name (with a default). Since schema description coverage is 0%, this compensates well, though it could provide more context like valid schema formats or limit constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('sample rows from a table'), making it immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'redshift_query' or 'redshift_describe_table', which could also retrieve data or metadata from tables, leaving some ambiguity about when this specific sampling function is preferred.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'redshift_query' for custom queries or 'redshift_describe_table' for metadata. It lacks context about typical use cases (e.g., quick data inspection vs. detailed analysis) or prerequisites, leaving the agent to infer usage based on the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redshift_queryB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions that results are returned as JSON and errors are handled, it lacks critical behavioral details like authentication requirements, query timeout limits, result size constraints, whether queries are read-only or can modify data, or any rate limiting considerations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with three clear sentences that each serve a purpose: stating the action, describing the parameter, and explaining the return. It's front-loaded with the core functionality. The only minor improvement would be integrating the parameter description more seamlessly rather than using 'Args:' formatting.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (database query execution), lack of annotations, and presence of an output schema, the description is minimally adequate. The output schema existence means the description doesn't need to detail return values, but it should provide more behavioral context about query execution constraints and safety considerations for a database operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides basic semantic information about the single parameter ('The SQL query to execute'), which adds meaning beyond the bare schema. However, it doesn't elaborate on SQL dialect specifics, query validation, or parameter binding capabilities that would be helpful for proper usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Execute a SQL query on Redshift') and resource ('Redshift'), distinguishing it from sibling tools like redshift_list_tables or redshift_describe_table. It provides a complete verb+resource+scope statement that leaves no ambiguity about what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like redshift_get_sample_data or redshift_describe_table. There's no mention of prerequisites, appropriate query types, or limitations that would help an agent choose between this and other data retrieval tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redshift_list_tablesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states it's a list operation. It doesn't disclose behavioral traits like whether it requires specific permissions, how it handles large result sets, or potential rate limits. This leaves significant gaps for a tool that interacts with a database.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by clear Arg and Return sections. Every sentence earns its place with no wasted words, making it highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter) and the presence of an output schema (which handles return values), the description is reasonably complete. It covers the purpose, parameter meaning, and return type, though it could benefit from more behavioral context given the lack of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for the single parameter by explaining that 'schema' is the schema name with a default of 'public'. Since schema description coverage is 0%, this compensates well, though it doesn't detail constraints like valid schema names or case sensitivity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and resource 'tables in a specific schema', making the purpose unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'redshift_describe_table' or 'redshift_query', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the phrase 'in a specific schema' and mentions a default value, suggesting context for when to use it. However, it lacks explicit guidance on when to choose this tool over alternatives like 'redshift_describe_table' or 'redshift_query', leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.