Server Quality Checklist

Tool Scores

• describe_tableC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves information, implying a read-only operation, but doesn't specify permissions required, rate limits, error handling, or the format of returned column information, leaving significant gaps for a tool with no structured safety hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded and wastes no space, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and low schema coverage, the description is incomplete. It lacks details on behavioral traits, parameter constraints, and return values, which are critical for a tool that retrieves structured data. This leaves the agent with insufficient context for reliable invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'tableName' implicitly by referring to 'a table', but with 0% schema description coverage and one parameter, it adds minimal semantic value beyond what the schema's property name suggests. Since schema coverage is low, the description doesn't compensate with details like valid table name formats or examples, meeting the baseline for a single parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Retrieves') and resource ('column information for a table'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from siblings like 'list_tables' (which likely lists table names) or 'query' (which might execute SQL queries), missing full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_tables' or 'query'. It lacks context about prerequisites, such as needing an existing table name, or exclusions, leaving the agent to infer usage from the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queryB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses 'read-only', indicating non-destructive behavior, which is useful. However, it lacks details on permissions, rate limits, error handling, or response format, leaving significant behavioral gaps for a tool that executes SQL queries.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, clearly stating the core action without unnecessary elaboration, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of executing SQL queries, no annotations, no output schema, and low schema coverage, the description is incomplete. It lacks information on return values, error cases, or operational context, making it inadequate for safe and effective tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter details. The description adds minimal semantics by implying the 'sql' parameter is for SQL queries, but it doesn't explain syntax, supported SQL dialects, or constraints, failing to compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('executes') and resource ('SQL query'), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'test_execute' or 'describe_table', which might also involve query execution or database operations, so it's not fully specific to sibling context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'test_execute' or 'list_tables'. It mentions 'read-only', which hints at usage context, but lacks explicit when/when-not instructions or named alternatives, leaving the agent with minimal direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• test_executeB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: the tool checks executability and performs a rollback, indicating it's a safe, non-destructive operation. However, it doesn't mention error handling, performance implications, or what 'checks' entails (e.g., syntax validation, permission checks), leaving gaps for a mutation-related tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence: 'Checks if an SQL query can be executed and rolls back afterward.' It's front-loaded with the core purpose, has zero waste, and appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool involves SQL execution testing (a mutation-related operation), no annotations, no output schema, and low parameter coverage, the description is incomplete. It lacks details on return values (e.g., success/failure indicators), error responses, or rollback specifics, making it inadequate for safe agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'SQL query' as the input, which adds meaning to the 'sql' parameter by specifying it's an SQL query string. However, it doesn't provide details on query format, supported SQL dialects, or constraints, failing to fully compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Checks if an SQL query can be executed and rolls back afterward.' This specifies the verb ('checks') and resource ('SQL query'), and distinguishes it from siblings like 'query' (which likely executes queries) and 'describe_table'/'list_tables' (which are read-only metadata tools). However, it doesn't explicitly differentiate from 'query' beyond implying a test vs. execution distinction, keeping it at 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context: it's for testing SQL queries before actual execution, suggesting when to use it (to validate queries) versus alternatives like 'query' (for actual execution). However, it doesn't explicitly state when-not-to-use or name alternatives, and lacks prerequisites (e.g., database connection requirements), so it's not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states it 'retrieves' data (implying read-only), but doesn't mention any constraints like permissions needed, rate limits, pagination, or what format the list returns. This leaves significant behavioral gaps for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It's front-loaded with the core purpose and efficiently communicates the essential function without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list-retrieval tool with no parameters and no output schema, the description is minimally adequate. However, without annotations or output details, it lacks completeness regarding behavioral aspects like return format or constraints. It meets basic needs but has clear gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100% (though trivial since there's no schema). The description doesn't need to explain parameters, so it appropriately avoids unnecessary detail. A baseline of 4 is justified as it doesn't mislead about parameters that don't exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Retrieves') and resource ('list of tables in the database'), making the tool's purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'describe_table' or 'query', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'describe_table' (for table details) or 'query' (for data retrieval). The description only states what it does, not when it's appropriate, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.