Server Quality Checklist

Tool Scores

• describe_tableB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description should disclose behavioral traits like permissions, speed, or side effects, but it only states the output. No additional transparency beyond the basic verb.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise (8 words), front-loaded with the output. However, it may be too sparse for effective use. A balanced score of 4 reflects efficiency without being wasteful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema, the description adequately lists what is returned, but does not specify format, order, or limitations. Adequate but with clear gaps for a complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0% for the 'table' parameter, and the tool description does not clarify the expected format (e.g., fully qualified name, case sensitivity). It adds no value over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Return' and the resource 'columns, types, and row count for a table'. It distinguishes from siblings like list_tables (lists tables) and profile_table (more stats) by specifying what it returns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives (e.g., profile_table, run_sql). The description only states functionality without context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sample_rowsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries the burden. It discloses the row limit (max 100) and default (10), which is helpful, but doesn't mention ordering, randomness of sampling, or any side effects (though likely none). It could be more transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence. It is front-loaded with the action ('Return up to n preview rows') and includes the key constraint (max 100). Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema), the description covers the basics: what it returns, the limit, and the required table. However, it leaves ambiguity about 'preview' (first N rows vs. random sample) and lacks detail on return structure. Adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% (no descriptions for params in schema). The description mentions 'n' and 'table' but adds no new details about their format, constraints (beyond max), or semantics. It fails to compensate for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns preview rows from a table, with a specific verb and resource. It distinguishes from siblings like 'describe_table' (schema info) or 'run_sql' (arbitrary queries), though it doesn't explicitly contrast them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for quick data previews via 'preview rows', but lacks explicit guidance on when to use this over siblings, prerequisites, or when not to use it (e.g., for full data extraction).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• profile_tableA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It implies a read operation (returning statistics) but does not explicitly confirm read-only behavior, required permissions, or whether it can be called on any table. The lack of detail about side effects or access control leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One concise sentence (11 words) with no filler. It front-loads the verb and key outputs, making it efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core outputs (null fraction, distinct count, min/max) and assumes the output schema details the return structure. Given one parameter and no nested objects, it is mostly complete. Missing context: whether all columns are profiled or only specified ones, but that is implied.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description adds 'for a table,' clarifying that the single required parameter 'table' is the table name. While minimal, it adds enough meaning beyond the raw schema. A higher score would require explicit format details, but the param is simple enough.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns per-column statistics (null fraction, distinct count, min/max) for a table. It uses a specific verb ('Return') and resource ('per-column... for a table'), differentiating it from siblings like describe_table (schema) or run_sql (arbitrary queries).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. Siblings like describe_table and sample_rows exist, but the description does not clarify scenarios (e.g., use profile_table for data quality, describe_table for column metadata). This forces the agent to infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_sqlA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses read-only nature, rejection of non-SELECT queries, and automatic LIMIT enforcement, which are key behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The first sentence states the core action, the second adds restrictions. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a single parameter, no annotations, and no output schema, the description covers essential behavior: read-only, restrictions, and automatic LIMIT. It could mention maximum LIMIT or error handling, but it's adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'query' has no schema description (0% coverage). The description adds value by specifying allowed query types and constraints, but lacks details like format or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a read-only SELECT query, which is the core purpose. While it doesn't explicitly differentiate from sibling tools, the verb 'run' and restriction to SELECT queries imply a distinct use case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use (run a SELECT query) and what is rejected (DDL/DML/multi-statement, automatic LIMIT). It provides clear constraints but doesn't contrast with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries the burden. It adds that column counts are included, but lacks details on scope, performance, or limitations. Adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with the action, no wasted words. Perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and an output schema present, the description provides the essential information. Minor omission: 'warehouse' context is vague but acceptable for a simple listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so no additional explanation needed. Baseline 4 applies as schema coverage is 100% and description adds nothing beyond.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists tables with column counts, a specific verb-resource combination that distinguishes it from siblings like describe_table (single table) and run_sql (arbitrary queries).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While not explicitly stating when to use versus alternatives, the distinct purpose makes it obvious. Siblings are clearly different, so usage context is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.