Server Quality Checklist

Tool Scores

• explain_queryA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only and idempotent, but the description goes further: 'no rows are read and nothing is executed', explains why EXPLAIN ANALYZE is omitted, and clarifies total_cost as relative. It also discloses failure modes, adding significant context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the primary action, and uses a bulleted list for failure cases. Every sentence adds value—no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having only one parameter and no param descriptions, the description covers usage context, safety, failure handling, and relative output interpretation. An output schema exists, so return values need not be described. It is fully complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and there is one parameter (sql). The description compensates by implying sql must be a plain SELECT (via 'Rewrite as a plain SELECT') and explains column-error behavior. It does not explicitly label the parameter but the meaning is clear from context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb+resource: 'Plan a SELECT without executing it.' This clearly distinguishes the tool from run_query (which executes) and explains its core function. It also scopes it to SELECT statements, avoiding ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to 'Use this before run_query' under conditions (large table, no WHERE, joins), and names the alternative run_query. On failure, it directs the user to call describe_table for schema errors, providing clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_queryA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses read-only policy enforcement details: only single SELECT accepted, rejects DML/DDL/stacking (including DML in CTEs), with error prefix 'Blocked: ...' explained. Also discloses result capping and the meaning of the truncated flag—all beyond the readOnlyHint and idempotentHint annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with a clear opening purpose and organized failure/recovery sections. Every sentence adds operational value; no filler or redundancy. Length is justified by the number of constraints and failure modes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, usage timing, policy restrictions, failure remediation, and output limits. With an output schema present, return values need no further description. The description is sufficiently complete for the agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema provides no descriptions (0% coverage). The description thoroughly explains the sql parameter: must be a read-only SELECT, single statement, WITH allowed. However, max_rows is only indirectly addressed via result capping/truncation; it doesn't explicitly state that max_rows controls the cap. Still, it adds substantial meaning for the required parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States specifically: 'Execute a read-only SELECT and return the rows.' This clearly conveys the verb (execute), resource (SQL SELECT), and output (rows). It differentiates from sibling search_schema (schema discovery), describe_table (schema details), and explain_query (plan analysis).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs: 'Use this once you know which tables and columns you need. If you do not yet know, call search_schema first rather than guessing table names.' Also provides fallback guidance to call describe_table after schema errors and explain_query after timeouts, making alternatives and exclusions explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_tableA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive, but the description adds valuable behavioral details: accepted name formats (qualified vs bare), return contents (columns, types, comments, PK, FKs), and failure semantics with close-match suggestions and explicit 'do not retry/guess' instructions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, then organized into usage context, accepted inputs, return values, and failure handling. Every sentence adds operational value and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Considering the tool's moderate complexity and the presence of an output schema, the description is complete: it covers when to use, input formats, what the output contains (including how to find joining tables), and exact failure recovery steps. The sibling context (search_schema) is also directly addressed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only says 'name' is a required string (0% coverage), so the description fully compensates by explaining that the parameter accepts qualified or bare names, provides concrete examples, and clarifies the failure-response behavior associated with the name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Return full detail for one table by name.' It explicitly distinguishes itself from search_schema by noting it does not use ranking, making it the reliable way to reach a table with poor retrieval scores.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use scenarios: when the table name is already known from search_schema or when the table is believed to exist despite search_schema not returning it. It also provides fallback guidance on failure, telling the agent to try close matches and then fall back to search_schema with a description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_schemaA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds valuable behavioral context beyond this: ranking based on column names/types/comments, returning highest-scoring tables with columns and keys, foreign-key edges between returned tables, and failure-mode behavior. This significantly enhances the agent's understanding of how the tool operates.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average but well-structured and front-loaded with the core purpose. Every sentence serves a purpose: explaining the tool's behavior, providing usage context, or detailing failure handling. The bullet-point structure for failure cases improves scannability without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for the tool's complexity. It covers the tool's output (highest-scoring tables, columns, keys, foreign-key edges), input semantics, usage strategy, failure handling, and cost considerations. The presence of an output schema reduces the need to describe return details, and the description goes beyond the minimum.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must carry the burden. It explains that 'question' should be natural language ('Pass the user's question in natural language, not a table name guess') and clarifies the 'limit' parameter's cost/benefit ('Raising limit costs tokens and rarely fixes a vocabulary mismatch'). This fully compensates for missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence explicitly states the tool's purpose: 'Find the tables relevant to a question and return their DDL.' This uses a specific verb and resource, clearly distinguishing it from siblings like describe_table (which retrieves a single table) and run_query (which executes SQL).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Prefer this over trying to read the whole schema' and outlines specific fallback instructions, including when to call describe_table ('If a table you know you need is missing'), how to rephrase the question with domain words, and a caution against inventing table names. This creates a clear decision tree for when to use this tool vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.