Server Quality Checklist

Tool Scores

• execute_queryC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only states the action and return type, omitting crucial details such as whether the query is read-only, impacts on data, or required permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise with a clear structure (Args, Returns). The purpose sentence is front-loaded, and every sentence serves a purpose despite the parameter mismatch.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema and no annotations, the description is insufficient. It lacks details about error handling, supported SQL syntax, database selection, and whether the tool is safe for read-only queries.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning for the 'query' parameter ('SQL query to execute'), but it also mentions 'ctx' and 'database' which are not in the input schema, causing confusion. Schema description coverage is 0%, so the description should compensate but instead introduces inconsistency.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Execute a SQL query and return the results,' which is a specific verb and resource. However, the mention of parameters not present in the input schema (ctx, database) slightly reduces clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus siblings like stream_query or copy_data. The description does not specify prerequisites or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• copy_dataC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It mentions using COPY command but lacks details on permissions, error handling, or side effects. Returns a status message but no specifics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise with Args and Returns sections, front-loading the purpose. No unnecessary sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, no annotations, and 3 required params, the description is incomplete. It doesn't mention table existence requirements, data format constraints, or behavior on conflict.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description adds minimal meaning: 'vertica schema', 'Target table name', 'List of rows to insert'. The 'data' parameter lacks structure details beyond list of rows, leaving ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it copies data into a Vertica table using COPY command, which is a specific verb and resource. It is easily distinguishable from siblings like execute_query or list_indexes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives like execute_query for single inserts. No prerequisites or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stream_queryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description should fully disclose behavioral traits. It mentions batching and progress reporting but omits critical details like whether the tool modifies data, safety concerns, or authentication requirements. The description does not warn about potentially destructive SQL operations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a single-purpose summary and structured Args/Returns. No extraneous text, though it could be more informative without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (SQL execution with optional batching, no output schema) and zero annotations, the description is incomplete. It fails to explain return value format, error handling, or when to adjust batch_size. The agent lacks sufficient context to use it safely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must add meaning. It provides minimal descriptions for 'query' and 'batch_size' (e.g., 'SQL query to execute') but lacks constraints, valid ranges, or examples. The batch_size default and meaning are implied but not elaborated.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Execute a SQL query') and the unique output format ('return the results in batches as a single string'). This distinguishes it from siblings like execute_query, which likely returns results differently.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives (e.g., execute_query). There is no mention of prerequisites, limitations, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_indexesB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must disclose behavioral traits. It states returns are a string but omits details on performance, side effects, permissions, or error handling, leaving the agent under-informed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and includes an Args/Returns structure, making it reasonably scannable. It could be slightly more concise by merging the header, but overall it is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 parameters and no output schema, the description is minimal. It lacks details on the return format, edge cases, or database prerequisites, making it only barely adequate for operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description adds meaningful parameter descriptions ('Name of the table to inspect' and 'Schema name (default: public)'), compensating for the lack of schema descriptions. This adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List all indexes for a specific table,' specifying the verb and resource. However, it does not differentiate from sibling tools like get_table_structure, which might have similar functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternative tools. The description only states what the tool does, without any when-not or alternative references.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_table_structureB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It mentions returning 'Table structure information as a string' but does not describe the format, side effects (e.g., read-only), error behavior, or performance characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively concise, with the purpose stated in the first line. However, the Args/Returns block adds some verbosity without significant extra value, and could be streamlined.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and a potentially complex return value, the description should elaborate on the format of the returned string (e.g., JSON, plain text). It also lacks error handling details and examples.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description adds value beyond the schema. It defines table_name and schema with a default, but does not provide examples, constraints (e.g., required format for table_name), or clarification of schema parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves table structure including columns, data types, and constraints. It uses a specific verb (Get) and resource (table structure), and is distinct from sibling tools like execute_query or list_views.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives. No mention of prerequisites, cases where other tools are preferred, or scope limitations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_viewsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It mentions returning a string but does not reveal side effects, authentication needs, or performance implications. The included 'ctx' parameter is not in the input schema, causing inconsistency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and uses a structured 'Args/Returns' format. However, it includes redundant technical formatting and a parameter (ctx) that does not appear in the schema, which could confuse an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Missing important details: the return format of the string, progress reporting behavior (despite mentioning ctx), and how to handle large results. The discrepancy between description params and schema further reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% (low), so the description must compensate. The description provides meaning for the 'schema' parameter ('Schema name (default: public)'), adding semantic value beyond the schema's default and type. However, it includes an undocumented 'ctx' parameter, slightly reducing clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'List all views in a schema,' clearly identifying the resource (views) and action (list). It distinguishes from sibling tools like list_indexes and get_table_structure by specifying it is for views.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. It does not mention prerequisites, exclusions, or comparison with siblings. The description only states what it does, not the context for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.