Server Quality Checklist

Tool Scores

• execute_queryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden, yet it fails to disclose critical behavioral traits: whether the tool supports write operations (INSERT/UPDATE/DELETE), read-only safety, result format/size limits, or execution timeouts. 'Return results' is vague given the high-stakes nature of arbitrary SQL execution.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with action verb, no redundancy. However, given the tool's high complexity and lack of annotations/output schema, the brevity crosses from concise to under-specified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Arbitrary SQL execution is high-complexity with significant safety implications. With no annotations, no output schema, and minimal description, the definition lacks essential context about data modification risks, result structure, and operational scope required for safe agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage with clear parameter description. The description adds context by specifying 'StarRocks' as the target system (not in schema), but adds no detail on SQL syntax requirements, supported dialect features, or parameter binding beyond the schema's basic declaration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear specific verb ('Execute') and resource ('SQL query') against StarRocks. However, it does not distinguish from sibling metadata tools (describe_table, list_tables), which also retrieve data but via specific patterns rather than arbitrary SQL.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides no guidance on when to use this versus the specific metadata tools (list_tables, describe_table) or when to prefer built-in tools over raw SQL. No mention of prerequisites or query complexity considerations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'List' implies read-only behavior, the description does not confirm safety, disclose return format (e.g., array of table names vs objects), mention pagination limits, or note the default 'ads' database behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely terse at only 5 words, but it efficiently conveys the core operation without redundancy. However, it is so minimal that it fails to front-load any distinguishing characteristics or behavioral constraints that would help tool selection.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description should disclose what the user receives (e.g., 'returns table names') and highlight the default database behavior. As written, it leaves significant gaps in the agent's understanding of the tool's contract.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% description coverage, establishing a baseline of 3. The tool description adds no semantic context about the 'database' parameter (e.g., not mentioning that it defaults to 'ads'), but no additional documentation is required given the complete schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (List), resource (tables), and scope (in a database). However, it does not explicitly differentiate from sibling tools like 'describe_table' or 'list_databases', which would help the agent choose between listing metadata versus detailed schema information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'describe_table' (which gets details for one table) or 'list_databases' (which scopes higher). No prerequisites or conditions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_tableB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, yet it only states the basic action. It does not clarify that this is a safe read-only operation, what format the schema information returns in, or how errors (e.g., table not found) are handled.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at five words, with the action verb front-loaded. There is no redundancy or wasteful text—every word contributes to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter introspection tool with complete schema coverage, the description is minimally adequate. However, given the lack of annotations and output schema, it could have benefited from mentioning the specific information returned (data types, constraints, indexes).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, meaning the schema already documents both 'table' and 'database' parameters including the default value. The description adds no additional parameter context, but the baseline score of 3 is appropriate given the complete schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action ('Get') and resource ('table schema and column information'), accurately describing the tool's function. However, it lacks explicit differentiation from sibling tools like 'list_tables', which could confuse agents about whether this returns metadata or just names.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives (e.g., when to use 'describe_table' vs 'execute_query' with a DESCRIBE statement). There are no prerequisites, workflow hints, or explicit exclusions mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_databasesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full disclosure burden. While 'List' implies read-only behavior and 'all' defines scope, it omits details about return format, pagination limits, or required permissions to view all databases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Five words, zero redundancy. Front-loaded with action verb ('List') followed immediately by target resource. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no annotations, the description adequately covers the tool's function. Minor gap: without an output schema, it could briefly mention what data is returned (database names vs metadata), but the scope is sufficiently clear for a simple listing operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has zero parameters. Per rubric rules, zero parameters establishes a baseline score of 4. The description correctly implies no filtering parameters are accepted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description provides a specific verb ('List'), clear resource ('databases'), and scope ('all...in StarRocks'). It clearly distinguishes from sibling 'list_tables' (databases vs tables) and 'describe_table'/'execute_query' (listing vs describing/executing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states what the tool does but provides no explicit guidance on when to use this versus siblings like 'list_tables' (e.g., 'use this first to find database names before listing tables') or any prerequisites needed to access database metadata.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.