Server Quality Checklist

Tool Scores

• describe_tableC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves schema information but doesn't describe what that includes (e.g., column names, types, constraints), whether it's a read-only operation, potential errors (e.g., if the table doesn't exist), or any rate limits. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences, but the structure could be improved. The first sentence states the purpose clearly, but the second sentence is formatted as an 'Args:' section, which might be redundant with the input schema. It's front-loaded but includes unnecessary formatting that doesn't add value beyond the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a schema retrieval tool with no annotations, no output schema, and low parameter documentation, the description is incomplete. It doesn't explain what the output includes (e.g., JSON structure, error handling), making it hard for an agent to use effectively without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate for the lack of parameter documentation. It adds minimal meaning by specifying that 'table_name' is the 'name of the table to describe', but this is basic and doesn't provide details like format, constraints, or examples. For a single parameter with no schema documentation, this is inadequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('schema information for a specific table'), making it easy to understand what it does. However, it doesn't explicitly differentiate from sibling tools like 'list_tables' (which might list table names without schema details) or 'read_query' (which might execute queries rather than describe structure), missing full sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'list_tables' or 'export_table_query', nor does it specify prerequisites or contexts for use, leaving the agent to infer usage based on the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the basic action. It doesn't disclose behavioral traits like whether it's read-only, pagination, rate limits, or error handling, leaving significant gaps for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any unnecessary words or fluff. It is appropriately sized and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a simple but incomplete description, the tool lacks context on behavior, output format, or usage scenarios. This is inadequate for a tool that might return a list of tables without further details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so no parameter information is needed. The description appropriately doesn't discuss parameters, earning a high baseline score for this dimension.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('all tables in the IoTDB database'), providing a specific purpose. However, it doesn't explicitly differentiate from sibling tools like 'describe_table' or 'export_table_query', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as 'describe_table' or 'export_table_query'. The description only states what it does, without context for selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_queryB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the SQL dialect and time format, which is useful context. However, it doesn't disclose critical behavioral traits: whether this is read-only (though implied by SELECT), authentication requirements, rate limits, error handling, result format, or pagination. For a query execution tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by specific guidance and parameter details. The 'Args:' section is clear and adds necessary information. While efficient, it could be slightly more structured by separating usage guidelines from parameter documentation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (query execution with SQL), no annotations, no output schema, and 0% schema description coverage, the description is moderately complete. It covers the purpose, basic usage guidelines, and parameter semantics adequately. However, it lacks information about return values, error conditions, and behavioral constraints that would be needed for robust agent usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds substantial meaning beyond the input schema, which has 0% description coverage. It explains that 'query_sql' is 'The SQL query to execute' and provides crucial context about the TABLE dialect and ISO 8601 time format. With only one parameter and the schema providing no descriptions, the description effectively compensates by explaining the parameter's purpose and constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Execute a SELECT query on the IoTDB' - a specific verb (execute) and resource (SELECT query on IoTDB). It distinguishes itself from siblings like 'describe_table', 'export_table_query', and 'list_tables' by focusing on query execution rather than metadata or export operations. However, it doesn't explicitly differentiate itself from potential write operations or other query types beyond SELECT.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context: 'Please use table sql_dialect when generating SQL queries' and mentions the TABLE dialect and ISO 8601 time format. However, it doesn't explicitly state when to use this tool versus alternatives like 'export_table_query' (which might handle results differently) or provide clear exclusion criteria for non-SELECT queries. The guidance is helpful but incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_table_queryA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and discloses key behavioral traits: it describes the export process, file format options, filename generation behavior, and preview limitations (max 10 rows). However, it doesn't mention potential side effects like file creation impacts, authentication needs, rate limits, or error handling for invalid SQL.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, SQL Syntax, Returns) and front-loaded purpose statement. While comprehensive, some details like the full SQL syntax example might be excessive if the agent already understands SQL, but overall it's efficiently organized with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (query execution + file export), no annotations, no output schema, and 3 parameters, the description provides good coverage: purpose, parameters, SQL syntax, and return information. However, it could better address behavioral aspects like error conditions or file system impacts to be fully complete for this mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed semantic information for all 3 parameters: query_sql (SQL query with dialect and time format specifics), format (export format options with default), and filename (optional with auto-generation behavior). It adds substantial value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('execute a query and export the results') and resources ('to a CSV or Excel file'), distinguishing it from sibling tools like describe_table, list_tables, and read_query which don't mention export functionality. It explicitly combines query execution with file export, making its unique role evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (to execute queries and export results to files), but doesn't explicitly state when not to use it or name alternatives among sibling tools. It implies usage for file exports but lacks explicit guidance on choosing between this and read_query for non-export scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.