Server Quality Checklist

Tool Scores

• get_table_detailsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It only states 'retrieves', hinting at read-only, but doesn't disclose auth needs, performance impact, or other behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence is concise, but it is too brief to be fully informative. It could include more detail without being verbose, such as stating it's a read operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists, so return values are covered. However, the description lacks context on when to use, parameter details, and behavioral traits, making it incomplete for an agent's decision.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has one parameter (table_name) with 0% description coverage. The description does not explain the parameter format, constraints, or valid values, adding no value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves table details and lists specific metrics (TotalRowCount, HotExtentSize). It distinguishes from sibling tools like get_table_schema (schema) and sample_table_data (samples).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives. Usage is implied by the description of retrieving row count and extent size, but no conditions or exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_table_schemaB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must fully disclose behavior. It states the action but does not mention read-only safety, error handling (e.g., if table does not exist), or any prerequisites. For a read operation, minimal but still insufficient given zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence that is concise and directly states the purpose. No wasted words, but could be slightly more structured (e.g., separating purpose and details). Still effective and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given presence of output schema, description need not explain return values. For a simple one-parameter tool, the description covers the core function but misses usage guidelines and parameter details. Minimal viable but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has one parameter (table_name) with 0% description coverage. The tool description does not elaborate on table_name format, expected values, or constraints. With low schema coverage, the description should compensate but fails to add meaning beyond the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'retrieves', the resource 'schema information for a specified table', and includes specifics like 'column names, data types, and other schema-related metadata'. It easily distinguishes from siblings like 'execute_query' (runs queries) and 'list_tables' (lists tables).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. For example, it does not explain that this is for schema metadata only, while 'get_table_details' might include more. Lacks any explicit context or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_queryB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must disclose behavioral traits. It only states that the tool returns a list of dictionaries but does not mention whether queries can modify data, rate limits, or pagination behavior. The lack of safety disclaimers is a gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately conveys the core action and result format. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (one parameter) and the presence of an output schema (not shown but indicated), the description covers the essential action. However, it lacks usage guidance and behavioral transparency, making it only minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'query' has no schema description (0% coverage). The description adds the phrase 'Kusto Query Language (KQL)' which clarifies the language but does not explain expected syntax, format, or examples. The value added beyond the schema is minimal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as executing a KQL query against Azure Data Explorer, with a specific verb ('executes') and resource ('KQL query'). It distinguishes itself from siblings like get_table_details by being the only tool that runs arbitrary queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like list_tables or sample_table_data. There is no mention of prerequisites, safety considerations, or typical use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sample_table_dataB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior but only mentions sampling and sample_size default. It does not state whether the operation is read-only, the nature of randomness, or implications for large tables. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no redundant words. Efficiently communicates core function and key parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description need not detail return values, but lacks context like the source database name (implied in description), column selection, or ordering. Sufficient for a simple sampling tool but minimal.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% so description must compensate. It explains sample_size (default 10) but provides no meaning for table_name beyond its existence. Half the parameters are undocumented in meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves a random sample of rows from a specified table in Azure Data Explorer. It uses a specific verb and resource, and implicitly distinguishes from sibling tools like list_tables or execute_query by specifying sampling behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when a random sample is needed, but provides no explicit guidance on when to use this tool versus alternatives (e.g., execute_query for custom queries). No when-not or context about prerequisites is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It indicates a read operation (retrieves) without mentioning side effects, auth needs, or rate limits. For a simple list operation, this is minimally adequate but lacks explicit disclosure of read-only behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with verb 'Retrieves'. Every word adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complexity is low (empty schema, 0 parameters). Description covers what the tool returns (table names, folders, database associations). Presence of an output schema reduces burden, and the description aligns well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has no parameters (100% coverage), so baseline is 3. The description adds no parameter details, which is acceptable since there are none.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool retrieves a list of all tables in the Azure Data Explorer database, specifying the returned fields (names, folders, database associations). This distinguishes it from sibling tools like get_table_details or get_table_schema which focus on individual tables or schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The description does not mention when to use list_tables instead of execute_query, get_table_details, etc. Context about use cases or exclusions is absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.