Server Quality Checklist

Tool Scores

• ch_list_tablesA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden. It only mentions the default database behavior, but does not disclose read-only nature, error handling, or performance implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with one sentence and a brief parameter note. Every word is purposeful and there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with an output schema, the description adequately covers its core purpose and the parameter's default. It is complete enough for basic usage, though it could mention potential failure modes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions for its parameter, so the description must compensate. It clearly explains the 'database' parameter and the default behavior, adding meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all tables in a ClickHouse database, using the specific verb 'List' and resource 'tables in a database'. This distinguishes it from sibling tools like ch_list_databases and ch_describe_table.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives or in what workflow it fits. The description only states the function, leaving the agent without explicit selection instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• es_get_mappingA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the burden. It indicates a read-only inspection operation but does not disclose potential error cases (e.g., non-existent index) or response behavior. Since it is a simple get-mapping tool, the basic behavior is transparent, but not deeply disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one sentence stating the purpose plus a single-argument explanation. It is front-loaded and contains no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one simple parameter and an output schema, the description is complete. It covers what the tool does and what the index parameter means. Additional behavioral details are not necessary for this straightforward operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema description coverage is 0%, the description explicitly explains the only parameter: 'index: The index name to inspect.' This adds meaningful context beyond the schema's 'title: Index' and is sufficient given the single parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Show the field mapping (schema) for an Elasticsearch index.' It uses a specific verb (show) and resource (field mapping of an index), which distinguishes it from siblings like search_elasticsearch or es_list_indices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to inspect an index's schema but provides no explicit guidance on when to choose it over alternatives or any exclusions. This gives clear context but lacks directional advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ch_describe_tableA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It reveals the default database behavior, which is useful. However, it does not state whether the operation is read-only, what happens if the table is missing, or any other behavioral details. It is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and efficiently structured: a single purpose sentence followed by parameter details. No unnecessary words, and it is front-loaded with the main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, and the description covers purpose and parameters. An output schema exists, so return values need not be described. It lacks explicit mention of edge cases or read-only status, but for a straightforward describe tool it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no descriptions, so the description must compensate. It does: it explains 'table' as 'Table name' and 'database' as 'Database name' and adds the default behavior ('If omitted, uses the default database'). This adds meaning beyond the schema's bare titles and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Show' and clearly identifies the resource (schema of a ClickHouse table) and what it includes (columns, types, comments). This distinguishes it from siblings like ch_list_tables (lists tables) and query_clickhouse (runs queries).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose clearly implies when to use this tool: when you need the schema of a specific table. It does not explicitly mention alternatives or exclusions, but the context is clear and no competing tool is suggested by the description. This aligns with 'clear context, no exclusions'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ch_sample_dataA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of disclosing behavior. It adds that the tool returns up to 100 rows and that omitting database uses the default database, which goes beyond the schema. However, it does not disclose sampling method, determinism, or permission requirements, leaving behavioral transparency partial but not wholly absent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the primary purpose in the first sentence. The Args section is compact and avoids fluff, making it easy to scan. Every sentence contributes useful information without redundancy, achieving high information density.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity and the presence of an output schema, the description is largely complete. It covers purpose, usage context, and parameter semantics. It lacks a brief note on how the sample is selected or a pointer to alternative tools for non-sampling queries, but these are not critical gaps for a simple read-only sampling tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning for 'database' ('If omitted, uses the default database') and 'limit' ('Number of rows to return (1-100, default 10)'), which are not fully expressed in the schema. The 'table' parameter is merely restated as 'Table name,' which adds no semantic value beyond the schema, but overall the description meaningfully enriches parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get sample rows from a ClickHouse table (up to 100 rows).' It specifies a concrete verb ('Get'), a resource ('ClickHouse table'), and a scope ('sample rows'), which distinguishes it from sibling tools like ch_list_tables, ch_describe_table, and es_sample_data. The purpose is immediately clear and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'Useful for understanding telemetry data structure before writing queries.' This tells the agent when to invoke this tool, but it does not explicitly state when not to use it or mention alternatives like query_clickhouse for arbitrary querying. Because it gives a clear context without exclusions, it earns a 4 rather than a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ch_list_databasesA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. The verb 'List' implies a read-only, non-destructive operation, but the description does not explicitly disclose any behavioral traits such as lack of side effects, performance considerations, or return format. For a simple listing tool, this is minimally adequate but lacks extra context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that immediately states the tool's purpose with no filler. It is front-loaded and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters), the output schema exists (so return values are covered), and the description fully conveys the tool's scope and action. There are no gaps in understanding for an agent to invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description correctly makes no parameter claims, and the empty input schema matches. There is nothing more to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' with the resource 'databases' and scopes it to 'all databases accessible in the ClickHouse instance.' This clearly distinguishes it from sibling tools like ch_list_tables (which lists tables) and query_clickhouse (which runs queries). It leaves no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use this tool: whenever the agent needs to discover available databases in ClickHouse. It doesn't explicitly mention alternatives or exclusions, but the tool's name and description make it the obvious choice for listing databases, and there is no misleading context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• es_list_indicesA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It implies a read-only operation by using 'List', and it discloses output components (document counts and sizes). Yet it lacks details on potential side effects, performance implications, or whether special permissions are needed, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the main action and purpose. Every sentence contributes: the first specifies what it does, the second explains when to use it. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with zero parameters and an output schema present, the description is complete. It states the output (document counts and sizes) and gives a clear use case, so the agent has sufficient context to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema already provides complete coverage (100%). According to the rubric, a baseline of 4 is appropriate when there are 0 params, and the description adds no conflicting or redundant information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List all Elasticsearch indices') and additional specifics ('with document counts and sizes'). It distinguishes itself from sibling tools like es_get_mapping and es_sample_data by focusing on listing indices rather than retrieving mappings or sample data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides a clear use case: 'Use this to discover available download/package data indices.' This guides the agent toward discovery scenarios. However, it doesn't explicitly mention alternatives or when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• es_sample_dataA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the burden. It does disclose the 100-document limit and implies a read operation ('Get'), but it does not explicitly state safety, permissions, or error behavior. The read-only nature is reasonably inferable, but more detail would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a short summary line, a usage note, and an Args list. Every sentence adds value with no filler or redundancy. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple sampling tool, the description covers the essential aspects: purpose, document limit, and parameter semantics. An output schema exists (though not shown), so return values are handled structurally. It lacks error handling or prerequisites, but these are not critical for this tool type.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description's Args section fully explains both parameters: 'index: The index name' and 'size: Number of documents to return (1-100, default 10).' It adds the constraint range and default, which the schema lacks. This fully compensates for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get sample documents from an Elasticsearch index (up to 100).' This is a specific verb+resource combination that distinguishes it from siblings like search_elasticsearch (searching) and es_list_indices (listing indices). The mention of 'download data structure' adds context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: 'Useful for understanding download data structure before writing queries.' This implies a use case (pre-query exploration) but does not explicitly name alternatives or when not to use them. It offers guidance without exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_elasticsearchA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It notes supports for Elasticsearch query features (match, term, range, bool, aggregations) and a default size of 500, which adds useful context. However, it does not mention read-only nature, error handling, rate limits, or what happens if the query is invalid, leaving some uncertainty.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a one-line definition, a brief use-case paragraph, and clearly labeled Args with examples. Every sentence adds value, and the format makes it easy to scan. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is complex (arbitrary Elasticsearch DSL), yet the description covers use cases, parameters, and examples. An output schema is present, so return values need not be described. The inclusion of example queries and aggregations makes the description sufficiently complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no descriptions (0% coverage), but the description fully compensates by explaining each parameter: index, query_body, and size, including the effective default (500). Examples demonstrate valid query_body structures, providing meaningful guidance beyond the schema's bare type declarations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs Elasticsearch queries (JSON DSL) against download/package data, with a specific verb ('Run') and resource. It also lists concrete use cases (downloads by product, package type, etc.), which distinguishes it from sibling tools like query_clickhouse or es_list_indices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this for product download data' and lists example uses, providing clear context for when to choose this tool. It does not explicitly state when not to use it or name alternatives, but the sibling tool names (e.g., query_clickhouse) imply the boundary. Lacks explicit exclusions, so it falls shy of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_clickhouseA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the read-only nature, allowed statement types (SELECT, SHOW, DESCRIBE, EXPLAIN), and the 500-row cap. This transparently sets expectations about safe usage and limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by constraints and use cases. Every sentence serves a purpose: allowed statements, row cap, telemetry use cases, parameter description, and examples. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one parameter and an output schema (not shown but present), the description covers purpose, constraints, use cases, and parameter semantics. It does not need to explain return values thanks to the output schema. It is sufficiently complete for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It defines sql as "A read-only SQL statement" and provides three concrete examples showing valid query forms, which gives the agent clear guidance beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource: "Run a read-only SQL query against ClickHouse (telemetry data)." It clearly distinguishes from sibling tools by being the general ad-hoc SQL interface, while siblings like ch_list_tables and ch_sample_data cover specific narrower operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear use cases: "Use this for product telemetry: active instances, version distribution, storage engines, deployment types, CPU architecture, cluster metrics." It does not explicitly contrast with alternatives (e.g., when to use ch_list_tables instead), but the context makes those alternatives obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.