Server Quality Checklist

Tool Scores

• execute_sql_queryB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, and the description doesn't disclose any behavioral traits beyond the action itself. It doesn't mention whether the tool supports read-only queries, if there are permissions required, or any limitations on SQL statements. The description adds no transparency beyond the tool's name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence that is concise and front-loaded with the core action. There is no unnecessary filler or restatement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description is clear, it lacks context about return formats, error behaviors, or how the 'context' parameter affects execution. Since there is no output schema, an agent might not know what to expect in the results. However, the tool is straightforward enough that a lengthier description may not be required.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides full descriptions for both parameters, including an example for the 'query' parameter. The description offers no additional parameter semantics, so the baseline of 3 applies due to high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb ('Execute'), resource ('SQL query against Apache Druid'), and outcome ('return results'). This distinguishes it from sibling tools like list_datasources or test_connection, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention, for example, that this is for querying data while list_datasources is for exploring available datasources. The agent must infer use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• test_connectionB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. 'Test connection' implies a read-only check, but the description does not disclose what the test entails, whether it has side effects, or what it returns on success/failure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no filler, making it appropriately sized for a simple tool. It could be slightly more explanatory, but it is efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description does not explain return values or expected behavior. It is too minimal to fully prepare an agent to use the tool correctly, especially given the lack of structured context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema confirms an empty object. Baseline for 0 parameters is 4; the description does not need to compensate for missing parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Test' and identifies the resource 'connection to the Druid cluster' clearly. This distinguishes it from sibling tools like execute_sql_query or list_datasources which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided regarding when to use this tool versus alternatives. The description only states what it does, without any context on prerequisites, exclusions, or recommended usage patterns.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_datasource_metadataA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It implies a read-only operation via 'Get' and lists return content (schema, size, segments), but does not explicitly state side effects, permissions, or error behavior. The 'Get' verb provides some transparency, but it is not fully explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. It immediately communicates the purpose and key content, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and no output schema, so the description sufficiently covers the essentials by mentioning 'schema, size, and segments' as return values. It does not need to explain additional details like pagination or complex return structures, but could have added a note about error conditions or auth requirements for completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully defines the single 'datasource' parameter with a description, so the schema coverage is 100%. The tool description adds no extra semantic detail about the parameter beyond what the schema already provides, thus the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('datasource'), clearly stating the action and what is included ('schema, size, and segments'). It distinguishes itself from sibling tools like list_datasources (listing vs. details) and execute_sql_query (query vs. metadata).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes clear the tool is used for a specific datasource, which implicitly sets it apart from list_datasources. However, it lacks explicit 'when to use' or 'when not to use' guidance or named alternatives, so it earns 4 rather than 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_datasourcesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description is the sole source of behavioral information. It conveys that the tool is a read-only operation returning a list, but it does not disclose details such as potential errors, completeness guarantees, or whether the list contains only metadata or full objects. It is neither misleading nor rich, so a midpoint score is warranted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant words. It states the action and the target resource concisely, earning a perfect score for brevity and focus.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (zero parameters, no output schema, no annotations), the description provides adequate context for an agent to know what the tool does. It could be more explicit about the return format (e.g., names vs. full metadata), but it remains complete for a basic listing operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts zero parameters, so the schema is empty and the description adds all necessary context. According to the rubric, the baseline for 0 parameters is 4. The description clarifies that no input is needed to list all datasources, which is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('list of all available datasources in Druid'), making its purpose immediately clear. It distinguishes itself from sibling tools like execute_sql_query, get_datasource_metadata, and test_connection by virtue of focusing solely on listing datasources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use the tool: when you need an overview of available datasources. While it does not explicitly exclude alternatives or mention sibling tools, the context is unambiguous. A score of 4 is appropriate because the guidance is implicit but strong; a 5 would require explicit 'when not to use' or named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.