Server Quality Checklist

Tool Scores

• redash.get_queryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully convey behavioral traits. It warns that returned content is untrusted external data, which is helpful, but it does not explicitly state that the tool is read-only, causes no side effects, or lacks rate-limit constraints. The safety profile is undercommunicated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at two sentences. The first sentence front-loads the purpose, and the second adds a critical security warning. Every sentence earns its place without superfluous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two parameters and no output schema, the description should explain what metadata is returned and clarify parameter usage. It fails to do so, leaving gaps in the agent's understanding of inputs and outputs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, meaning the description adds no context about the two parameters (query_id and connection_id). It does not explain what connection_id refers to or how to obtain it, leaving the agent to guess from parameter names alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Read metadata for one allowlisted Redash query', specifying the verb (read), resource (metadata of query), and scope (one, allowlisted). It distinguishes itself from siblings like get_cached_query_results and fetch_query_results by focusing on metadata rather than results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives (e.g., for getting results instead of metadata). No explicit when-to-use or when-not-to-use instructions are given, leaving the agent to infer usage from the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redash.fetch_query_resultsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes polling behavior, fallback on 403, and untrusted data note. With no annotations, it partially covers behavioral traits but omits mutability, auth needs, and rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with core action. Efficient but could benefit from structured lists. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters, no output schema, and no annotations, the description is too brief. Lacks parameter behaviors, return format, error handling, and prerequisites.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description should clarify parameters. Only mentions 'parameters' in passing but offers no details on max_age, max_rows, etc. Fails to add meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The verb 'Fetch rows' with 'one allowlisted Redash query' clearly states the action and resource. Distinguishes from siblings like get_cached_query_results (cached) and get_query (definition).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context like optional refresh and fallback caching, but no explicit when-to-use vs alternatives or exclusions. Usage is implied but not clearly differentiated from sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redash.get_cached_query_resultsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that returned rows are 'untrusted external data' and that no query execution occurs. This is good, but it does not cover what happens if no cache exists or if the query is not allowlisted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise sentences with no unnecessary words. It is front-loaded with key action and purpose, and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description minimally covers return values (untrusted rows). It lacks details on error cases or what 'allowlisted' entails. For a simple read tool, it is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema coverage is 0%, and the description adds no parameter details. While parameter names like query_id, connection_id, and max_rows are moderately self-explanatory, the description could clarify that connection_id comes from list_connections and that max_rows has a maximum of 10000.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the verb 'Read', the resource 'cached rows for one allowlisted Redash query', and explicitly states the tool does not trigger query execution. It distinguishes itself from siblings by focusing on cached data without execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises using this tool for reading cached results without execution. It implies an alternative (likely fetch_query_results) for fresh execution but does not explicitly name it. The guideline is clear for the intended use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redash.list_connectionsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that URLs and API keys are never returned, an important security guarantee. No annotations exist, so description carries full burden; provides valuable behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences; first states primary purpose, second adds critical behavioral detail. No superfluous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple list tool with no input schema and no output schema. Tells agent it will get connection IDs and a security guarantee. Could mention if other fields like names are returned, but not necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters (baseline 4 since 0 params). Description adds meaning beyond empty schema by clarifying what is not in the output.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists connection IDs, and distinguishes from siblings by noting what is not returned (URLs, API keys). Siblings are query-focused, making this tool's purpose unique.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implicitly suggests use before tools requiring a connection ID; no explicit when-not or alternatives, but given the simple parameter-less nature, context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.