Server Quality Checklist

Tool Scores

• query_metricsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits such as read-only status or side effects. It only mentions the instant vs. range query modes, but does not state whether the operation is non-destructive, what it returns, or any datasource implications. This leaves the agent to assume safety on a tool that executes arbitrary expressions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. It front-loads the purpose and then gives a succinct behavioral directive. Every word earns its place, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential query modes and ties to datasources, but omits any mention of return value structure or error handling, which is significant given there is no output schema. The schema documents parameters thoroughly, so the core usage is clear, but the lack of result details leaves a minor gap for such a complex query tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully documents all six parameters with descriptions, so the baseline is 3. The description adds only a rephrased note about omitting `from` for instant queries, which is already present in the schema's `from` field. It adds no new meaning for parameters like `stepSeconds` or `maxDataPoints`.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a precise action: running a PromQL or LogQL expression against a Grafana datasource. It clearly distinguishes this tool from the SQL-oriented sibling (query_sql) by explicitly naming the expression languages, leaving no ambiguity about which tool to use for metric or log queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs when to omit or include the `from` parameter to switch between instant and range queries, providing clear context for invocation. Though it doesn't explicitly name query_sql as the SQL alternative, the expression-language distinction effectively routes the agent to the correct tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_sqlA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It mentions read-only by convention and returning rows, which is helpful, but omits other traits like error handling, rate limits, or potential side effects beyond the stated convention. It adds some value but is not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences with no redundant words. The core purpose is front-loaded, and the usage note follows immediately, making it easy to scan and internalize.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a raw SQL tool with a straightforward purpose, the description covers purpose, prerequisites, and read-only behavior. It does not detail time-range semantics (covered in schema) or return size limits, but given the tool's nature and lack of output schema, it is sufficiently complete for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description does not add new parameter-level detail; it only reinforces the datasourceUid dependency on list_datasources, which is already in the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs raw SQL against SQL-family datasources and returns rows, using specific verbs (run) and resources (SQL datasource). It distinctively positions itself from siblings like query_metrics by emphasizing 'raw SQL' and explicit datasource family names.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear prerequisite ('Use list_datasources first to get the uid') and notes a read-only convention. However, it does not explicitly compare with alternatives such as query_metrics or state when not to use this tool, leaving selection to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_dashboardA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the read-only nature implicitly with 'Fetch' and specifies exactly what is returned, including panels, datasource, raw query languages, and template variables, which is sufficient for a simple read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that front-loads the core action ('Fetch a dashboard by uid') and then adds valuable details about the return payload and use case. It is efficient with no fluff, though slightly long with the parenthetical and appositive.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description covers the key aspects: input, output, and purpose. It says what the dashboard fetch returns, and the schema covers the parameter source, making it complete enough for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with 'uid' already described as 'Dashboard uid from search_dashboards'. The description mentions 'by uid' but adds no new meaning beyond the schema, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'Fetch a dashboard by uid' and the resource, then details the output: panels, datasource, raw queries, and template variables. It distinguishes itself from siblings like query_sql and search_dashboards by emphasizing query reuse rather than execution or searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'the fastest way to reuse a query someone already wrote' provides clear context for when to use this tool. It implicitly excludes query execution tools, though it does not explicitly name alternatives or state when not to use it, giving clear context without exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_dashboardsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It states the search functionality and implies read-only use ('find what is already instrumented'), but does not mention potential limitations, pagination, or return format. It adds only minimal behavioral detail beyond the tool name and schema, leaving the agent to infer common search semantics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences. The first sentence gives the core action and criteria, front-loading the most important information. The second sentence adds usage guidance with no fluff. Every word earns its place, achieving conciseness without losing substance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with all optional parameters and no output schema, the description is mostly complete. It explains the tool's purpose and provides a usage workflow, including the next step. It does not explicitly mention the return type (e.g., a list of dashboard references), but the follow-up 'then get_dashboard' implies that the results are dashboard identifiers, making the missing detail a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (all three parameters have descriptions), so the baseline is 3. The description adds usage context (e.g., 'find what is already instrumented') but does not add new semantic meaning beyond the schema's 'Title substring' and 'Filter by dashboard tag'. It reinforces but doesn't extend the parameter docs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Search'), resource ('dashboards'), and criteria ('by title or tag'), making the tool's purpose immediately clear. It also differentiates from get_dashboard by implying that search_dashboards finds dashboards while get_dashboard reads their content, and from query tools by focusing on discovery rather than data retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly specifies when to use the tool ('before writing a query from scratch') and provides a follow-up action ('then get_dashboard to read the panel queries'). This gives the agent a clear workflow and implies when not to use it (e.g., when querying data directly, use query_sql/query_metrics).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• healthA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It reveals what the tool inspects (auth layers, credential identity, datasource visibility), the meaning of the failingLayer value, and the remediation step. It does not explicitly state whether the check is read-only, but a health check's non-mutating nature is strongly implied and no contradiction exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, all substantive. The main purpose is front-loaded, the diagnostic details are compactly listed, and the remediation instruction is concrete. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter diagnostic tool with no output schema and no annotations, the description is complete: it states what is checked, how results are interpreted, and what action to take on a specific failure mode. It gives the agent enough to decide when to call it and what to do with its output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the input schema is empty, so there is nothing for the description to clarify about parameters. Per the zero-parameter baseline, a score of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Check Grafana connectivity and authentication.' It enumerates exact diagnostic outputs (failing auth layer, credential identity, visible datasource count), which distinguishes this health-check tool from sibling data-access tools like query_sql and list_datasources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit trigger: 'Call this first whenever another tool fails.' It also specifies a follow-up action when failingLayer=grafana. It does not spell out when not to use it, but the context makes the tool's diagnostic role clear relative to the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_datasourcesA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It correctly implies a read-only listing operation, but does not explicitly mention pagination, rate limits, or any side effects. For a simple list operation, this is adequate but not thorough; it lacks explicit confirmation of safety or response limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two compact sentences. The first states the core function and return fields; the second immediately provides actionable guidance on next steps. There is no redundancy or filler, and the critical scoping information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters, no output schema, and low complexity, the description fully covers what the agent needs: what it returns, and how to proceed. It is complete and self-sufficient for correct invocation and routing to sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is inherently complete. The description adds no parameter details because none exist. Per the rubric, a baseline of 4 is appropriate for zero parameters, and the description does not need to explain anything about parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: listing Grafana datasources with specific fields (uid, name, type). It also differentiates from sibling query tools by framing itself as the prerequisite step, so an agent can immediately understand what it does and how it differs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: it tells the agent to start here whenever querying is needed, explains that a datasource uid is required, and directs the agent to choose between query_sql and query_metrics based on the type. This clearly distinguishes when to use this tool versus the query siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.