Server Quality Checklist

Tool Scores

• prom_targetsD
  1.8/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It only states what the tool does ('Get scrape target information') without any details on permissions, rate limits, response format, or side effects. This leaves the agent with insufficient information about how the tool behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with a single sentence 'Get scrape target information'. It's front-loaded and wastes no words, though this brevity contributes to its inadequacy in other dimensions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (inferred from sibling tools suggesting Prometheus monitoring context), no annotations, no output schema, and poor parameter documentation, the description is completely inadequate. It doesn't explain what 'scrape target information' includes, how results are returned, or any operational constraints.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter 'state' with enum values, but schema description coverage is 0%, meaning the schema doesn't explain what 'state' means. The description doesn't mention parameters at all, failing to compensate for the lack of schema documentation. This leaves the agent guessing about parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get scrape target information' restates the tool name 'prom_targets' in slightly different words, making it tautological. It specifies the verb 'Get' and resource 'scrape target information', but doesn't clarify what 'scrape target information' means or how it differs from sibling tools like prom_discover, prom_query, or prom_range.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention any context, prerequisites, or exclusions, and it doesn't reference sibling tools to help the agent choose appropriately among Prometheus-related functions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prom_metadataC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Get metric metadata' implies a read-only operation, but it doesn't specify whether this requires authentication, has rate limits, returns structured data, or involves any side effects. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at three words, with no wasted language. It's front-loaded and to the point, making it easy to parse quickly, though this conciseness comes at the cost of detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a simple parameter, the description is incomplete. It doesn't explain what 'metric metadata' entails, how results are returned, or any behavioral traits. For a tool in a set with siblings like prom_query, more context is needed to guide proper use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'metric' parameter documented as 'Metric name (optional)'. The description doesn't add any meaning beyond this, such as examples or format details. With high schema coverage, the baseline score of 3 is appropriate, as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get metric metadata' clearly states the action (get) and resource (metric metadata), which is better than a tautology. However, it doesn't specify what 'metric metadata' includes (labels, types, help text) or differentiate it from sibling tools like prom_discover or prom_query, making it somewhat vague.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like prom_discover or prom_query. It lacks any context about use cases, prerequisites, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prom_discoverC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. 'Discover all available metrics' implies a read-only operation but doesn't specify whether this returns a complete list, paginated results, cached data, or real-time discovery. It lacks details about performance characteristics, rate limits, or authentication requirements that would help an agent understand how to properly invoke it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise at four words. It's front-loaded with the core action ('Discover') and resource ('available metrics'), with zero wasted words. Every element earns its place in this minimal but complete statement of function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and multiple sibling tools, the description is insufficiently complete. It doesn't explain what 'available metrics' means in context, how results are structured, or how this differs from prom_metadata which might provide similar information. The agent lacks enough context to understand when and how to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema description coverage, so the baseline is 4. The description appropriately doesn't waste space discussing non-existent parameters, though it could theoretically mention if there are implicit filters or scoping considerations not captured in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Discover all available metrics' states a clear purpose (discovering metrics) but lacks specificity about what resource or system this applies to. It doesn't differentiate from sibling tools like prom_metadata or prom_query, leaving ambiguity about what distinguishes this discovery operation from those alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings. With tools like prom_metadata, prom_query, prom_range, and prom_targets available, there's no indication whether this is for initial exploration, comprehensive listing, or a specific type of metric discovery. The user must infer usage context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prom_queryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Execute') but doesn't describe what happens during execution—whether it's read-only, has side effects, requires authentication, has rate limits, or what the response format looks like. For a query execution tool, this leaves critical behavioral traits unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse. Every word earns its place, and there's no redundant or verbose language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of executing a PromQL query (which involves query language syntax, evaluation, and potential side effects), the description is incomplete. With no annotations and no output schema, it doesn't explain return values, error handling, or behavioral traits. For a tool with 3 parameters and no structured safety hints, this leaves significant gaps in understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters (query, time, includes) with descriptions. The tool description adds no additional parameter semantics beyond what's in the schema. This meets the baseline of 3, as the schema does the heavy lifting, but the description doesn't compensate with extra context like examples or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Execute') and resource ('PromQL instant query'), making the purpose immediately understandable. It distinguishes from sibling tools like prom_range (which likely handles range queries) and prom_discover/prom_metadata/prom_targets (which are likely for metadata discovery). However, it doesn't explicitly contrast with these siblings, keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like prom_range or prom_discover. It doesn't mention prerequisites, typical use cases, or exclusions. The agent must infer usage from the name and context alone, which is insufficient for optimal tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prom_rangeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but offers minimal information. It states what the tool does but doesn't describe what happens during execution - whether it's read-only, whether it requires authentication, potential rate limits, error conditions, or what the response format looks like. For a query execution tool with zero annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - just four words that directly state the tool's function. There's zero wasted language, and it's front-loaded with the essential information. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a query execution tool with 5 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what a 'range query' means versus other query types, what the tool returns, or any behavioral characteristics. The agent would need to guess about the output format and appropriate usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are documented in the schema. The description adds no additional parameter information beyond what's already in the schema properties. The baseline of 3 is appropriate when the schema does all the parameter documentation work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Execute') and resource ('PromQL range query'), making the purpose immediately understandable. However, it doesn't differentiate this tool from sibling tools like 'prom_query' - both appear to execute queries, so the distinction between 'range query' and regular query isn't explained.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance about when to use this tool versus alternatives. With sibling tools like 'prom_query' and 'prom_discover' available, there's no indication of when a range query is appropriate versus other query types or discovery operations. No prerequisites or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.