Server Quality Checklist

Tool Scores

• metrics.searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'fuzzy search' but doesn't explain what that entails (e.g., partial matches, case sensitivity), nor does it cover aspects like permissions, rate limits, or response format. This leaves significant gaps in understanding how the tool behaves beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's front-loaded with the core action and resources, making it easy to parse quickly, with no wasted information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't address behavioral traits like search behavior, error handling, or result format, which are crucial for a search tool. The high schema coverage helps with parameters, but overall context for effective use is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear documentation for 'q' (search query) and 'limit' (maximum results with default). The description adds minimal value by implying search over 'name, description, or tags', but this doesn't provide syntax or format details beyond what the schema already covers, aligning with the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Performs fuzzy search') and target resources ('over metrics by name, description, or tags'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like metrics.list (which might list all metrics without search) or metrics.get (which might retrieve a specific metric by ID), so it doesn't reach the highest clarity level.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like metrics.list or metrics.get. It implies usage for searching metrics but doesn't specify scenarios, exclusions, or comparisons to siblings, leaving the agent to infer context without explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• metrics.listB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states it returns a list but doesn't mention any behavioral traits such as pagination, rate limits, permissions required, or what 'basic information' entails. This leaves significant gaps for a tool that likely interacts with data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without any wasted words. It is front-loaded and appropriately sized for a simple list operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'basic information' includes, the format of the returned list, or any behavioral aspects like error handling. For a tool with no structured support, this leaves too much unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100%, so there's no need for parameter details in the description. The baseline for this scenario is 4, as the description appropriately avoids redundant information about parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Returns a list') and resource ('all metrics with basic information'), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'metrics.search' which might also list metrics, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'metrics.search' or 'metrics.get'. It lacks context about use cases, prerequisites, or exclusions, leaving the agent with minimal direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• metrics.getB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states it returns details without disclosing behavioral traits like error handling, permissions needed, rate limits, or response format. It's a read operation but lacks context on what 'full details' includes or any constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste, front-loaded with the core action. It's appropriately sized for a simple tool with one parameter and clear purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a simple tool, the description is incomplete. It doesn't explain what 'full details' returns, error cases, or behavioral context, leaving gaps for the agent to understand the tool fully beyond basic usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'name' parameter fully. The description adds no additional meaning beyond implying exact name matching, which is already covered. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Returns full details') and resource ('of a metric'), specifying it's by exact name. It distinguishes from siblings like metrics.list (which lists multiple) and metrics.search (which searches), though not explicitly named. It's specific but lacks explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need details for a specific metric by exact name, suggesting an alternative to list/search for bulk operations. However, it doesn't explicitly state when to use this vs. siblings or any exclusions, leaving some ambiguity for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• metrics.refreshNowB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 're-parses' and 'reloads cache' which imply mutation/refresh operations, but doesn't specify side effects (e.g., whether this blocks other operations), permission requirements, error conditions, or what 'reloads' entails. Significant behavioral gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states what the tool does without any unnecessary words. It's appropriately sized for a zero-parameter tool and front-loads the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has zero parameters and no output schema, the description adequately covers the basic action. However, as a mutation/refresh tool with no annotations, it should ideally mention more about behavioral aspects like side effects, timing implications, or error handling to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema description coverage, so the schema already fully documents the parameter situation. The description appropriately doesn't waste space discussing parameters, maintaining focus on the tool's action. Baseline for zero parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Re-parses', 'reloads') and target resource ('Markdown file', 'in-memory cache'), making the purpose understandable. However, it doesn't explicitly differentiate this tool from its siblings (metrics.get, metrics.list, metrics.search), which appear to be read operations while this seems to be a refresh/cache operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites (e.g., after file changes), exclusions, or comparisons with sibling tools, leaving the agent with no contextual usage information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.