Server Quality Checklist

Tool Scores

• ask_ezstatA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It discloses that the tool returns a grounded answer plus underlying data, which is useful. However, it doesn't mention that this is a read-only operation, any potential latency, or possible limitations of the natural-language model. The 'agent-read path' hint implies read-only but it is not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, and includes a concrete example. Every sentence earns its place with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with a clear description, the definition covers the core behavior, gives examples, and states the return value. It lacks some context such as authentication requirements or error conditions, but these are not critical for an agent to select and invoke the tool correctly in most cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already contains a well-described query parameter with examples, and schema coverage is 100%. The description reinforces the same meaning by giving examples but does not add meaning beyond what the schema provides. The baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Ask') and resource ('the account's metrics'), and clarifies the behavior: it returns a grounded answer plus the underlying data. It distinguishes itself from siblings like read_stat and list_stats by calling itself the 'agent-read path' for natural-language questions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use this tool: when the agent needs to ask a natural-language question about metrics. It provides examples of appropriate queries. However, it does not explicitly name alternative tools or state when NOT to use it (e.g., when you need a single numeric read via read_stat).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_statsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description carries the transparency burden. It conveys a read-only listing operation with no side effects, but does not disclose pagination, response format, or authorization requirements. The 'call this first' guidance adds useful context, but behavioral depth is limited.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. It states the action, scope, and usage guidance efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description covers purpose, usage, and basic output (names and types). It lacks an explicit return structure or pagination details, but these are not critical for a discovery tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter 'type', with a detailed description including enum meanings. The tool description adds no additional parameter semantics, so the baseline 3 applies when the schema already does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'List', the resource ('the account's metrics'), and the output content ('names and types'). The added instruction 'Call this first when you don't know what stats exist' further distinguishes it from sibling tools like read_stat or track_metric, positioning it as the discovery tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is given: 'Call this first when you don't know what stats exist in the account.' This provides a clear when-to-use. It does not explicitly name alternatives or when-not-to-use, but the scoping to names/types implies other tools are for values or tracking.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• track_metricA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden. It discloses key behavior: counters increment by +N (default +1) and values act as gauges. This adds useful context. However, it omits side effects like auto-creation of stats (only in schema) and does not explicitly state that this is a write/mutating operation or any persistence behavior, leaving some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. The first sentence front-loads the purpose and use cases; the second efficiently explains the counter/gauge semantics. It is appropriately sized and well structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 params, 1 required), the high schema coverage, and the absence of an output schema, the description adequately covers the core purpose and usage. It lacks only minor context like return behavior or explicit mention of timestamp, but these are not critical given the schema covers parameter details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description reinforces the counter/gauge distinction and how count vs value map to these modes, which adds a little meaning. But the schema already explains each parameter thoroughly, so the description does not significantly elevate clarity beyond it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Record a metric point your app produced.' It clearly states the tool's job and distinguishes it from sibling read/query tools (read_stat, list_stats, ask_ezstat) by focusing on recording vs retrieving metrics. Concrete examples (page view, checkout, latency) further clarify the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Use this whenever the surrounding code emits a counter or gauge.' It provides relevant examples and explains the counter vs gauge distinction. However, it does not explicitly name alternative tools or state when not to use it, so it stops short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_statA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the nature of the operation (read), the scope (single stat), and the exact contents of the response (latest value, recent series, count/min/max/avg/sum). It does not delve into error conditions or explicit read-only guarantees, but the term 'read' and the output summary provide sufficient transparency for a non-destructive tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, extremely concise, and front-loaded with the purpose and output. The second sentence adds usage guidance without redundant details. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, complete schema descriptions, and the description's explicit enumeration of return values, the information is sufficient for an agent to select and invoke the tool correctly. No output schema is present, but the description covers what the tool returns, so no gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description does not add parameter-specific semantics beyond the schema, but it does reinforce the 'single stat' concept that maps to the 'name' parameter. Since the schema already documents all parameters well, no additional compensation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific verb ('Read'), a specific resource ('a single stat'), and enumerates the delivered output ('latest value, recent series, and basic summary (count/min/max/avg/sum)'). It also distinguishes from the sibling tool by noting that 'ask_ezstat' is for free-form questions over many stats, so the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Use this for a structured read of one metric' and contrasts it with an alternative: 'prefer ask_ezstat for free-form questions over many stats.' This provides clear guidance and differentiates from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.