Server Quality Checklist

Tool Scores

• get_performance_summaryB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses format, interval, and data fields (requests, errors, 95th percentile RT), but critically omits how to specify the time range or what the default range is. This is a significant gap for a data retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loading the main action and output. While efficient, it could be slightly more compact without losing clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 optional param, no output schema), the description explains what is returned and the format. However, it fails to specify the time range of the data, leaving a gap in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter 's'. The tool description does not add any additional meaning or context to the parameter, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the verb 'Retrieve' and the resource 'time-series performance summary data', including format and interval. It distinguishes from siblings like 'get_performance_metrics' by emphasizing aggregated summary data, but does not explicitly differentiate from 'get_historical_data'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states 'for trend analysis and historical comparison', implying usage context, but lacks explicit guidance on when not to use or alternatives. No exclusion criteria are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_issuesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the output format (JSON) and scope (errors, exceptions, performance issues) but lacks details on pagination behavior, rate limits, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, front-loaded sentence with no redundant words. Efficiently conveys purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Lacks output schema. Description covers basic purpose and JSON return but does not explain return structure, pagination details, or example usage. Adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%; all parameters have descriptions. The description adds minimal extra meaning beyond 'issue types' and 'statuses' already covered. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves and analyzes errors, exceptions, and performance issues in JSON format. It uses specific verbs and resources, and distinguishes from sibling tools like get_performance_metrics and get_traces.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives such as get_traces or get_performance_metrics. Does not specify prerequisites or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_historical_dataA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must convey behavioral traits. It specifies JSON format and granularity options but does not disclose read-only nature explicitly, auth requirements, or error handling. The verb 'Retrieve' hints at idempotency but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences that are front-loaded: the first states core function, the second elaborates use cases. No superfluous text, every sentence is purposeful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 parameters and no output schema, the description covers the retrieval purpose but lacks detail on return structure (beyond JSON) and does not fully differentiate from sibling performance tools. Adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with parameter descriptions. The description adds 'JSON format' and mentions 'trends, transaction reports, time-series metrics', but the schema already details granularity options. The added value is moderate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves historical performance data for a specific date with configurable granularity, using specific verbs like 'Retrieve' and 'Analyze'. It distinguishes from siblings like get_issues and get_performance_summary by focusing on historical data with granularity options.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for analyzing historical trends but does not explicitly state when to use this tool versus alternatives like get_performance_summary. No exclusions or when-not guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tracesA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It only mentions JSON output and purpose, but lacks details on returned volume, pagination, or potential side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with purpose and usage, no superfluous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 11 parameters, no output schema, and no annotations, description covers purpose and usage well but lacks behavioral details like pagination or return format. Schema descriptions fill parameter details, but completeness is moderate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% description coverage, so description adds minimal extra value beyond schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool analyzes individual traces for bottleneck identification, and explicitly differentiates from get_performance_metrics for aggregate data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Directly states when to use (specific slow requests) and when not (system-wide statistics), naming the sibling alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_performance_metricsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It states the output is in JSON format and describes the nature of the data (aggregate, system-wide). It implies a read operation, but could explicitly say 'read-only' and mention any limitations like time range.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, both front-loaded: first sentence states purpose, second provides usage guidance. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description should explain the return structure more. It only says 'in JSON format' and 'aggregate performance metrics', which is vague. For a complete understanding, the agent would benefit from knowing what specific metrics are included (e.g., latency, error rate).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds no extra meaning to the parameters beyond what is already in the input schema. It does not explain how 'ts', 'm', 'env', 's' relate to the aggregate metrics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Retrieve' and the resource 'aggregate performance metrics and system-wide statistics'. It distinguishes the tool from its sibling 'get_traces' by mentioning it is for high-level overview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use for monitoring overall application health, trends, and high-level performance overview' and provides an alternative: 'use get_traces for detailed individual request analysis'. This gives clear guidance on when to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.