Server Quality Checklist

Tool Scores

• analyze_sales_trendA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the safety profile is known. The description adds useful behavioral context by detailing the return structure (revenue, order count, growth percentage), which helps the agent understand what to expect. However, it does not disclose limitations like date range handling or edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no extraneous information. Each sentence earns its place: first states purpose, second specifies outputs. Well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists and annotations cover read-only, the description is largely complete. It names the key outputs. However, it could mention that the tool compares periods or handles time ranges implicitly, but for a simple read tool, this is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameters are already well-documented in the schema. The description does not add new semantic meaning beyond the schema; it only restates the tool's focus on growth. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly specifies the verb 'analyze' and the resource 'sales trends' with a focus on period-over-period growth rates. It lists specific output metrics (revenue, order count, growth percentage) and distinguishes from siblings like get_revenue_report or get_top_products by emphasizing trend analysis and growth rates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The description does not mention conditions, prerequisites, or explicitly state scenarios where another sibling tool might be more appropriate. The sibling list exists but is not leveraged in the description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_regional_performanceA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, so the description's mention of 'ranked' adds minor context. It does not discuss other behavioral traits like rate limits or data freshness, but it aligns with the read-only nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: the first front-loads the action and key metrics, the second specifies the scope (all 5 regions, ranking by revenue). No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless, read-only tool with an output schema, the description covers purpose and result structure. It could mention the default sort order explicitly, but the current detail is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, and schema description coverage is 100%. The description compensates by explaining the tool's output (ranked performance across regions) without needing parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'ranked sales performance by region', listing specific metrics (revenue, orders, customers, AOV). It distinguishes from siblings like 'get_revenue_report' by specifying ranking across all five regions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives such as 'analyze_sales_trend' or 'get_revenue_report'. The description only states what it does, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_top_productsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the description does not need to restate that. It adds context about ranking by metric and optional category filter, but does not disclose ordering direction or other behavioral details beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the primary action, and no extraneous information. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 3 parameters fully described in the schema, an output schema, and annotations, the description covers the essential purpose and optional filtering. It could mention descending order or pagination, but the overall completeness is high given the existing structured data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Parameter schema coverage is 100% with descriptions. The description adds value by linking the 'metric' parameter to 'revenue or quantity sold' and the 'category' parameter to optional filtering, which clarifies the semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get a leaderboard of top-performing products by revenue or quantity sold' – a specific verb and resource. It distinguishes from siblings like get_revenue_report and analyze_sales_trend by focusing on product ranking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving top products but provides no explicit guidance on when to use this tool versus alternatives, nor any exclusion criteria or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_databaseA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation readOnlyHint=true already indicates the tool is read-only. The description adds no further behavioral details such as authentication requirements, rate limits, or consequences. It simply repeats the read-only nature without additional context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core purpose. It efficiently provides table names and examples in a structured format without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and annotations, the description is sufficiently complete for a read-only query tool. It could be slightly improved by mentioning query timeout limits or that DDL statements are blocked.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema documents the 'sql' parameter with a description. The description adds value by listing available tables and providing examples, which helps the agent understand query patterns and constraints beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs read-only SQL queries against the enterprise database. It specifies the verb 'run a read-only SQL query' and the resource 'enterprise database', distinguishing it from sibling tools that focus on specific reports or analyses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for ad-hoc SQL queries but does not explicitly state when to use this tool versus alternatives like 'get_revenue_report' or 'analyze_sales_trend'. It lacks guidance on when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_customer_profileA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so no contradiction. The description adds valuable behavioral context by listing the types of data returned (profile, order history, spending stats, favorite category).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the main action, and contains no unnecessary words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (implied but not shown), the description does not need to detail return values. It covers purpose, usage, and core data aspects completely for a read-only customer profile tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description for customer_id. The tool description does not add additional parameter semantics, but the schema already handles it adequately, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a comprehensive 360-degree customer view including profile, order history, spending stats, and favorite category. This distinct purpose from siblings like 'get_regional_performance' or 'analyze_sales_trend' is evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly mentions use cases: account reviews, churn analysis, and upsell identification. While it does not specify when not to use or name alternatives, the provided contexts strongly guide appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_revenue_reportA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, and the description adds the output metrics (total revenue, order count, avg order value, estimated profit), providing behavioral context beyond the annotation without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences: first states the action and constraints, second states the output. No extraneous words, highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema, the description appropriately complements it by listing the metrics. All parameters are documented, no required params, and the description covers the tool's purpose and output. Complete for the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with each parameter having a description and default. The description adds no new information about parameters beyond what the schema provides, so baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates a revenue report grouped by a dimension within a date range, specifying the metrics returned. This distinguishes it from siblings like analyze_sales_trend (trend analysis) and get_regional_performance (region-specific).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly defines use for revenue reports grouped by dimension but lacks explicit when-to-use or when-to-avoid guidance compared to siblings. While the purpose is clear, no alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.