Server Quality Checklist

Tool Scores

• get_top_productsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are present, the description carries the full behavioral disclosure burden. It delivers key behavioral details: the ranking basis is units sold, revenue per product is included, and cancelled orders are excluded—these are non-obvious and useful. It does not mention where the list is the entire pop-Nand any caveats, but for a simple read-list the core is covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences: the first fixes the core behavior and the second adds the usage and the exclusion nuance. Everything present earns its place, and important info is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  After reading the description, an agent understands what the tool returns, what is excluded, and when to call it. The main missing piece—then number of results limit—is the same parameter gap. There is no output schema, but the description defines what a row contains (units sold and revenue), so an agent has enough to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema description coverage at 0%, the description needed to compensate by explaining the single limit parameter—it doesn't mention limit at all. An agent must rely on the property name and the numeric constraints to infer that limit probably controls how many products are returned. That is a significant gap for the only parameter in the tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb and resource ('Get the best-selling products') and clarifies the meaning: ranked by units sold, with revenue per product. It also states a non-obvious scope detail—cancelled orders are excluded—which makes the purpose unambiguous. This clearly separates the tool from siblings such as get_top_customers or list_tables.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this tool for 'best-selling products' questions, and calls out the alternative: writing the order_items→products join yourself. This gives an agent both a positive trigger and a reason not to hand-write the query. It does not name the sibling get_top_customers or list exclusion conditions, which keeps it from a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_tableA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description absorization all burden for behavioral disclosure. It clearly states that the operation is a read-style introspection (get definitions, row count, sample rows) and does not suggest any modification of side effects. It leaves the exact number of sample rows unspecified, but that's minor for a describe type action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense sentences deliver the needed information without fluff. The first sentence says what it returns; the second positions it in the overall workflow. Every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose, output content, and workflow context. It doesn't describe how many sample rows are returned or whether row count is approximate, but for a one-parameter introspection tool with an enum parameter, it's sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is exactly one required table parameter with an enum; the schema already enumerates valid tables, but the description's coverage is 0%. The description says 'one table' and implies the table name comes from list_tables, adding some process context, but does not itself define the parameter values or formats. The enum carries most semantic weight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' with a distinct resource: column definitions, row count, and sample rows for one table. It clearly distinguishes itself from list_tables and query by naming when each is used, and says exactly what information to expect from the tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides an explicit workflow: call this after list_tables and before using the query tool. This conveys the intended portion in the schema-discovery flow. It does not explicitly state when not to use it, but the context and alternat structure make the situation reasonably clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_revenue_reportA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description correctly discloses the key behavioral details: canceled orders are excluded, and the `year` parameter acts as an optional filter that also applies when grouping by month. It doesn't discuss auth or rate limits, but the main data-behavior message is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three compact sentences deliver the essential info with meaningful detail; the first sentence is the core action, the second clarifies data inclusion/exclusions, and the third tells the expected alternative. No filler or repeated schema text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-parameter reporting tool with no output schema and no annotations, the description covers purpose, grouping semantics, optional filter, and usage context. Omitting the exact return format is minor because the tool is narrow and the input semantics are fully explained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schemas has zero description coverage, so the description adequately compensates: it explains the `year` parameter in context and the three allowed grouping values in `group_by`. It doesn't need to explain the enum bodies since the schema already enumerates them, and adding meaning to parameter usage is done well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: it gets total revenue grouped by product category, year, or month, which is distinct from sibling tools like get_top_customers and get_top_products. It clearly tells the agent what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this instead of writing an orders→order_items→products join yourself, which gives direct when-to-use guidance relative to the query-type workflow. It doesn't rule out all sibling tools, but this report is different enough that no exclusion is critical.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It communicates that the tool only lists tables and descriptions, and it states the nature of the output. This is sufficiently transparent for a read-only discovery tool, though it doesn't mention potential permissions or data volume considerations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: the first states the action and output content, and the second provides usage direction. No wasted words, and the key functionality is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, no-output-schema tool, the description is fully sufficient. It tells the agent what it returns (table names, descriptions, row counts) and why to call it first. No critical missing information affects correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so parameter semantics are trivial. The description adds appropriate context about what the returned list represents, fulfilling the baseline guidance for a no-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List', the resource 'tables in the shop database', and specifies the output content (short description and row count). It distinguishes itself from the sibling query/specific-report tools by serving as a discovery entry point.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells the agent to use this tool first before writing queries or calling other tools, giving clear when-to-use guidance. It does not name specific alternative tools or state when not to use it, but the instruction to use it before others is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_top_customersA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the behavioral burden. It discloses the important ordering rule that cancelled orders are excluded from the calculations, which materially affects results. It does not spell out the return shape, but 'get' strongly implies a read-only list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, each earning its place: the operation, the non-obvious data restriction, and the usage routing. No filler, no tautology, and intent is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given only two parameters and no output schema, the description covers the main decision points: what the ranking means, how to choose the ordering, and which sibling tool to prefer. Omitted details like pagination or exact response fields are minor for this simple, well-scoped reader.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has0% description coverage, but the description maps 'spending' to total amount spent and 'orders' to order count, giving meaning to the enum. The `limit` parameter's purpose is also inferable from its default/minimum/maximum, so the critical required parameter is fully interpreted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies both the verb+resource ('Get the top customers') and the two exact rankings that define the tool's behavior ('total amount spent' or 'number of orders placed'). This clearly separates it from generic querying and from other top_N tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly tells the agent when to use this tool instead of the generic `query` tool: for 'who spent the most' / 'who ordered the most' questions. This is direct, practical routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queryA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, and it does so well: it states the operation is read-only, only single SELECT or WITH...SELECT statements are accepted, write operations and multiple statements are rejected, and results are capped with pagination support. No major behavioral surprise remains.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the core purpose, and each sentence adds meaningful guidance. No fluff or repeated schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an open-ended SQL query tool, this is complete: it identifies what tool does, what queries are permitted, what happens on violation, and how to handle large result sets. There is no output schema to document, and the sibling tools supply the schema-discovery path.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clarifies that sql must be a SELECT-style query, that a session limit caps returned rows, and that offset pages through additional results. It does not spell out defaults or maximums, but those constraints are already visible in the JSON schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Run a read-only SQL SELECT query against the shop database.' It also distinguishes itself from the specialized analytics tools by positioning itself as the general-purpose query tool for questions those tools do not directly cover.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear when-to-use guidance: use it for anything not covered by specialized analytics tools and for free exploration/joins once the schema is known. It stops short of explicitly naming siblings like list_tables or get_revenue_report, but the exclusion of 'specialized analytics tools' is sufficient guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.