Server Quality Checklist

Tool Scores

• add_expenseC
  2.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for disclosing behavioral traits. It only implies a mutation ('Add') but does not mention side effects, validation, failure modes, or any other behavior. This is insufficient for an agent to anticipate consequences.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence with no unnecessary words. It is appropriately front-loaded, stating the action and resource immediately. However, it is so minimal that it borders on underspecification rather than efficient conciseness, so a 4 is appropriate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters (3 required), zero schema coverage, no annotations, and no output schema, the description is severely incomplete. It fails to address parameter formats, required input details, return values, or any operational context. An agent has almost no information to call this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the input schema contains no descriptions for any of the 5 parameters. The description does not compensate at all—it fails to explain what 'date', 'amount', 'category', 'note', or 'subcategory' expect in terms of format, range, or semantics. An agent cannot infer parameter meaning from either the schema or the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Add') and resource ('expense entry'), making the core purpose clear. It is distinct from siblings like list_expenses, but it doesn't explicitly mention alternatives or differentiators, so it earns a 4 rather than a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives, no context on prerequisites, and no mention of scenarios where it should not be used. The one-sentence description provides zero usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• summarizeB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While the verb 'summarize' implicitly suggests a non-destructive read operation, the description does not explicitly state that the tool has no side effects, does not modify expenses, or what the response structure will be. The burden falls on the description and it only partially meets it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with zero wasted words. The verb, resource, grouping dimension, and date-range semantics are all delivered efficiently. Nothing extraneous.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description is adequate but has notable gaps. With no output schema present, the description does not explain what the summarization returns, and there is no usage guidance to distinguish it from list_expenses. It covers the essentials of what the tool does but leaves output format and selection criteria undocumented.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the schema provides no types or descriptions for any of the three parameters. The description does add some meaning — it reveals that the date range is inclusive and that category is the grouping dimension — which compensates partially. However, it does not clarify whether category is a filter or a pure grouping key, nor does it specify expected date formats, leaving real gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Summarize') with a resource ('expenses') and states the grouping dimension ('by category') and scope ('inclusive date range'). This distinguishes it from siblings like add_expense and list_expenses, since summarizing implies aggregation. It could be clearer about what the aggregation output looks like (totals per category vs. overall totals), so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus its siblings. The natural distinction from list_expenses (summarize aggregates, list enumerates) is left entirely implied, and there is no mention of exclusions, prerequisites, or alternatives. An agent must infer the appropriate use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_expensesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. 'List' implies a read operation but does not explicitly state it is read-only or side-effect-free. It also fails to describe behavior when filters are omitted (e.g., returns all expenses) or any limits/pagination. The inclusive date range note is useful but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence conveys the primary purpose and optional filtering without wasted words. It is appropriately compact and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two optional parameters and no output schema, the description gives the core purpose but lacks parameter format details, explicit read-only behavior, and return value expectations. It is minimally sufficient for basic use but leaves gaps that could cause incorrect calls.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% – the schema provides no parameter explanations. The description only clarifies that the date range is inclusive, but does not specify date format, semantics of null defaults, or behavior when a single bound is provided. This leaves significant ambiguity for an agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List') and resource ('expense entries'), and mentions optional date filtering. This clearly distinguishes it from add_expense (creation) and the unrelated dice/number/summary tools, so an agent can easily tell what it does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus siblings, such as when to list vs. add expenses. The agent must infer usage from the name alone; there is no explicit context, exclusions, or alternatives mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_numbersA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the operation but does not explicitly mention that it is pure, has no side effects, or what the return value is. While obvious for addition, it falls short of full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with zero filler. It is front-loaded and every word earns its place, exemplifying appropriate conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a trivial addition tool with a clear input schema and an output schema (as indicated by the signal), the description is largely sufficient. It lacks explicit mention of edge cases or preconditions, but none are expected for this operation, making it nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%—the description does not mention parameters by name or add any nuance. The schema already declares a and b as required numbers, but the description adds no value beyond that, failing to compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Add two numbers together' uses a specific verb and resource, clearly stating exactly what the tool does. It is distinct from siblings like roll_dice and add_expense, so agents can easily differentiate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives, nor any exclusions. There is no context about when addition is appropriate or not, leaving the agent to infer usage from the name and schema alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• roll_diceA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It correctly implies a random stochastic process and states the output ('return the results'), but does not explicitly mention randomness, the range of outcomes (1–6 per die), or that subsequent calls produce independent results. This is adequate for a simple tool but lacks any deeper context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the core action, and contains no filler words. Every term contributes to understanding the tool's behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (one parameter, simple output) and the presence of an output schema, the description is complete enough. It explains the parameter's role and the action's purpose. It could mention the non-deterministic nature more explicitly, but that is implicit for a dice-rolling tool, and the output schema likely covers return formatting.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero description coverage for the parameter n_dice, so the description must compensate. It does so by framing n_dice as the number of dice to roll, adding semantic meaning to an otherwise bare integer field. However, it does not specify constraints like minimum value or positivity, which would have improved clarity further.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear, specific action: 'Roll n_dice 6-sided dice and return the results.' The verb 'Roll' and resource 'dice' are unambiguous, and the tool is easily distinguished from siblings like add_numbers or list_expenses, which handle arithmetic or expense management.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus alternatives, nor any exclusions or prerequisites. Sibling tools are unrelated, but the description provides no context for selection, leaving the agent to infer its suitability from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.