Server Quality Checklist

Tool Scores

• math.batchA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover read-only, idempotent, and non-destructive behavior. The description adds genuinely useful behavioral context beyond that: operations are independent, executed in order, and subject to per-item limits. It does not describe partial-failure behavior, but that gap is mitigated by the output schema and annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence contains only high-signal details: count range, independence, ordering, and per-item limits. There is no fluff or repetition of schema defaults.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Together with a detailed input schema, an output schema, and read-only/idempotent annotations, the description is nearly complete for selecting and invoking the tool. The main omissions are explicit routing to math.run for single operations and any note about how item failures affect the batch, but neither blocks correct basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67% and the schema carries detailed descriptions for timeoutMs, maxOutputBytes, and resultMode. The description contributes only the general notion of per-item limits and the 1-32 item count, which mostly restates schema constraints. This matches the high-coverage baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action ('Run'), a clear resource ('1 to 32 independent operations'), and an ordering constraint ('in order'), so an agent can tell it is a batch-execution tool. It does not explicitly name math.run or another sibling to differentiate, so it misses the top score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'independent operations' and the 1-32 range imply this tool is for multiple independent computations, while the sibling math.run suggests single-operation use. However, the description never explicitly says when to prefer this over math.run or warns against dependent operations, leaving the guidance implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• math.searchA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive behavior, so the description doesn't need to repeat those. It adds the routing rule but no additional behavioral detail like result scope or pagination.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one focused sentence with no filler, and the most important routing rule is front-loaded. Every word contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity, rich annotations, and existing output schema, the description covers tool selection well. However, the complete absence of parameter guidance leaves a clear gap for an agent constructing a query.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain query or category semantics, syntax, or expected values. The parameter names provide some meaning, but the description fails to compensate for the missing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly names the action ('Search') and resource ('operations'), and the clause 'only when the id is unknown' distinguishes this tool from math.run. It is specific and non-tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit condition for use—only when the id is unknown—and directly names the alternative, math.run, for the known-id case. This is clear when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• math.describeA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish read-only, idempotent, non-destructive behavior. The description adds meaningful behavioral context beyond that: it clarifies the tool returns only schema and example arguments, does not execute operations, and that the returned examples are argument objects intended for later use with math.run. This complements the annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no wasted words. The first sentence states the core purpose and its read-only nature, the second gives an exclusion condition, and the third explains how to consume the output. Each sentence earns its place and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with a rich enum, full schema coverage, output schema, and comprehensive annotations, this description is complete. It covers what the tool returns, how to select the parameter, when not to use it, and how to feed the result into math.run. Nothing essential is missing for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the schema already describes the 'operation' parameter with a full enum and a clear meaning. The description adds contextual usage guidance—that the operation should come from math.search and be unfamiliar—but does not significantly expand the parameter's semantic definition. A baseline of 3 is appropriate when the schema carries the weight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Get schema and argument examples only for one unfamiliar operation selected by math.search.' It clearly distinguishes this from the sibling math.search and math.run, and even names specific operations it should not be used for, making the tool's scope immediately identifiable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use the tool ('for one unfamiliar operation selected by math.search'), when not to use it ('Do not call this for known integer.machine_arithmetic or combinatorics.count shapes'), and how to use its output ('nest one under math.run.arguments and pass its id as math.run.operation'). This is direct, actionable guidance that leaves little to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• math.runA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds meaningful behavioral details beyond the annotations: unknown fields are rejected before execution, exact and approximate results are kept separate, and a single ordinary call is sufficient, which helps the agent avoid unnecessary retries or fallback behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but well organized: use cases first, exclusions second, mandatory calling convention third, and high-value examples last. Every sentence earns its place, and the most important structural instruction is front-loaded rather than buried.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 44 operations, nested argument objects, and an output schema, the description is strong: it covers selection criteria, invocation shape, when to call describe, and exact/approximate result behavior. It could say slightly more about how math.run relates to math.batch and math.search, and 'one successful ordinary call is sufficient' is a bit ambiguous, but overall it is complete enough for reliable agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 67%, so the description needs to add some value, and it does. It explains the critical arguments-object contract ('{operation, arguments}; never flatten'), warns that unknown fields are rejected, and gives concrete argument shapes for integer.machine_arithmetic and combinatorics.count. It does not enumerate every operation's parameters, but it appropriately defers to math.describe for unfamiliar shapes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: run a mathematical operation, with a clearly scoped purpose ('exact or reliability-sensitive mathematics'). It names the domains it covers, explicitly excludes trivial low-risk arithmetic, and distinguishes itself from math.describe by noting which operations need no describe call.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit when-to-use guidance ('Use for exact or reliability-sensitive mathematics...'), when-not-to-use guidance ('Do not use for trivial low-risk arithmetic'), and directs the agent to math.describe when operation arguments are unfamiliar. It also gives a concrete structural rule for passing arguments that prevents a common invocation error.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.