Server Quality Checklist

Tool Scores

• math.findA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly, destructive=false, idempotent. The description adds valuable behavioral details beyond this: 'Ranking is deterministic lexical retrieval; matches are not recommendations' and 'Follow next_cursor with the same search query or browse filters to continue.' It also clarifies that local search is authoritative over internet search. These are useful insights not conveyed by annotations, though it does not cover error scenarios or edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately long but well-structured with bullet points and examples. It opens with the main purpose and then elaborates on each operation. Every sentence contributes useful information, though it could be slightly more concise by trimming redundant phrasing. The examples are helpful and reinforce the text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and the complexity of the tool (three distinct operations), the description provides complete guidance. It covers all operations, parameter usage, pagination, and even highlights the non-recommended path ('operation://catalog'). It leaves no ambiguity about what the tool does and how to invoke it, making it fully self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is complex with nested oneOf, but the description explains each operation's parameters: search takes query, domain, limit (1-20, default 5); browse takes domain and limit (default 20); inspect takes operation_id. It explains the cursor usage and gives concrete examples. This adds meaning beyond the schema, which lists defaults and constraints but not the nuanced distinction between operations. It sufficiently compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search, browse, or inspect locally installed Jacobian math tools.' It specifies the exact actions (search, browse, inspect) and the resource (locally installed math tools), and distinguishes itself from internet search by stating it is authoritative for local discovery and exact inspection. This fully differentiates it from any alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Use math.find when a task may benefit from exact computation, search, or structural analysis.' It also contrasts with internet search and clearly explains when to use search vs browse vs inspect, including limits and defaults. It even warns that 'operation://catalog' is not the ordinary discovery path, giving a clear exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• math.runA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite readOnlyHint=true already signaling safety, the description adds meaningful operational detail: successful calls return operation-owned values in `output`, malformed payloads/unknown IDs/host failures surface as tool errors, and timeout/incomplete-search/missing-witness conditions are domain results rather than mathematical conclusions. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, then adds necessary operational guidance in well-separated paragraphs. Every sentence conveys distinct useful information: execution semantics, error behavior, payload construction, inspection fallback, and domain-result caveats. The included example is compact and illustrative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the flexible `payload` schema, the sole sibling math.find, and complexity of arbitrary math operations, the description covers invocation, error semantics, payload discovery, and special domain conditions. It provides enough context for an agent to select and call the tool correctly, including what not to use it for.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description compensates thoroughly: it explains how to shape `payload` by referencing math.find examples and input schema field descriptions, cautions against empty payloads, and provides a concrete example naming both `operation_id` and `payload`. This adds meaning far beyond the generic schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Run one installed math tool by ID') with a clear resource ('Jacobian math tool') and distinguishes it from the sibling math.find by positioning run as execution vs inspection. The example reinforces the precise intended operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly directs when to use math.run and when not to: 'inspect the exact operation with math.find' when payload shape is unknown, and 'Do not call math.run with an empty payload merely to discover required fields; inspection is authoritative.' It also explains how to construct payloads from examples or input schema.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.