Server Quality Checklist

Tool Scores

• computeA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It accurately indicates a computational tool via examples, but does not describe potential limitations, output formatting behaviors, or error handling. It does not contradict any annotations, but lacks richer behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused paragraph that front-loads the core purpose and then provides illustrative examples. While somewhat long due to the many examples, each example adds practical value for understanding supported syntax, so it earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description covers the purpose and input syntax well but omits details about return values, output structure (unless using 'format' parameter), and situational guidance relative to siblings. It is adequate for invoking the tool but leaves some gaps in full context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description enriches the 'problem' parameter with detailed CAS-style examples (e.g., 'solve(x^2-4=0, x)', 'det([[1,2],[3,4]])'), which adds meaningful guidance beyond the schema. Other parameters are well-documented in the schema itself.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool solves math problems across many domains and provides explicit CAS-style examples. It distinguishes itself from siblings primarily through its focus on computation, but does not explicitly contrast with 'plot' or 'verify'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for math problem-solving through examples, but it does not provide explicit when-to-use or when-not-to-use guidance. It does not mention alternatives or contexts where 'verify' or 'plot' might be more appropriate, leaving the agent to infer usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verifyA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It mentions that verification uses symbolic and/or numeric checks, adding some context, but does not describe limitations, error behavior, or what happens when a claim cannot be verified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the purpose statement, followed by clear, relevant examples. Every sentence adds value without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with strong schema coverage, the description adequately covers purpose and examples. However, without an output schema or annotations, a brief note about the verdict structure (beyond the format parameter) would improve completeness, though it is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% parameter coverage with detailed descriptions for claim, format, and method, including enums and examples. The description adds no significant parameter semantics beyond what the schema already conveys.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool verifies mathematical claims using symbolic and/or numeric checks. It specifies three distinct use cases (identity verification, solution checking, computation assertions) and distinguishes it from sibling tools compute and plot.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through its examples but does not explicitly state when to use verify over alternatives like compute or plot. No exclusions or alternative recommendations are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• plotA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it discloses the output type (SVG image), the presence of axes/grid/labels, and asymptote detection in an example. It does not mention error handling or limitations, but covers the key behavior of returning an image.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core statement, then a brief output description, followed by concise, well-chosen examples. Every sentence earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 9 parameters and no output schema, the description is relatively complete: it states the return type, mentions asymptote detection, and shows usage patterns. It lacks explicit details on expression syntax limitations, but the examples plus full schema coverage make it sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value with examples that illustrate usage of x_min/x_max and expression syntax, going beyond the schema's individual parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool plots a mathematical function as an SVG graph, with a specific verb and resource. It distinguishes itself from sibling tools (verify, compute) by focusing on graphical output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context through examples, showing typical usage like 'plot sin(x) from -2*pi to 2*pi'. It does not explicitly exclude alternatives or name when not to use, but the examples and focus on graphing imply its intended use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.