Server Quality Checklist

Tool Scores

• search_knowledge_baseA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the basic search function and does not mention whether the operation is read-only, any side effects, or limitations. This is a gap for a tool whose safety profile is unknown.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a single leading sentence followed by a clean Args section. Every word earns its place, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, but the description lacks details about the return format or the scope of the knowledge base. Since there is no output schema, a bit more context on what 'passages' look like or how results are ordered would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explicitly explains both parameters: 'query' as a natural-language search query and 'top_k' as the maximum number of results. This fully compensates for the 0% schema coverage, adding meaning beyond the raw types and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states exactly what the tool does: 'Search the bundled knowledge base for passages relevant to a query.' It uses a specific verb ('search') and target resource ('bundled knowledge base'), and it is clearly distinct from sibling tools like 'calculate' and 'text_stats'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies clear usage context: use this tool to query the knowledge base with natural language. While it does not explicitly list alternatives or exclusions, the context is self-evident and sufficient for a simple search tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• text_statsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It lists the computed outputs and the text input, but does not address edge cases (e.g., empty text), output structure, or side effects. For a pure computation tool, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence defining purpose, followed by a minimal argument description. It is front-loaded with the function's goal and avoids redundancy with the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and no output schema. The description covers the operation but omits details about the return format, which an agent would need to interpret the result. This is a notable gap, though the listed metrics hint at the output shape.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only provides the parameter name and type (string). The 'Args' section adds a plain-language explanation ('The text to analyze'), which clarifies the purpose of the single parameter. Since schema coverage is 0%, the description compensates effectively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Compute' and enumerates exact metrics (character count, word count, sentence count, reading time), making the tool's function unambiguous. It clearly distinguishes from siblings by domain (text analysis vs calculation/search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for analyzing text passages, which is clear enough given unrelated siblings. However, there is no explicit 'when not to use' or mention of alternative tools, so it stops short of full guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculateA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It states the tool evaluates expressions and returns results, and lists supported operators, but does not mention error handling (e.g., invalid expressions, division by zero) or confirm it is non-destructive. This is adequate but leaves some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (three sentences plus an Args line) and front-loaded with the core purpose. Every sentence earns its place: purpose, supported operators, usage guidance, and parameter explanation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description covers purpose, usage, parameter semantics, and supported operators. It could mention error cases, but overall it is sufficiently complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only the parameter name and type (string). The description adds meaningful semantics by explaining 'A numeric expression' and giving a concrete example '2 * (3 + 4) / 7'. This fully compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states precisely what the tool does: 'Evaluate a numeric arithmetic expression and return the result.' It uses a specific verb and resource, and the supported operators distinguish it from sibling tools like search_knowledge_base and text_stats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided: 'Use this for any arithmetic instead of computing it yourself — language models are unreliable at multi-digit math.' This clearly tells the agent when to use the tool and implies not to use it for non-arithmetic operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.