Server Quality Checklist

Tool Scores

• break_evenA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden of behavioral disclosure. It does explain the result interpretation (below/above threshold), but lacks details on underlying pricing assumptions, computation side effects, or any state changes. For a calculation tool, this is minimally adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that states the core function and its result interpretation with zero unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple calculation tool with no output schema, the description adequately conveys the purpose and the meaning of the result. However, it does not detail the response format or specify when to choose this over sibling tools, leaving a minor gap in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and each parameter has a clear description with defaults and ranges. The tool description adds no extra parameter-specific meaning, but the schema already handles this dimension, warranting the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Find' and clearly identifies the resource: 'daily usage hours at which API cost equals a subscription price'. It distinctly differentiates from siblings like compare_plans and recommend_plan by focusing on the break-even threshold calculation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus the listed siblings (compare_plans, run_scenario, recommend_plan). The description explains the threshold outcome but omits any exclusions or alternative tool references, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_scenarioA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns side-by-side costs for matching plans, which is useful behavioral context. However, it does not explain what determines 'matching' plans, assumptions, or any side effects, leaving moderate gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the verb and object. There is no filler or redundant information; it efficiently communicates purpose and output.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a five-parameter calculator with no output schema, the description covers purpose, usage pattern, and output shape ('side-by-side' costs). It could be more complete by clarifying differences from sibling tools, but the core scenario is adequately described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all five parameters are already documented in the input schema. The description adds no parameter-level information beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Calculate') and identifies the resource ('total monthly token usage and cost') as well as the output behavior ('Returns costs for all matching plans side-by-side'). It clearly conveys the tool's function, though it does not explicitly distinguish itself from sibling tools like compare_plans.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for a 'given usage pattern' and provides context about what it calculates. However, it offers no explicit guidance on when to use this tool versus siblings like compare_plans, break_even, or recommend_plan, and does not mention any exclusions or alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recommend_planA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It explains the tool considers subscription vs API cost and token caps, which is useful. However, it does not disclose output format, edge cases (e.g., no plan fits, budget exclusions), or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary purpose, and every clause adds value. The second sentence explains the decision logic without unnecessary detail. Ideal conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple, but with no output schema, the description does not clarify what the returned recommendation looks like (e.g., plan ID, estimated cost). It also does not reference sibling tools, leaving the decision context incomplete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not mention any parameters or add meaning beyond the schema, so it neither adds nor detracts from the schema's thorough documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('best-value plan recommendation'), clearly stating it returns a single recommendation based on usage profile. It distinguishes from siblings like compare_plans by emphasizing 'single' and 'best-value'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied: when you want a single recommendation for a usage profile. However, it does not explicitly contrast with sibling tools (e.g., when to use compare_plans) or state exclusions, so no when-not guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_plansA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description itself must convey behavior. It states the comparison is based on effective cost and that all plans are ranked, but doesn't disclose sort order, whether it returns a list with cost details, or any read-only/external-call behavior. Reasonably transparent for a non-mutating comparison but leaves some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no filler. The core action is front-loaded, and the use case is stated in the second sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With only 2 optional flat params and no output schema, the description provides the essential action and use case. It could mention the return format or exact sort order, but the tool's purpose is simple and the schema covers parameters well – not quite a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters with descriptions, so baseline is 3. The tool description adds 'effective cost per 1M tokens' and 'for a given model,' which align with output_ratio and model_id but don't materially extend the schema's parameter docs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with 'Rank all TokenLens plans by effective cost per 1M tokens' – a specific verb (rank), resource (TokenLens plans), and metric. This clearly distinguishes it from siblings like recommend_plan or run_scenario.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Useful when choosing between subscriptions and API access for a given model,' giving a clear use case. However, it does not mention when to avoid it or point to specific sibling alternatives such as break_even or recommend_plan.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.