Server Quality Checklist

Tool Scores

• explain_recommendationC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It states the action (explain) and mentions preference inputs, but it does not describe whether the operation is read-only, what the return structure is, whether it requires a prior recommendation, or any side effects. The description provides minimal behavioral context beyond the action itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is easy to parse and front-loads the action. There is no redundancy or filler. However, it is so brief that it sacrifices required detail, but that is more a completeness issue than a conciseness one.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters, no annotations, and a non-trivial action (explaining a choice), the description is too sparse to provide sufficient context. It does not explain how to use the tool, what the inputs mean, or any behavioral expectations. While an output schema exists, the description still fails to give a complete picture for the agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, and the description does not compensate by explaining individual parameters. It only groups them as 'preference inputs,' which is vague and does not clarify the role of coffee_id (the required parameter) or the optional fields like mood, caffeine, prefer_milk, and temperature. This adds little semantic value over the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'Explain a coffee choice,' which is a specific verb and resource. It also mentions 'against the same preference inputs,' distinguishing it from sibling tools like recommend_coffee (which recommends) and list_coffee_menu (which lists). However, it does not explicitly name these siblings or further elaborate on the output format, so it slightly misses full differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as recommend_coffee or list_coffee_menu. There is no mention of intended use cases, prerequisites, or exclusions. The only hint is 'same preference inputs,' which vaguely suggests it relates to recommendation, but this is not explicitly actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recommend_coffeeA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior on its own. It only states the tool's purpose and parameter meanings, without describing the return format, error handling, or underlying logic. The output of the recommendation (e.g., a coffee name, a ranked list) is not disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence plus a compact argument list, with no redundant text. Each line adds specific parameter guidance, and the structure is scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with four optional parameters, but there is no output schema and no annotations. The description does not specify what the recommendation returns or any behavioral constraints, leaving a moderate completeness gap given the lack of structured context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description compensates by explaining each parameter's meaning and allowed values (e.g., 'True for milk drinks, False for black coffee, None for either', 'low, medium, or high' for caffeine). This adds significant value beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses 'Recommend a coffee from mood and simple preferences' – a specific verb+resource and clarifies the input basis. This distinguishes it from sibling tools like list_coffee_menu and explain_recommendation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a user has mood and preferences, but it does not explicitly address when to prefer this tool over list_coffee_menu or explain_recommendation. There is no 'when not to use' guidance, so it falls at implied usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_coffee_menuA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must carry behavioral disclosure. It indicates a read-only listing operation but provides no details on safety, permissions, or side effects. The simplicity of the tool reduces the need for extensive disclosure, but the lack of any explicit notes is a minor gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no filler words. It is concise and front-loaded, achieving maximum clarity with minimal length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, zero-parameter list tool with an output schema present, the description is complete. It accurately states what the tool returns without needing to explain return values, as the output schema covers those details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so the schema fully covers them (vacuously at 100%). The description adds minimal semantic value beyond the tool's name, but the baseline of 4 for zero parameters is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'list' and identifies the resource as 'all available coffee menu items', clearly distinguishing it from siblings that recommend or explain recommendations. It fully conveys the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for retrieving the full menu, but it does not explicitly specify when to use it versus the sibling tools. No exclusions or alternative guidance are provided, leaving the usage context implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.