Server Quality Checklist

Tool Scores

• convert_temperatureC
  2.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description states the tool works 'based on file content or direct input', but the schema requires file_name, file_type, and file_content for all invocations, making direct input impossible without dummy file data. This contradiction is misleading and fails to disclose the actual required behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, short sentence that gets to the point quickly. It could be slightly more explicit about the file/direct input behavior, but overall it is concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks necessary context about the relationship between the file parameters and the temperature conversion parameters, especially given the required fields. There is also no explanation of output format or behavior when both file and direct input are provided, leaving the tool's usage incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters have descriptions, so schema coverage is complete, meeting the baseline. However, the descriptions are minimal, and the file-related parameters are confusing because they are marked required even for 'direct input' and provide no clarity on their purpose or interaction with the value/from_unit/to_unit parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool converts temperature units between Celsius and Fahrenheit, identifying the specific verb and resource. However, the mention of 'file content or direct input' introduces some ambiguity about the exact mode of operation, slightly detracting from full clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the sibling tools like 'calculate' or 'upload-and-analyze'. The description only explains what the tool does, not the scenarios in which it should be preferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculateC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description bears full responsibility. It states it performs calculations but omits potential side effects like division-by-zero errors or whether the tool is read-only, leaving behavior ambiguous.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using only four words to convey the entire purpose. There is no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks essential context such as the return format, error handling, or precision of results. Given no output schema and no annotations, the tool is incomplete for an agent to fully understand behavior without additional inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema descriptions cover all parameters, giving a baseline of 3. The tool description itself adds no extra meaning beyond the schema, but the schema already defines 'a', 'b', and 'operation' sufficiently for basic arithmetic.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs basic arithmetic calculations, which is a specific action. It distinguishes itself from sibling tools like convert_temperature by being generic, though it lacks detail on exact operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as convert_temperature. There is no mention of typical use cases or conditions for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_commitmentsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure, but it only states the basic action. It fails to mention expected output format, assumptions about transcript structure, or potential limitations (e.g., language support, commitment definition), leaving significant behavioral unknowns.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that is immediately understandable. It contains no redundant information and is front-loaded with the action verb, making it highly scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema or annotations, yet the description does not explain what the extraction result looks like or how it should be interpreted. For a tool with a single input, this is a significant gap—the agent cannot anticipate the response structure or downstream usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'transcript' has 0% schema description coverage, and the tool description adds no extra context about the expected format, language, or length. It merely echoes the parameter name, providing no value beyond the schema's type declaration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('extract') and resource ('commitments from a meeting transcript'). It distinguishes itself from sibling tools like 'calculate' and 'convert_temperature' by specifying domain and output concept, and from 'get_commitments' by implying extraction from a transcript rather than retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a meeting transcript is available and commitments need to be identified. However, it does not explicitly state when to prefer this over 'get_commitments' or any other alternative, leaving the decision to the agent's inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_commitmentsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral disclosure. It does not explicitly state that the operation is read-only, nor does it mention authentication requirements, limitations, or side effects. 'Return' implies a read operation, but this is not sufficient for full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded with the action verb. Every word contributes to the meaning, with no unnecessary filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema), the description adequately conveys the core function. However, it lacks explicit details about the return format (e.g., list, array) or any potential caveats, so it is not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty. Per scoring baseline, this earns a 4 because there are no parameters to describe. The description adds nothing about parameters, but none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Returns all stored commitments', using a specific verb and resource. However, it does not explicitly differentiate this from sibling tool extract_commitments, which may also involve retrieving commitments.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The description only explains what it does, without mentioning context or exclusions, so agents receive no decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.