Server Quality Checklist

Tool Scores

• convert_temperatureB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions input modes but omits critical details such as which parameters are required per mode, whether file_content is base64 and injected, return format, or any edge cases. The phrase 'or direct input' conflicts with the schema's required file parameters, adding confusion rather than clarity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with front-loaded purpose ('Convert temperature units') and supporting detail about supported units. Every word earns its place—no filler, redundant phrasing, or verbose structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the schema being rich (6 parameters with 100% description coverage), the description creates a significant gap: it claims 'direct input' is possible while the schema marks file_name, file_type, and file_content as required. This misalignment would mislead an agent about how to invoke the tool correctly. No output schema exists, and the description does not explain return values or file-handling behavior, leaving the tool incomplete for correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameters are already well-documented in the schema. The description adds minimal semantic value beyond mentioning Celsius/Fahrenheit and 'file content or direct input', but it does not clarify parameter relationships or usage in direct-input mode, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Convert temperature units' and explicitly names the supported units (Celsius/Fahrenheit). This clearly distinguishes it from sibling tools like 'calculate' or 'get_commitments', though the phrase 'based on file content or direct input' introduces slight ambiguity about the exact input mode.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for temperature conversion with 'Convert temperature units' and 'Supports Celsius (C) and Fahrenheit (F)'. However, it does not explicitly state when to use this tool versus alternatives like 'calculate', nor does it clarify whether direct input or file content is preferred, and it provides no exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculateB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states 'Perform basic arithmetic calculations' without explaining error handling (e.g., division by zero), return format, or side-effect-free nature. This is insufficient for a clear behavioral profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundancy. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity and complete schema, the description is minimally adequate. However, with no output schema or annotations, it would benefit from stating the return value or edge cases like division by zero. The description is not entirely complete but is not severely lacking for a basic arithmetic tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage, with each parameter (a, b, operation) already documented. The description adds no extra semantic meaning, so baseline 3 is appropriate. It does not clarify nuances like numeric types or result precision.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('perform') and identifies the resource ('basic arithmetic calculations'), making it clear what the tool does. It distinguishes itself from sibling tools like convert_temperature and get_commitments, which are for different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention contexts, exclusions, or comparisons to sibling tools like convert_temperature or extract_commitments.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_commitmentsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure, but it only states the basic action. It fails to mention expected output format, assumptions about transcript structure, or potential limitations (e.g., language support, commitment definition), leaving significant behavioral unknowns.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that is immediately understandable. It contains no redundant information and is front-loaded with the action verb, making it highly scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema or annotations, yet the description does not explain what the extraction result looks like or how it should be interpreted. For a tool with a single input, this is a significant gap—the agent cannot anticipate the response structure or downstream usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'transcript' has 0% schema description coverage, and the tool description adds no extra context about the expected format, language, or length. It merely echoes the parameter name, providing no value beyond the schema's type declaration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('extract') and resource ('commitments from a meeting transcript'). It distinguishes itself from sibling tools like 'calculate' and 'convert_temperature' by specifying domain and output concept, and from 'get_commitments' by implying extraction from a transcript rather than retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a meeting transcript is available and commitments need to be identified. However, it does not explicitly state when to prefer this over 'get_commitments' or any other alternative, leaving the decision to the agent's inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_commitmentsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral disclosure. It does not explicitly state that the operation is read-only, nor does it mention authentication requirements, limitations, or side effects. 'Return' implies a read operation, but this is not sufficient for full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded with the action verb. Every word contributes to the meaning, with no unnecessary filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema), the description adequately conveys the core function. However, it lacks explicit details about the return format (e.g., list, array) or any potential caveats, so it is not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty. Per scoring baseline, this earns a 4 because there are no parameters to describe. The description adds nothing about parameters, but none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Returns all stored commitments', using a specific verb and resource. However, it does not explicitly differentiate this from sibling tool extract_commitments, which may also involve retrieving commitments.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The description only explains what it does, without mentioning context or exclusions, so agents receive no decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.