Server Quality Checklist

Tool Scores

• add_employeeC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral details like permissions, side effects, or idempotency. It merely says 'Adds a new employee,' which reveals nothing about consequences or requirements, similar to the low-scoring update_drive example.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that avoids bloat, but it is under-specified. It earns its place as a basic purpose statement, yet it lacks the substance needed for a fully effective tool description, so it's not particularly well-rounded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With three required parameters, no annotations, and no output schema, the description is woefully incomplete. It provides no information about input expectations, return values, or edge cases, leaving the agent without critical context for correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, and the description does not mention any of the three required parameters (name, department, salary). It fails to compensate for the lack of schema descriptions, leaving parameter meaning completely unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool adds a new employee, with a specific verb and resource, which distinguishes it from the sibling tools that list, search, or compute. However, it offers no additional specifics beyond the action, so it's not a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus the alternatives. It does not mention any exclusions, prerequisites, or contexts where another sibling would be more appropriate, leaving the agent without decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• employees_by_departmentB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations to rely on, so the description carries the full burden of behavioral disclosure. It only states the basic read operation but does not disclose order, pagination, error behavior, permissions, or whether the result is limited to active employees. Minimal behavioral context is provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly worded sentence that directly states the function. There is no redundant information or fluff. For a simple tool, this is appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description should provide more context. It does not cover usage guidelines, parameter semantics, or return details. While the tool is simple, the absence of any guidance on how it relates to sibling tools or handles varying inputs makes it incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, and the tool description provides only minimal compensation. It implies that the 'department' parameter is the department to filter by, but does not specify whether it is an ID, name, code, or what values are accepted. No format or edge-case behavior is explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Returns') and resource ('employees') with a scope ('from one department'). This clearly distinguishes it from siblings like list_employees (which would return all) and search_employee (which likely searches by other criteria).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus the sibling tools. The description does not mention alternatives, prerequisites, or any context that would help an agent decide between 'employees_by_department', 'list_employees', and 'search_employee'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_employeeB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only states the basic action without disclosing behaviors such as handling multiple matches, not-found cases, or output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that is front-loaded with the action and target. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It does not explain what the search returns, whether matches are exact or partial, or what happens if no employee is found.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'name' is self-explanatory, and the description reiterates it without adding extra semantics like case sensitivity, partial matching, or format requirements.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb 'Find' and specifies the resource 'employee' with the search criterion 'name'. It clearly distinguishes from siblings like list_employees (list all) and employees_by_department (filter by department).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied: this tool is for finding an employee by name. However, there is no explicit mention of when to use it over alternatives, or any exclusions or conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_employeesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the basic return scope ('all employees') but does not mention potential issues like pagination, ordering, authentication, or whether the operation is read-only. The lack of any side-effect or safety information is a notable gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that conveys the essential purpose without any filler. Every word earns its place, making it highly concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with no parameters and no output schema, the description is minimally adequate. However, it does not explain the structure of the returned employee objects or any sorting behavior, which could matter in practice. Given the lack of annotations and output schema, a bit more detail would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema provides no parameter details. The description adds meaning by clarifying that the tool returns 'all' employees, implying no filtering or pagination controls. This is useful context beyond the empty schema, justifying the baseline of 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Returns all employees' uses a specific verb ('Returns') and resource ('employees') with an explicit scope ('all'). This clearly distinguishes it from sibling tools like search_employee (filtered search) and highest_salary (aggregate).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. While the sibling names suggest usage contexts, the description itself does not mention any exclusions or alternatives, leaving the AI agent without explicit decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• highest_salaryA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It merely restates the tool's name with no details about tie-breaking, ordering, return format, or edge cases. This adds no insight beyond what the name already conveys.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, concise and to the point, with no unnecessary words. It conveys the essential information efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no params, no output schema), the description is minimally adequate. However, it does not clarify tie-handling, empty-result behavior, or the structure of the returned employee object, which could be important in edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the input schema is fully covered by default. The description adds no parameter information, but the baseline for zero parameters is 4, which is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Returns highest paid employee' clearly states the action (returns) and the target resource (highest paid employee), distinguishing it from siblings like list_employees, search_employee, employees_by_department, and add_employee.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose implies when to use this tool (when you need the top-paid employee), but there's no explicit guidance on when to prefer it over alternatives like list_employees or employees_by_department. The usage context is clear from the description, but exclusions or alternatives are not mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.