Server Quality Checklist

Tool Scores

• approve_leaveC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It does disclose the key side effect (deducting days from the balance), which is valuable. However, it doesn't state whether approval is reversible, what happens if the balance is insufficient, whether approver permissions are required, or what the response looks like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single tight sentence that efficiently delivers the core purpose and the key side effect. No wasted words. However, given the number of undocumented parameters, slightly more length for parameter clarification could be justified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutating action (approving with balance deduction) with no annotations, no output schema, and 0% parameter coverage, the description is insufficient. It should clarify prerequisite state (pending only), approver authorization requirements, irreversible nature, and param semantics. The description covers the core behavior but leaves significant operational gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and no parameters are documented in the description. The description implies request_id is the leave request being approved and approver_id is the authorized approver, but add no explicit parameter-level semantics. The note parameter is entirely unexplained. With three parameters and zero coverage, the description should compensate but largely doesn't.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (approve), resource (pending leave request), and a key behavioral consequence (deducts days from balance). This distinguishes it from siblings like reject_leave and cancel_leave, though it doesn't explicitly name those alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says this targets 'pending' leave requests, which provides some context on when to use it, but offers no explicit guidance on when not to use it or how it differs from reject_leave/cancel_leave. No alternative tools are named. For a tool with a sibling set including reject and cancel, better differentiation would be expected.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reject_leaveB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses one key behavioral trait: 'The balance is left unchanged,' which is valuable because users might expect rejection to restore or deduct leave balance. However, with no annotations provided, the description carries full burden, and it doesn't disclose other side effects like whether notifications are sent, whether approver_id has authority requirements, or what happens to associated workflow state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, both earn their place. The first states the action, the second adds the balance side-effect. No wasted words. It could arguably benefit from parameter clarification, but what's present is tight and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having only 3 parameters and no nested objects, the tool has 0% schema description coverage and no annotations, forcing the description to carry full weight. For a mutation tool that rejects requests, it should disclose authority requirements, side effects on request status, and what the note parameter does. The description is incomplete for the operational context an agent needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description provides no parameter details at all. None of the three parameters (request_id, approver_id, note) are explained beyond their schema titles. The description does not clarify that request_id must reference a pending request, what authority approver_id must have, or the meaning/format of the optional note field.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb+resource: 'Reject a pending leave request.' This distinguishes it from siblings like approve_leave and cancel_leave. However, it doesn't reference the sibling distinctions explicitly or note any limitations about which leave types can be rejected, keeping it just shy of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when a leave request is pending and should be rejected, contrasting implicitly with approve_leave. However, it provides no explicit when/when-not guidance, no mention that this applies only to pending requests (it says 'pending' but doesn't clarify that approved/rejected requests can't be re-rejected), and no alternatives named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_pending_requestsB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It implies a read-only operation (listing requests), which is consistent with a read operation, but doesn't explicitly state that no changes are made. It also doesn't disclose what happens for edge cases like managers with no direct reports (empty list?) or whether the manager_id must be unique. The read-only nature is implied but not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that conveys the core purpose efficiently with no wasted words. It's appropriately short for a tool with only one parameter. However, it's slightly under-specified given the complete lack of parameter documentation, so it could arguably contain a bit more useful detail without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with one parameter and no output schema, the description captures the core purpose adequately. The key gap is the parameter semantics — with 0% schema coverage and only one param, the description should clarify what manager_id refers to and its expected format. However, given the tool is inherently simple (list of requests), it doesn't need extensive elaboration.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the description must compensate. The single parameter 'manager_id' is described only as 'Manager Id' in the schema. The description says requests are for 'employees reporting to a manager,' which partially clarifies that manager_id identifies the manager, but it doesn't specify the format (name? email? UUID?), whether it refers to the current user's ID, or any validation requirements.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list), resource (pending leave requests), and scope (employees reporting to a manager). It's distinguishable from sibling tools like apply_for_leave, approve_leave, reject_leave since those are action-oriented, while this is a read/list operation. However, it doesn't explicitly differentiate it from check_leave_balance which is also a read operation, though the resource differs (balance vs requests).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is for managers to see pending requests from their direct reports, which provides some context for when to use it. However, there's no explicit when-not-to-use guidance or mention of alternatives among the siblings (e.g., no mention of how this differs from check_leave_balance for viewing leave status). The scope is implied to be 'pending' only, not approved or rejected requests.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• apply_for_leaveA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that working days are computed automatically and checked against remaining balance, and that a request is returned as a dictionary. However, it doesn't disclose what happens on failure (e.g., insufficient balance error), whether this is a mutation with side effects, or auth/precondition requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the core purpose first, then supplementary details (date format, auto-computation, return value). Efficient and readable with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 5-param tool with no annotations, no enums, and no output schema, it explains the core workflow: date format, auto-computation of days, balance validation, and return shape. It covers the essential behavioral contract well. A minor gap: no guidance on error/failure outcomes, but the tool is otherwise adequately described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate but provides no per-parameter details. It does add the key semantic that dates use ISO format (YYYY-MM-DD) and that leave_type/working-days interplay is auto-computed, which helps. But leave_type values and reason usage are undocumented; reason has a default and is optional but that's schema-visible.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb+resource ('Submit a leave request for an employee') and distinguishes the action from siblings (approve_leave, reject_leave, cancel_leave are all management actions; this is the initiating submission). It's clear but doesn't explicitly differentiate from siblings in text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating it submits a leave request and mentions balance checking against working days. However, it doesn't explicitly state when to use this vs alternatives, nor when NOT to use it (e.g., leave already pending). With siblings like approve/reject/cancel, some usage contrast would help.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_leaveA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the side-effect of credit-back for approved requests, which is useful. However, it does not mention the reverse direction, prerequisites (e.g., can all requests be canceled?), or what happens to pending vs rejected requests besides approved ones.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no filler. The core purpose and a key behavioral nuance (credit-back) are communicated efficiently. Slightly more detail on edge cases could have been added without hurting length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter cancellation tool with no output schema, the description is reasonably complete. It covers the main action and the most consequential side-effect (balance credit-back for approved requests). Missing: what happens to already-rejected requests, whether cancellations require permissions, and any constraints on when cancellation is allowed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter exists (request_id), which is self-explanatory as an integer identifier given the tool name. Schema coverage is 0%, but with a single obvious parameter, the schema plus name largely convey meaning. The description adds no additional param detail, but very little is genuinely needed here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource: 'Cancel a leave request.' The description explicitly distinguishes the operation and adds behavioral context (credit-back on approved requests). It differentiates from siblings by targeting a specific request state (approved vs pending) effect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states what happens when canceling an approved request (days credited back), giving implied context on when this is appropriate. However, it does not explicitly say when to use this vs reject_leave or other alternatives, nor provide exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_leave_balanceA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description indicates a read operation (returns balance), which implies a non-mutating behavior distinct from siblings like apply/approve/reject/cancel. However, it doesn't disclose details like whether employees with no leave type assigned return zero or are omitted, what happens for invalid employee IDs, or whether the response includes all leave types or only those with remaining balance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It front-loads the verb ('Return') and clearly states the object and scope. Every word contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple: one obvious parameter, no output schema, no nested objects, and no annotations. The description adequately captures the core function. However, with no annotations and no output schema, the description could benefit from mentioning the return format (e.g., leave type to remaining days mapping) or edge cases. The simplicity of the tool keeps the gap moderate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so with a single parameter the description does not compensate. However, there's only one parameter (employee_id) whose meaning is fairly self-evident from its name—it identifies which employee's balance to check. The parameter semantics are inherent in the schema field name, and a single obvious parameter reduces the need for elaboration. Baseline 3 is appropriate since there's minimal ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb+resource: 'Return the remaining balance for each leave type for an employee.' It's a read-only query tool distinct from siblings like apply_for_leave, approve_leave, reject_leave, cancel_leave (which are all mutations) and list_pending_requests (which lists requests rather than balances). The purpose is unambiguous and differentiates well from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context—it's a leave balance query—but doesn't explicitly state when to use it vs alternatives or any exclusions. It doesn't mention that it requires an employee_id, nor does it suggest this is for checking before applying for leave. Some context is implicit given the sibling names, but no explicit guidance on choosing between tools is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.