Server Quality Checklist

Tool Scores

• schedule_appointmentB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description itself carries the burden of explaining behavior. It only states the intent to book an appointment; it does not disclose side effects, validation requirements, conflict handling, persistence, or any other observable behavior beyond 'book'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single well-structured sentence includes the action, allowed appointment categories, and the target audience. Every part adds useful information and there is no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation product with no annotations and no output schema, the description is too thin. It leaves out expected results, potential failure modes, scheduling availability, and how duration interacts with the selected service type.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 75%, so the schema handles most parameters. The description adds some semantic value by clarifying that the customer must already exist and by enumerating serviceType examples that match the schema's guidances.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Book') and resource ('appointment'), names concrete appointment types, and scopes to existing customers. This clearly distinguishes it from sibling tools like create_support_ticket and create_task.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives. It does not mention when to choose a support ticket, how to validate an existing customer first, or any conditions that would rule out this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_taskA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description must carry the behavioral burden, but it only describes intent, not observable behavior. It does not mention side effects such as whether the task is immediately visible, if assignees are notified, whether tasks can be updated later, or what permissions are required.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence with no filler. It front-loads the action and purpose, and the example clarifies when this tool should be used.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description gives a solid scenario but remains thin for actual invocation: half the parameters are undocumented, there is no output-schema guidance, no sibling differentiation, and no mention of side effects or lifecycle behavior. For a mutation tool with zero annotations, this is below what an agent needs to confidently use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 50%, covering dueDate and assignee, while the required 'title' and 'description' fields are undocumented. The description's example suggests possible task content but does not map the example to specific parameter values or explain how they should be composed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly names the action ('Create a follow-up task') and the resource ('a human teammate'), and grounds it with a concrete example about calling a customer back. It reads as distinct from the sibling create_support_ticket, since this is explicitly about assigning manual follow-up to a person after automation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear when-to-use condition: use it when something needs manual handling after the AI-driven part of the interaction ends. It does not explicitly say when not to use it or name alternative sibling tools, but the intended context is reasonably evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_support_ticketA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The wording 'File a new support ticket' indicates a mutation action, but does not mention side effects, auth requirements, idempotency, or what the returned response looks like. The statement about using retrieve_status afterward adds a little context about lifecycle, but falls short of explaining the immediate behavior of this call.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no verbosity: the first states the action and trigger, the second gives a useful next-step hint. Information is front-loaded—the action is in the first word—and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description gives the trigger condition and a follow-up instruction, but no output schema exists to state the returned ticket ID or other result fields. Since the agent will need an identifier to call retrieve_status afterward, this gap is not entirely covered. Still, the essential context for the creation decision is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes all 4 parameters with high coverage, so the baseline is 3. The description doesn't add any new details about just for those fields; it only paraphrases the concept of a support ticket. It correctly fits the semantic framing of 'customer', but the schema still holds the detailed meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb and resource—'File a new support ticket'—and scopes it to the concrete case 'when their issue can't be resolved in the current interaction'. It also distinguishes the action from checking an existing ticket by referring to retrieve_status, so the tool's identity is easy to grasp and separate from its most natural sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly states a condition for calling this tool: only when the issue can't be resolved in the current interaction. The second sentence implies that already-created tickets should be handled via retrieve_status rather than creating another one, giving the agent a basic decision rule. However, it does not compare against other siblings like create_task or schedule_appointment, leaving some nuance unspecified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• customer_lookupA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must reveal behavioral traits. 'Look up' and 'returns' imply a read-only, non-mutating operation, and the mention of returned fields gives some transparency. However, it does not address what happens when no customer is found, how the tool behaves when multiple identifiers are provided, or whether any authorization is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: two sentences, a clear, simple lookup action, and a usage note. All sentences are used to fill a purpose-oriented or behavioral gap, with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool shape (had 3 params, no output schema), the description is a very complete level: names the keys, provides return values, and positions itself as the first step for a typical flow. You may still be missing the empty-result/one-of-three requirement, but that is an edge condition that is not normally needed to drive a first lookup.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and each field already has a clear description, so the baseline is appropriate. The text adds the concept of three comparable lookup keys, but it does not specify that at least one key is needed or what happens with conflicting/multiple inputs. The description adds meaningful context, but does not exceed the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('look up'), a clear resource ('customer record'), and the exact lookup keys (customer ID, email, phone). It also previews the returned data (account status and tier), making the tool's purpose distinct from the sibling tools at a glance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear ordering guidance: 'Call this first in the context of nearly every flow,' which is an explicit usage directive. But it does not enumerate exclusions or explicitly reference an alternative tool, so the contrast with siblings (e.g., retrieve_status, update_crm_record) is left slightly to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• retrieve_statusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry this burden. It implies an read-only operation with 'Check' and gives examples of possible statuses, but it doesn't disclose error handling, permissions, or whether status updates propagate elsewhere. This is adequate for a simple retrieval but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each with a distinct purpose: the first states the action, the second maps it to a real-world usage context. No padding or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter, no-output-schema, no-annotation tool, the description sufficiently covers purpose, input, and use context. It doesn't list all possible statuses, but an agent can infer the general return shape. This is complete enough for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with ticketId already described as 'Ticket ID returned by create_support_ticket, e.g. tkt-5001.' The description adds no further detail about the parameter, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies a unique verb-resource pair: 'Check the current status of a previously created support ticket by its ticket ID.' This distinguishes it from siblings such as create_support_ticket, customer_lookup, and retrieve_knowledge through its object and action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit when-to-use scenario: 'Use this before telling a customer their issue is still in progress vs. resolved.' However, it doesn't name alternatives or exclusions, so the guidance isn't as complete as it could be.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_crm_recordA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It effectively communicates partial/full patch semantics: 'Patch fields' and 'Only include the fields you want changed.' However, it omits side effects, authorization requirements, and behavior when invalid fields or unknown customer IDs are provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The core purpose is front-loaded, the example is useful, and the 'only include fields you want changed' instruction earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple patch tool with one required parameter and no nested objects, the description covers the main usage context and partial update behavior. Missing output/return info and permission notes are minor given the simplicity, but a small note on failure behavior would make it fully complete for agents.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 20% (only customerId has a description), so the description must compensate. It names phone and email as example fields to update, but does not clarify the meaning of tier or accountStatus. The enums help partially, but a richer explanation of non-obvious parameters would have been beneficial.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb ('Patch') and resource ('existing customer's CRM record'), clearly distinguishing it from sibling tools like create_support_ticket or customer_lookup. It communicates the intended operation and the record being modified without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a concrete use case ('after verifying a new phone number or email during a call') and instructs to include only changed fields. It does not explicitly mention when not to use the tool, but the context and sibling tool names make the boundary reasonably clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• retrieve_knowledgeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It makes clear this is a read-only search that returns top matches with relevance scores and URLs, so the agent understands its output and non-destructive nature. It does not go into details like empty-result behavior or ranking beyond 'top matches'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences and front-loads the core action before the output and use intent. Every sentence contributes meaningful information and there is no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter retrieval tool with no output schema, the description covers the core use case, the result format, and the purpose. It is slightly incomplete only in that it does not explicitly explain relevance cutoff, sorting, or tell the agent to prefer it over sibling lookup tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already explains that 'query' is the natural-language search and 'topK' is the maximum result count. The description does not add further parameter-level guidance, so it stays at the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb and resource: 'Search the knowledge base for articles'. It also states what the tool returns, including relevance score and source URL, making it clearly distinct from sibling tools like customer_lookup or create_support_ticket.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for use: it is for answering a customer's question with grounded, citable knowledge articles. It does not explicitly mention sibling alternatives or when not to use the tool, so it misses the strict when-not guidance needed for a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.