Server Quality Checklist

Tool Scores

• createSupportTicketA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden, and it does explain key creation behavior: a new incident/inquiry is created, a unique ID is assigned, and priority-based routing occurs. It does not disclose permissions, potential side effects, or what the caller receives in response, which matters for a mutation tool without an output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences and no filler; the action is front-loaded and the second sentence adds concrete behavioral detail. The slight redundancy between 'Open' and 'Creates' prevents this from being a perfect 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and its parameters are fully documented, but no annotations and no output schema mean the description should convey expected return value or side effects. It covers core behavior but omits what the caller gets back and any preconditions such as customer existence.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters; the description adds no parameter-level detail beyond reaffirming that priority affects routing. This meets the baseline but adds no extra semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Open'/'Creates') and resource ('support ticket in the CRM'), and specifies two concrete behaviors: assigning a unique ticket ID and routing to the support queue by priority. This clearly distinguishes it from the sibling read/list tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied: this is the creation tool among get/list siblings, so the agent can infer it is for opening a new ticket rather than retrieving existing data. However, there is no explicit statement of when to choose it over alternatives, no prerequisites, and no exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getBillingHistoryA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It clearly discloses the sensitive redaction behavior ('Payment cards ... automatically redacted with [REDACTED] by the DLP engine'), and the verb 'Retrieve' implies a read-only operation. It does not discuss authorization or rate limits, but the redaction disclosure is valuable and goes beyond the name/schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded. The first sentence immediately states the resource and operation, and the second sentence adds one crucial behavioral detail. No wasted words or repetition of schema fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with one parameter and no output schema, the description covers the key return contents (statements, balance, payment methods, invoice IDs) and the DLP redaction behavior. It is missing potential error scenarios, but this is not critical for a straightforward retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% coverage for the single parameter, including an example and format description. The description adds no new semantics beyond referring to 'a specific customer', which is already implied by schema coverage. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Retrieve') and a distinct resource ('customer billing statements and payment methods'), and enumerates concrete data components (balance due, payment methods, invoice IDs). This clearly separates it from sibling tools like getCustomerDetails or listSupportTickets, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the listed sibling tools. The description explains what the tool does but not when an agent should choose it over alternatives, nor does it mention any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getCustomerDetailsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are present, the description carries the burden of explaining behavior. It clearly indicates a read-only retrieval and enumerates the returned fields, but it does not mention failure modes, authorization requirements, or data freshness. This is acceptable for a simple lookup but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded with the core action. There is slight redundancy between 'Retrieve detailed customer record by ID' and 'Retrieves the complete customer account record', but the overall structure is efficient and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter tool with no output schema, the description adequately covers what is returned and when to call it. It could mention behavior when the customer is not found, but nothing essential is missing for selecting and invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the customerId parameter already documented with format and example. The description adds no new parameter-level semantics beyond reinforcing that retrieval is by ID, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Retrieve'), a specific resource ('detailed customer record by ID'), and lists exactly what the record includes. It clearly differentiates from sibling tools like getBillingHistory and listSupportTickets by framing this as account-metadata lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage context: use this tool after discovering a customerId to inspect account metadata. It does not explicitly name alternatives or exclusion conditions, but the 'by ID' constraint and typical lookup purpose make the usage intent clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• listSupportTicketsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It correctly signals read-only behavior via 'Query', states that both open and resolved tickets are included, and notes the org-wide scope. However, it does not mention response shape, pagination, default behavior when no filters are supplied, or other operational caveats.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: purpose first, then scope, then filters and typical use cases. There is no fluff, and the structure makes it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only listing tool with two optional parameters and no output schema, the description plus schema provides enough context for correct invocation. The main gaps—return format and behavior without filters—are not blocking for this complexity level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters are fully documented in the schema with descriptions and examples (100% coverage), so the description adds little semantic value beyond restating that filters exist. It does not clarify whether filters combine or what the default behavior is when omitted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Query') and a specific resource ('customer support tickets'), then adds scope ('open and resolved across the organization'). This clearly distinguishes it from siblings like createSupportTicket, getCustomerDetails, and getBillingHistory, which cover creation or other customer/billing resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It tells the agent exactly when to reach for the tool ('Use this tool to check active issues, SLA status, or service incidents') and the available filters (customerId, priority). It does not explicitly name alternatives or state when not to use it, but the use cases are concrete enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• listCustomersA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral disclosure burden. It clearly states that this is a read-only paginated list operation and discloses the return shape: an array of customer objects with customerId, name, email, and account status. This gives meaningful behavioral transparency for a simple list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded, and every sentence contributes: the first sentence states the resource and operation, the second gives usage guidance, and the third describes the return value. There is no fluff or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description compensates by specifying the return array's fields. It also conveys pagination and usage intent. It could have explicitly guided the agent toward getCustomerDetails for single-record lookups, but the core selection and invocation context is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents both parameters (limit and status) with descriptions, defaults, ranges, and enum values, so schema coverage is 100%. The description adds little parameter-level meaning beyond labeling the result as paginated and mentioning account status in the output, which is consistent with the status parameter but not additive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb-resource pair ('List enterprise customer profiles') and specifies a paginated retrieval of customer accounts. It distinguishes itself from obvious siblings like getCustomerDetails by framing this tool as a browse/search/list operation, not a single-record detail lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use the tool: 'when you need to search, discover, or browse customer records.' It does not explicitly name alternatives or give when-not-to-use guidance, but the usage context is sufficient for an agent to distinguish it from detail, billing, and support-ticket tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.