Server Quality Checklist

Tool Scores

• getCustomerOrdersA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions a maximum of 20 orders per call, adding behavioral context beyond the schema. However, it does not disclose potential error conditions or pagination behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that conveys the essential information without any fluff. Every word is meaningful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (a list operation) and the absence of an output schema, the description is fairly complete: it explains the action, constraints (max 20, role, tenant). It could benefit from mentioning error handling or result ordering, but it is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the description adds minimal value beyond what the schema already provides. The mention of 'Máximo 20 por llamada' is redundant given the limit parameter's maximum constraint.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Lista los pedidos de un cliente' (List the orders of a customer), specifying the verb and resource. It distinguishes the tool from siblings like getCustomerProfile or createSupportTicket, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context about required role (AGENT) and tenant constraint, but does not explicitly state when to use this tool vs. alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• createSupportTicketA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses the SUPPORT role requirement and implies creation is destructive. It does not mention idempotency, rate limits, or side effects, which limits transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise sentences, no redundant information, and is front-loaded with the core action. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with 4 required parameters, no output schema, and no annotations, the description provides core purpose, category constraints, and role requirement. However, it lacks explanation of return values or error responses, making it somewhat incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (all 4 parameters described in schema). The description adds context for the category enum and role requirement but does not enhance meaning beyond what the schema already provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Crea un ticket de soporte para un cliente' (Creates a support ticket for a client), providing a specific verb and resource. It lists allowed categories and role requirement, distinguishing it from sibling tools like calculateRefundEligibility or getCustomerOrders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a precondition ('Requiere rol SUPPORT') and enumerates valid categories, guiding proper usage. However, it does not explicitly state when not to use this tool or provide alternatives. The context with sibling tools implies distinct purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getCustomerProfileA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that the tool is read-only (returns profile), requires specific role and tenant membership, and returns partially masked email. This is sufficient for a simple read operation, though it could explicitly state no side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences that cover purpose, output, and prerequisites. No redundant or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema), the description adequately covers the essential aspects: purpose, return fields, and auth constraints. It could optionally mention error cases or response format, but it is complete enough for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for both parameters (customerId format and callerToken). The description adds no additional parameter information beyond what the schema already provides, so baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Obtiene' (gets), the resource 'perfil básico de un cliente', and explicitly lists the returned fields (id, nombre, tier, email parcialmente enmascarado). It also mentions prerequisites, differentiating it from sibling tools (refunds, tickets, orders, etc.).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on when to use (requires AGENT role and same tenant) but does not explicitly state when not to use or mention alternative tools. Usage is implied through the prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• requestRefundApprovalA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavioral traits: it is a non-executing request that requires human approval, a maximum amount, and a role requirement. This adds essential context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core purpose, and includes critical caveats (no execution, human approval, max amount, role). No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers essential aspects but lacks information about the tool's output (e.g., returns a request ID or status). For a multi-step process, knowing what the response contains would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing full parameter details. The description adds overall workflow context but does not enhance per-parameter meaning beyond what the schema already offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Solicita la aprobación de un reembolso' and explicitly distinguishes that it does not execute the refund but generates a pending request for human approval. It also specifies maximum amount and required role, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use (to request refund approval) and includes constraints (max 500€, requires FINANCE role). However, it does not explicitly mention when not to use or suggest alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculateRefundEligibilityA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes full responsibility for behavioral disclosure. It clearly states read-only and no action execution, and mentions the required role. Missing details such as error handling or data source, but the core behavioral trait (read-only) is well communicated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of two short sentences. The first sentence defines the purpose immediately, and the second adds the read-only nature and role requirement. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low parameter count, high schema coverage, and no output schema, the description is mostly complete. It covers purpose, behavior, and prerequisites. It could be enhanced by stating what the output contains (e.g., 'returns eligibility boolean and max amount').

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% as both parameters (orderId and callerToken) have descriptions. The tool description does not add additional meaning beyond what the schema already provides. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool checks refund eligibility and maximum amount, using a specific verb ('consulta') and resource ('pedido'). It distinguishes itself from sibling tools like requestRefundApproval, which performs the actual refund, and createSupportTicket, which has a different purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies the required role (AGENT) and explicitly states the tool is read-only, implying it should be used before requestRefundApproval. However, it does not explicitly list when not to use or compare to other siblings beyond the role and readonly hint.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sendCustomerEmailA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. Discloses role requirement, template restriction, and no free content. Could mention idempotency or side effects, but is fairly transparent for a simple email tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no waste. Front-loaded with main purpose, immediately followed by constraints. Efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, allowed templates, role, and param usage. Does not describe return value or error behavior, but for a simple send tool this is mostly sufficient given no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds that templateId must be one of the three listed and that params are substitution variables, but does not further explain the nested structure or authentication token format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it sends an email with an authorized template, lists specific templates, and notes the required role. Distinct from sibling tools which handle refunds, tickets, orders, and profiles.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use (send email with template) and lists allowed templates and required role. Does not explicitly mention when not to use or alternatives, but context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.