Server Quality Checklist

Tool Scores

• revenue_summaryB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full disclosure burden. It does communicate that the tool is an aggregate, read-only summary by naming computed metrics, which reasonably implies no side effects. However, it does not explicitly state behavioral traits such as scope, timezone handling, data freshness, or whether the summary is limited to certain charges.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The definition is a single sentence with no filler, placing the resource first and then listing the key output components. It is immediately scannable and efficiently worded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter aggregate tool, the description covers the main output components and the time window. However, it omits details such as the population scope, currency, failure-rate calculation, and any exclusions relative to sibling tools. Given no output schema or annotations, an agent would still have to infer some return shape and edge-case behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, days, is referenced by 'N días', which correctly maps the parameter to the lookback window. The description does not explain the default of 7, the min/max bounds, or whether days are calendar days, although those constraints are already present in the schema. It adds a thin layer of meaning beyond the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a revenue summary for the last N days and lists the concrete metrics returned: total collected, refunded, and charge failure rate. It is specific enough to be understood, though it does not explicitly contrast it with sibling tools like list_recent_charges. The lack of an explicit verb keeps it from a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool instead of list_recent_charges, find_duplicate_charges, or get_customer. It only describes what the tool computes, not the conditions under which an aggregate revenue summary is preferred over a detailed charge list. The agent is left to infer the appropriate use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_recent_chargesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the burden of explaining behavior. It does disclose that the response includes amount, status, and failure reason when applicable, and that charges are scoped to a customer. However, it does not explain ordering, time window, pagination, read-only guarantees, or what 'recent' means, leaving meaningful behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The entire description is one efficient, front-loaded sentence. It puts the action and resource first, then adds the relevant output facets without any filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and the description covers the basic return content, but with no annotations, no output schema, and no exclusionary guidance, an agent is missing information about ordering, recency bounds, and how this tool differs from charge-related siblings. It is minimally adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 50%: customer_id is described, but limit is not. The description does not compensate for this by explaining how limit affects the result, what ordering applies, or how 'recent' is determined. Without extra guidance in the description, an agent must infer the semantics of the undocumented parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Lista'), a specific resource ('cargos recientes'), and a target scope ('de un customer') while naming the returning fields: amount, status, and failure reason. This clearly distinguishes it from siblings like revenue_summary and get_customer, and even from find_duplicate_charges by focusing on recent charges rather than duplicates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case: when an agent needs a customer's recent charges with amount, status, and failure reason. However, it gives no explicit guidance about when not to use it, nor does it mention alternatives such as find_duplicate_charges or revenue_summary. The context is understandable but relies on inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_customerA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the disclosure burden. It makes clear this is a read-style lookup returning profile data, but it does not mention failure behavior, permissions, or whether the response includes only active subscriptions or additional customer fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-structured sentence that front-loads the core purpose and scope. Every phrase contributes essential information without filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter lookup with no output schema, the description gives enough high-level information: what to search by and what will be returned. It could mention edge cases like customers not found, but this is a minor gap given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: the query parameter is documented as 'Email o ID (cus_...) del customer'. The description restates this same information without adding new semantic detail, so it meets the baseline but does not go beyond it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the resource (Stripe customer), the action (search), the allowed lookup keys (email or ID), and the returned content (profile with active subscriptions). This distinguishes it sharply from sibling tools focused on charges and revenue.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied: it is the tool for retrieving a customer profile by email or ID. However, there is no explicit guidance on when to prefer this over siblings or when not to use it, leaving routing somewhat to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_duplicate_chargesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It adds meaningful detail: only successful charges are considered, amounts must match, and duplicates are evaluated within a time window. However, it does not clarify whether the operation is read-only or what the return shape is.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence with no filler. It front-loads the main purpose and includes only high-value qualifiers: 'posibles', 'exitosos', 'mismo monto', 'mismo customer', and 'ventana de tiempo'.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool, the description plus the fully documented schema is adequate for selecting and invoking the tool. However, there is no output schema and no annotation coverage, and the description does not explain what the result contains, making it only partially complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description reinforces the 'ventana de tiempo' concept but adds no parameter-specific detail beyond what the schema already documents for customer_id and window_minutes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Detecta posibles dobles cobros') and a precise scope: successful charges of the same amount on the same customer within a time window. This distinguishes it clearly from siblings like list_recent_charges and revenue_summary, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: when the agent needs to surface possible duplicate successful charges for a customer within a window. It does not explicitly name alternatives or state when not to use it, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.