Server Quality Checklist

Tool Scores

• get_new_usersA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral disclosure. It implies a read operation via 'Get' but does not explicitly state safety (read-only), required permissions, pagination, limits, or return shape. This leaves the agent without key context for invoking the tool safely.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that immediately states the action and scope. There is no redundant text or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description captures the essential purpose ('get users') and the key filter (time window). It could mention return format or pagination, but the simplicity of the operation makes the current description minimally sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the only parameter ('days' with explanation). The tool description's 'N days' aligns with the schema but adds no additional meaning beyond what is already documented, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') with a clear resource ('users') and a defining scope ('created within the last N days'). This distinguishes it from sibling tools like get_business_summary and get_pending_payments, which focus on different resources or metrics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when needing users added within a timeframe) but provides no explicit guidance on when not to use it or how it compares to alternatives. No references to sibling tools or exclusions are present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pending_paymentsA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavioral details. It only says 'Get all customers,' which is a read operation, but it does not disclose any additional traits such as data freshness, potential size limits, sorting, or if it only returns active customers. The description is minimal and leaves the agent uncertain about return format or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that fully states the tool's purpose. There is no wasted wording, and the structure is clear and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there are no parameters, no output schema, and the tool is a simple read operation, the description is mostly complete. However, it does not specify what fields are returned for each customer or any filtering criteria like 'outstanding balance > 0' beyond the 'pending payments' phrasing, leaving a small gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts no parameters, so the description adds no parameter details. Since there are no parameters, the baseline is 4, and the description does not need to explain parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: it retrieves a list of customers with pending payments. The verb 'Get' combined with the specific resource ('customers with pending payments') distinguishes it from siblings like get_business_summary and get_new_users, which target different data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing a list of customers with pending payments, but it does not explicitly mention when to avoid using it or which alternative to prefer. Sibling tools like get_new_users suggest a similar query pattern, so some ambiguity remains about when this tool is the best choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_business_summaryA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It indicates a read-only operation via 'Get' and lists the returned metrics, but does not disclose data freshness, response format, or any prerequisites. Given the simplicity of the tool, this is adequate but lacks richer behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clearly structured sentence. It leads with the action and resource, then lists the metrics in a scannable format, making it both concise and front-loaded with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (no parameters, no output schema), the description sufficiently covers the main purpose and included metrics. It could be slightly more explicit about the expected response structure or how the metrics are calculated, but for a simple summary getter, it is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the baseline is 4. The description adds value by specifying which metrics are included in the summary, which is beyond the empty input schema and helps set expectations for the return value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get business summary metrics' and lists specific metrics (total users, active users, paid users, pending payments, new users in the last 7 days). This is a specific verb+resource+scope, and it distinguishes from siblings get_new_users and get_pending_payments by being the aggregate summary tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by enumerating the metrics included, implying that this is the comprehensive summary tool while the siblings focus on individual metrics. However, it does not explicitly state when to choose this over the siblings (e.g., 'if you need all metrics, use this; for just one metric, use the specific tool'), so it stops short of explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.