Server Quality Checklist

Tool Scores

• list_extra_purchasesA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, covering the safety profile. The description adds meaningful behavioral context beyond annotations: emails are redacted by default, and full emails belong to the account owner and require appropriate handling. This is useful privacy-related behavior not present in the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no filler. The primary action and default behavior are front-loaded, and the privacy caveat is delivered efficiently. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and low parameter coverage, the description leaves gaps around pagination, date filtering, and result shape. The strong annotations reduce safety concerns, and the parameter names are fairly self-explanatory, but an agent still lacks enough detail to confidently use limit, since, and max_pages correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only references include_emails implicitly and says nothing about limit, since, or max_pages. The schema provides raw constraints but no semantic meaning, leaving three of four parameters under-described.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List'), a specific resource ('recent Extras purchases'), and an important behavioral qualifier (emails redacted by default). This clearly distinguishes it from sibling tools like list_supporters and list_subscriptions, which target different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys clear context for what the tool does and hints at the privacy-sensitive handling when include_emails is true. However, it does not explicitly state when to use this tool instead of sibling tools like list_supporters or summary, nor does it provide exclusions or alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_subscriptionsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish read-only, idempotent, non-destructive behavior. The description adds meaningful behavioral context beyond annotations by disclosing that emails are redacted by default and that enabling include_emails exposes owner-identifying data. This is useful for safe and correct invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two efficient sentences that front-load the core action and redaction detail before the more nuanced privacy warning. Every sentence earns its place with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only list operation with two optional parameters and no required arguments, the description is largely complete. It covers the main invocation concern, email privacy, but it does not describe the expected return shape or exactly how max_pages affects the result, which would be helpful without an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must carry parameter meaning. It explains the critical behavior of include_emails, but max_pages is left only with name and schema constraints. The description adds some value but does not fully compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads memberships and that emails are redacted by default, which gives a specific verb and resource. It partially distinguishes itself from siblings by focusing on memberships and email handling, though it does not explicitly name or contrast the sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: when listing membership/subscription data is needed. It also provides important conditional usage guidance around include_emails, warning that full emails belong to the account owner and should be handled accordingly. It does not explicitly state when not to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• summaryA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this as read-only, idempotent, and non-destructive. The description adds meaningful behavioral context beyond annotations by disclosing that refunds and revoked purchases are excluded and that an error occurs when the page cap prevents a complete summary.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences. The first states the core aggregation, grouping, and exclusions; the second flags the error condition. No filler or redundant restatement of the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with two optional parameters and no output schema, the description provides the essential semantics: what is totaled, how it is grouped, what is excluded, and a failure condition. It could specify the exact return shape more, but 'by currency' gives enough to understand the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description carries the burden for parameter meaning. It hints at 'recent window' for days and 'page cap' for max_pages, giving useful context, but it does not explicitly map those concepts to the parameter names or explain their effect on the result.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool produces totals of one-off supports and Extras by currency, over a window, and explicitly excludes refunds and revoked purchases. This is specific enough to distinguish it from the sibling list tools, which provide itemized records rather than aggregate summaries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The use case is implied: it is the summary/aggregation counterpart to the sibling list tools. However, the description does not explicitly say when to prefer summary over list_supporters, list_extra_purchases, or list_subscriptions, nor does it state exclusions like 'use lists for itemized data'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_supportersA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the operation read-only, idempotent, and non-destructive. The description adds valuable behavioral detail beyond those hints: emails are redacted by default, full emails are sensitive account-owner data, and results are returned newest first. This gives the agent the privacy-relevant behavior it needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences carry the essential scope, ordering, and privacy caveat with no filler. The main behavior is front-loaded and every phrase contributes information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the read-only annotations, self-explanatory pagination parameters, and sibling context, the description covers the key operational facts: what is listed, in what order, and what to do about emails. It does not spell out pagination behavior or the exact since filter, but those are reasonably inferable from the schema and parameter names.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, description-level parameter guidance is required, and it only partially delivers. include_emails is well explained via the redaction and PII warning, and 'recent' loosely suggests the role of since, but limit, max_pages, and the exact filtering behavior of since are left to inference from their names and schema defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action and resource ('List recent one-off supporters'), adds ordering ('newest first'), and notes default redaction, making the tool's purpose unmistakable. The qualifier 'one-off' distinguishes it from the sibling tools list_subscriptions and list_extra_purchases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The scope is clearly implied: it lists one-off supporters rather than subscriptions or extra purchases, so an agent can apply it when that resource is needed. It does not explicitly state 'use list_subscriptions for recurring supporters,' so it stops short of a full when/when-not rule, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.