Server Quality Checklist

Tool Scores

• get_pricingB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It implies a read-only operation by using 'Get', but doesn't explicitly state safety aspects like whether it's non-destructive or if it requires authentication. The inclusion of specific price examples adds some context, but key behavioral traits (e.g., rate limits, error handling) are missing, making it adequate but with gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, consisting of a single sentence that directly states the tool's purpose and includes key pricing details. Every word earns its place, with no redundant or vague language, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is reasonably complete for a simple pricing lookup. It explains what the tool returns (pricing tiers with examples) but lacks information on output format or error cases. With no structured fields to rely on, it meets minimum viability but could be more comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description adds value by explaining the pricing logic (based on lifetime top-up amount) and providing example price ranges for USA and International, which goes beyond the empty schema. This compensates well, though it's not a perfect 5 as it could include more details like currency or update frequency.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: retrieving postcard pricing tiers based on lifetime top-up amount, with specific price examples for USA and International. It uses the verb 'Get' with the resource 'postcard pricing tiers', making the action and target explicit. However, it doesn't explicitly distinguish this from sibling tools like 'check_balance' or 'send_postcard', which prevents a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions pricing tiers but doesn't specify use cases, prerequisites, or comparisons to siblings like 'check_balance' (which might relate to costs) or 'send_postcard' (which might involve pricing). This lack of contextual usage information results in a low score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_statusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the return content (status, tracking info, delivery date) which is helpful, but doesn't mention error conditions, authentication needs, rate limits, or whether this is a read-only operation. It adds some behavioral context but leaves gaps for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose followed by return details. Zero wasted words, appropriately sized for a simple lookup tool. Every sentence earns its place by adding distinct value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple tool with 1 parameter, 100% schema coverage, and no output schema, the description is reasonably complete. It explains what the tool does and what it returns. However, without annotations or output schema, it could benefit from more behavioral details (e.g., error cases).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single parameter. The description adds no additional parameter semantics beyond what's in the schema (e.g., format examples or constraints). Baseline 3 is appropriate when schema does all the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('check the delivery status'), the resource ('previously sent postcard'), and distinguishes it from siblings like send_postcard (which creates) and check_balance/get_pricing (which are unrelated). It uses precise verbs and scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('previously sent postcard') and references send_postcard as the source of the ID, but doesn't explicitly state when NOT to use it or name alternatives. It provides clear context but lacks explicit exclusions or comparisons to siblings like bulk_send.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bulk_sendA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: cost deduction upfront, automatic refunds for failed cards, volume pricing based on tier, and the 500-recipient limit. However, it doesn't mention authentication requirements, rate limits, or what happens if the balance is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise and well-structured: three sentences that each earn their place by covering purpose, constraints, and financial implications. It's front-loaded with the core functionality and wastes no words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description does a good job covering the essential context: what the tool does, key constraints, and financial implications. However, it doesn't describe the return value or error conditions, which would be helpful given the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema by mentioning that recipients, message, image, and return address are the key components, but doesn't provide additional semantic context about parameter usage or constraints beyond what's in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Send the same postcard to multiple recipients at once' with specific details about what remains constant (message, image, return address) and what varies (recipient addresses). It distinguishes from the sibling 'send_postcard' by emphasizing bulk functionality with a maximum of 500 recipients.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for sending identical postcards to multiple recipients. It mentions the 500-recipient limit and volume pricing, but doesn't explicitly state when NOT to use it or name alternatives like 'send_postcard' for single recipients.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_balanceA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It describes what information is returned (balance, top-up amount, pricing tier) and the practical purpose (knowing costs and funds), but doesn't disclose behavioral traits like authentication requirements, rate limits, or whether this is a read-only operation. The description implies it's informational but doesn't explicitly state safety characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place. The first sentence states what the tool does, and the second provides clear usage guidance. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no annotations and no output schema, the description provides good contextual completeness. It explains what information is returned and when to use it. However, it doesn't describe the return format or structure, which would be helpful since there's no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, and instead focuses on the tool's purpose and usage context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('check account balance', 'lifetime top-up amount', 'current volume pricing tier') and resources ('account', 'funds', 'per-postcard cost'). It distinguishes from siblings by focusing on financial status rather than sending operations or status checks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Use this before sending to know your per-postcard cost and available funds.' This provides clear context for usage versus alternatives like send_postcard or get_pricing, though it doesn't explicitly name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_postcardA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: cost implications ('Price depends on volume tier... Charged from the user's prepaid Postcard.bot balance'), operational constraints ('printed and mailed to a real address'), and technical requirements ('publicly accessible HTTPS URL'). It doesn't mention delivery timeframes or error handling, keeping it from a perfect score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with core purpose, followed by cost, payment method, prerequisite, and requirements. Every sentence earns its place with zero waste, efficiently covering key aspects in a compact format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description does well by covering purpose, cost, payment, prerequisites, and key requirements. However, it lacks details on return values (e.g., confirmation ID, estimated delivery) and error cases, which would be helpful for a transactional tool with financial implications.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds minimal value beyond the schema: it mentions 'image URL (publicly accessible)' and 'message (max 350 characters)', which are already documented in the schema. No additional syntax or format details are provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Send a physical postcard'), resource ('postcard'), and scope ('printed and mailed to a real address'). It distinguishes from siblings like 'bulk_send' by focusing on single sends, 'check_balance' by being a transactional tool, and 'check_status/get_pricing' by being an execution tool rather than informational.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use ('Use check_balance to see current pricing tier before sending') and provides clear prerequisites ('Requires an image URL... and a message...'). It also implies when not to use by specifying single-send context versus bulk operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.