Server Quality Checklist

Tool Scores

• send_faxB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does not disclose behavioral traits such as authentication needs, rate limits, or what happens upon success or failure. The testMode parameter indicates a test mode, but the description does not elaborate on its behavior beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the core purpose and adding a clarifying detail about additional services. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite 10 parameters and no output schema, the description remains minimal. It does not explain operational behavior, return values, or error cases. The schema covers parameters, but the description lacks completeness for effective agent usage without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the existing parameter descriptions are detailed (e.g., for text, files, address). The tool description adds no new parameter-level information beyond what the schema provides, meeting the baseline expectation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it sends documents or plain text as a fax via PixelLetter, specifically noting it does not post a letter. This differentiates it from the sibling tool 'send_letter' and clarifies that additional services like registered mail do not apply.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for faxing rather than mailing, but does not provide explicit when-to-use or when-not-to-use guidance. The mention that registered mail does not apply offers some direction, but alternatives like 'send_letter' are not mentioned in the description itself.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_letterA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the action (sends), test mode to prevent charges, and return values. However, it omits details on idempotency, reversibility, authentication requirements, or rate limits, leaving behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences cover the core functionality efficiently without fluff. The structure could be slightly improved by grouping modes more clearly, but it is concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (22 parameters, nested objects), the description covers the return values and test mode requirement but lacks mention of prerequisites (e.g., PixelLetter account), error handling, or operational limits, leaving some gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds high-level context (e.g., how text and files interplay, signature insertion) but does not significantly deepen understanding of individual parameters beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool sends a letter via PixelLetter, with two modes (ready documents or text plus address). It also mentions optional faxing and return values, clearly differentiating from sibling tools like send_fax or sign_invoice.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for sending letters but does not explicitly contrast this with sibling tools (e.g., when to use send_fax instead). It provides no exclusions or when-not-to-use guidance, relying on the user to infer from context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sign_invoiceA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses test mode behavior, optional mailing, error condition (046), and merging of files. It lacks details on return values or side effects, but overall provides substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action and optional behavior, followed by a crucial prerequisite. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (5 parameters, nested objects, no output schema), the description covers the main purpose and a key error. However, it does not clarify how files and inlineFiles interact (e.g., can both be provided?), nor what the tool returns (e.g., order number). More detail would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the tool description adds meaning beyond the schema by mentioning the external service name (PixelLetter), action 4, and the optional mailing behavior which relates to the notification parameter. This extra context helps the agent understand the tool's integration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool sends documents to the PixelLetter invoice signing service (action 4) and optionally mails the signed PDF. This distinguishes it from sibling tools like send_letter and send_fax which are for physical sending, and get_account_info which is informational.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The description mentions a prerequisite (electronic signatures enabled) and an error condition, but does not compare to siblings or specify contexts for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_interface_referenceA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description explicitly states it performs no API call and is offline, implying no destructive or network-dependent behavior. Without annotations, it fully discloses the tool's safe, read-only nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first enumerates returned data, second adds offline context. No wasted words, front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Description covers what is returned but lacks detail on response structure or format. Given no output schema, a bit more detail would be helpful, but it is adequate for a simple lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema provides 100% coverage for the single optional parameter errorCode. Description references error codes but does not clarify behavior when parameter is omitted (likely returns all). Baseline 3 is appropriate as description adds minimal value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns interface documentation including action values, dispatch centres, etc. It is distinctly different from sibling tools like send_letter or send_fax, making its read-only reference purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description mentions 'Offline lookup, no API call' which implies safe usage, but does not explicitly state when to use versus alternatives. However, given sibling tools are all action-oriented, the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_account_infoA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description clearly indicates a read operation ('Reads'). Does not mention safety or side effects but is straightforward and correct for a read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. Front-loaded with purpose, followed by usage guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers usage and error code. No output schema, but return values not detailed. Reasonably complete for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters, so schema coverage is 100%. Description adds no param details, which is appropriate. Baseline of 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it reads customer data and current credit. Distinct from sibling tools which are for sending or signing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using before sending to check balance, and explains error 021 for low credit. Provides clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.