Server Quality Checklist

Tool Scores

• release_escrowD
  1.5/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The behavioral transparency section is detailed but describes an idempotent, read-only analysis tool. If the tool actually releases escrowed funds (as implied by the name), these claims are false. The description does not reconcile the contradiction or provide accurate side effects, authentication requirements, or error handling for a fund release operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured into sections (Behavior, When to use, etc.), but it is overly long due to repetition (e.g., each parameter has the same generic phrase) and inclusion of irrelevant analysis details. It could be half the length without losing clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of annotations and the existence of an output schema (not described), the description fails to explain the tool's role in the escrow workflow, what the release actually does (e.g., changes state, triggers events), how to handle errors in a real release, or how it differs from siblings like escrow_funds or process_payment. The description is fundamentally incomplete for a mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must explain parameters. However, it only copies parameter names with the generic suffix 'to analyze or process', e.g., 'The escrow id to analyze or process.' This adds no meaning beyond the schema titles and fails to clarify what each parameter does in the context of releasing escrow.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence 'Release escrowed funds to the designated agent after condition verification' clearly states a mutation operation, but the subsequent 'Behavior' section immediately contradicts it by claiming the tool is 'read-only and stateless' and produces analysis output. This internal contradiction makes the actual purpose ambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' and 'When NOT to use' sections describe an analysis/classification tool, not a fund release tool. They are entirely mismatched with the tool name and the first sentence. No guidance is given on when to use this instead of siblings like escrow_funds or process_payment.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_invoiceD
  1.9/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description internally contradicts itself: it claims to create an invoice but also states it is read-only with no side effects. This is a serious inconsistency that misleads the agent about the tool's true behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy and repetitive, with the 'Behavioral Transparency' section echoing information already in 'Behavior'. It lacks front-loading and contains irrelevant generic content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite providing details on rate limits and error handling, the description fails to explain the actual invoice creation process, output format, or side effects, leaving critical gaps for a tool that claims to create an invoice.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameter descriptions in the 'Args' section are generic placeholders ('The X to analyze or process') that do not add any meaning beyond the parameter names, offering zero value given 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description initially states 'Create a detailed invoice', which matches the name, but later contradicts it by claiming the tool is read-only and produces no side effects, undermining purpose clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' section suggests using the tool for structured analysis or classification, which is not aligned with invoice creation, and fails to differentiate from siblings like process_payment or release_escrow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• escrow_fundsD
  1.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Although the description includes a detailed 'Behavioral Transparency' section, it incorrectly claims the tool is read-only and idempotent, which contradicts the escrow action. This misleads the agent about side effects and state changes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is repetitive (e.g., the 'Behavior' bullet list and the 'Behavioral Transparency' section overlap) and contains boilerplate that does not earn its place. Shorter, tool-specific content would be more effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema and sibling tools, the description fails to explain how the escrow works, what the output contains, or how to use the tool correctly. The generic analysis framing is incomplete for a financial escrow operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate but only provides generic, unhelpful descriptions like 'The agent a to analyze or process.' No specifics about parameters like 'condition', 'expiry_hours', or 'currency' in the context of escrow are given.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence 'Place funds in escrow between two agents with conditions and expiry' clearly indicates a financial action, but subsequent statements claim it is 'read-only and stateless' producing 'analysis output', creating a major contradiction that undermines purpose clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' and 'When NOT to use' sections are generic and seem copied from a different tool (e.g., 'structured analysis or classification'). They do not differentiate from sibling tools like 'release_escrow' or provide context for using this escrow tool specifically.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• process_paymentC
  2.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The Behavioral Transparency section is detailed, covering side effects, authentication, rate limits, error handling, idempotency, and data privacy. However, the core contradiction between the tool's name (process_payment) and its claimed read-only nature undermines trust in the description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is verbose with redundant sections (e.g., 'Behavior' and 'Behavioral Transparency' overlap). While front-loaded with purpose, the structure could be tightened to reduce repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite detailed behavioral aspects, the description fails to coherently explain the tool's core functionality, leaving ambiguity about whether it actually processes payments or performs read-only analysis. This gap is critical given the tool's name and expected use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description's 'Args' section provides only generic phrases like 'The invoice id to analyze or process,' adding little meaning beyond the parameter names. No allowed values or constraints are specified for payment_method.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description initially claims 'Process payment for an invoice with fraud checks, fee calculation, and settlement,' but later contradicts itself by stating 'This tool is read-only and stateless — it produces analysis output without modifying any external systems.' This fundamental inconsistency makes the tool's actual function unclear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' and 'When NOT to use' sections are generic and do not specifically guide selection between this tool and siblings like escrow_funds or release_escrow. The guidance about 'structured analysis' does not align with the payment processing implied by the name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• payment_historyA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It thoroughly details side effects (read-only, stateless), authentication (none for basic), rate limits (10/day free), error handling (structured errors), idempotency, and data privacy. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose but becomes verbose with repeated information (e.g., behavioral traits listed twice in different sections). Could be more concise without losing clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters and an output schema, the description explains return values (structured errors) and covers authentication, rate limits, and idempotency. Missing detailed parameter descriptions but overall sufficient for context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the parameter list in description merely repeats names with generic 'to analyze or process' for each. This adds no meaningful semantics beyond the schema itself. For instance, status_filter and limit are not explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get payment history for an agent with filtering, totals, and transaction summary.' It uses specific verbs and resources, and the tool is distinct from mutation siblings like create_invoice and process_payment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Includes explicit 'When to use' and 'When NOT to use' sections. However, the guidance is somewhat generic (e.g., 'structured analysis') instead of being specific to payment history. Still provides clear context on when it is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.