Server Quality Checklist

Tool Scores

• diagnose_transactionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, covering safety. The description adds behavioral context: it decodes transaction/operation result codes, provides actionable guidance, and explains the need for a protocol hint. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that front-load the core function and immediately follow with usage guidance. Every word earns its place; no redundant or vague phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only diagnostic tool with full schema coverage and no output schema, the description sufficiently conveys the return value ('plain language', 'what to do about it') and the protocol hint's role. It lacks only explicit handling of invalid hashes, but that is not essential here.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with detailed descriptions for both 'hash' and 'protocol' including the rationale for the protocol hint. The description does not add new parameter semantics beyond restating the use case, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Look up a Stellar transaction by hash and explain its outcome in plain language.' It clearly distinguishes from sibling tools (explain_account, explain_contract) by focusing on transactions, with explicit mention of payments, swaps, and contract calls.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Use this whenever a user asks why a transaction, payment, swap, or contract call failed.' It does not explicitly name sibling tools as alternatives, but the usage context is clear and the sibling names imply when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_accountA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint and openWorldHint, so the description need not repeat safety. It adds value by disclosing that the output is an interpreted explanation (plain language) rather than raw data, and describes the scope (reserve, trustline authorization, etc.). This goes beyond the annotations, though it doesn't cover error cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise: two sentences front-load the purpose and details, followed by practical example questions. Every word earns its place, and the structure makes it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description clearly explains the return content (balance, reserve, assets, trustlines, signers, thresholds). It also covers representative user intents and distinguishes from sibling tools. The single parameter is fully documented in the schema, and the overall context is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes the only parameter (address) with details about format (starts with G, not C), so schema coverage is 100%. The description adds no extra parameter meaning beyond what the schema provides, making a baseline 3 appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to read and explain a Stellar account's state in plain language. It lists specific details (XLM balance, reserve, assets, trustlines, signers, thresholds) and differentiates from siblings like explain_contract by focusing on accounts. Example questions ('what do I hold') further clarify intent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit use cases ('Use this to answer...'), which indicates when to use the tool. It does not explicitly mention alternatives or when not to use it, but the account-specific focus and sibling tool names provide clear context. This is close to a 5 but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_contractA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description reinforces this with 'Read.' It adds useful context that it only shows the published interface, not source code, and that it includes documentation published by the contract author. This goes beyond the basic read-only hint to clarify scope.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is composed of two efficient sentences. The first states what it does; the second gives usage context. No redundant text or filler; every clause adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with one parameter and no output schema, the description adequately explains what is returned (functions, params, types, docs) and when to use it. Annotations cover safety, leaving no significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers the single parameter (contractId) with a description confirming it's a Soroban contract address. The tool description does not add extra parameter details beyond what the schema already provides, so it meets the baseline for 100% coverage without enhancing semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads a deployed Soroban contract's published interface and lists its functions, parameters, return types, and documentation. It distinguishes itself from sibling tools (explain_account, diagnose_transaction) by focusing specifically on contract interface discovery.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises using this tool before interacting with an unfamiliar contract or to answer specific questions about contract capabilities and function arguments. It does not mention alternatives or exclusions, but the context is clear enough given the sibling tools' different purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.