Server Quality Checklist

Tool Scores

• query_transactionA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint, openWorldHint), the description discloses that it reads public ledger data only, the meaning of not-found vs rejection, the network field repetition, and the separate-ledger warning with testnet value. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is detailed but well-structured: first paragraph presents core functionality and purpose; second paragraph adds critical caveats. Every sentence adds value, though it could be slightly more concise without losing substance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given it has an output schema, the description covers purpose, use case, error semantics, network context, and value warnings. All important behavioral aspects are addressed for a query tool of moderate complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with a detailed pattern and example for the hash parameter. The tool description does not add extra semantic detail beyond what the schema already provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('look up a single Pi transaction by its hash') and specifies the reported fields (success, ledger, submitter, fee, memo). It distinguishes from sibling tools by focusing on verifying a specific transaction, contrasting with balance checks or payment listings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to call: 'to verify that a specific transaction actually went through' and that a claim is not proof. Describes the distinction between not-found errors and rejection, warns about network confusion and testnet value. Provides clear context for correct interpretation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_wallet_balanceA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes beyond the readOnlyHint and openWorldHint annotations by detailing critical behavioral traits: the tool returns a not-found error for unfunded addresses (distinguishing 'absent' from 'holds nothing'), emphasizes that it reads Pi Testnet, and warns that balances are chain-specific. It also instructs to always report the network from the result field. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph with no wasted words. It front-loads the core purpose, then adds essential behavioral details in a logical order. Every sentence contributes to understanding the tool's use and constraints.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no nested objects, output schema present), the description covers all necessary aspects: purpose, when to use, parameter semantics, behavioral quirks, and output interpretation. It is self-contained and provides enough information for an AI agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds significant value beyond the schema's pattern and description: it explains the exact character set, provides an example with a warning about wrong addresses, and clarifies that the example is not a default. This enhances usability.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Read the current Pi and custom-token balances of a Pi wallet address.' It specifies the verb (read), resource (wallet address), and what it retrieves (balances). It also distinguishes its use case from potential alternatives by explicitly stating when to call it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use the tool: 'Call this whenever you need to know how much Pi an address holds, whether it holds a particular token, or whether the account exists on-chain at all.' It also mentions that it cannot move funds and needs no credentials. However, it does not explicitly exclude sibling tools (e.g., get_account_payments for transactions), so it lacks a clear when-not-to-use comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_userA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond annotations (readOnlyHint, openWorldHint) by disclosing the exact API endpoint called (https://api.minepi.com/v2/me), stating that data is not stored or logged, explaining error handling (invalid/expired returns valid:false with reason, only transport/server faults cause errors), and clarifying that uid is app-specific and the result lacks a network field. This fully informs the agent about the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph that efficiently covers purpose, usage, behavior, error handling, and caveats. Every sentence contributes value, but it could be slightly more structured (e.g., bullet points) for quicker scanning. Nonetheless, it is concise for the amount of information conveyed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (single parameter, output schema present), the description is fully complete. It explains return behavior for invalid/expired tokens and errors, notes the absence of a network field, and covers data handling. With an output schema, the agent does not need more detail about return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a detailed parameter description. The description reinforces the parameter's significance by emphasizing not to pass uid, username, or server API key. While the schema already defines the parameter well, the description adds critical context about why the token is the sole proof of identity, justifying a score above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Check whether a Pi user access token is genuine and, if so, who it belongs to.' It uses a specific verb (verify) and resource (user access token), and the tool is easily distinguished from siblings like get_wallet_balance or query_transaction, which serve entirely different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance: 'Call this to authenticate someone who claims a Pi identity' and warns 'never trust a client-supplied uid or username on its own; this is the only thing that proves it.' It also contrasts this tool with ledger reads, telling the agent when to use this identity check versus other queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_account_paymentsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare readOnlyHint=true and openWorldHint=true, indicating a safe read operation. The description adds crucial behavioral details beyond annotations: pagination mechanics (next_cursor/cursor), the not-found vs empty-list distinction, the testnet vs mainnet separation including the fact that the same address can hold different balances on each, and the monetary value warning for testnet. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose. However, it is somewhat lengthy at multiple sentences. While every sentence adds value, some could be more concise (e.g., the testnet monetary warning and ledger separation are important but add extra length). Still, it avoids redundancy and is well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, 1 required, output schema present), the description is highly complete. It covers all key aspects: what results contain (transaction types, network field), edge cases (unfunded address error vs empty page), pagination handling, and cross-chain context. The output schema exists, so the description rightly focuses on behavioral semantics rather than return value details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so a baseline of 3 is expected. The description adds significant value beyond the schema by explaining the pagination pattern (how to use cursor and next_cursor), interpreting the address parameter with a real usage warning about format vs intended address, and clarifying the order default behavior in context of the tool's purpose. Every parameter is enriched with practical usage context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool lists payments sent to or from a Pi wallet address, newest first, and covers specific transaction types (payments, account creations, path payments, account merges). It clearly distinguishes this from siblings like get_wallet_balance by focusing on transaction history rather than balances, and from query_transaction by addressing account-level history rather than individual transaction lookups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Call this to answer questions about an address's transaction history — whether a payment arrived, who funded an account, or what it recently sent.' It also implicitly excludes balance queries (handled by get_wallet_balance) and single-transaction lookups (query_transaction). Additionally, it warns about the not-found error for unfunded addresses, explaining how to interpret results correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.