Server Quality Checklist

Tool Scores

• transferB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a transfer operation (implying mutation) but doesn't mention critical behaviors: whether this is irreversible, what permissions are needed, if there are rate limits or fees, what happens on failure, or what the response looks like. The description only covers basic parameter requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that efficiently convey the core functionality and parameter requirements. It's front-loaded with the main purpose. Minor deduction for slightly redundant parameter restatement that doesn't add significant value beyond the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a financial transfer tool with no annotations and no output schema, the description is incomplete. It doesn't address critical aspects: what happens after transfer (confirmation? transaction ID?), error conditions, security implications, or return values. The agent lacks sufficient context to understand the full behavioral implications of invoking this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents all three parameters. The description adds minimal value beyond the schema - it repeats the recipient format options and mentions the purpose is required, but doesn't provide additional context like example purposes or amount constraints beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Transfer Shells to another wallet'), identifies the resource (Shells currency), and distinguishes this from sibling tools like get_balance (read-only) or sign (cryptographic operation). It provides a complete purpose statement with verb, resource, and target.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like compute or get_transactions. While it mentions the required parameters, it doesn't specify prerequisites (e.g., sufficient balance), appropriate contexts, or warnings about when not to use it (e.g., for non-Shell transfers).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transactionsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses that the tool returns a list of recent transactions with specific types (mints, burns, transfers), which adds behavioral context beyond just 'get.' However, it lacks details on authentication needs, rate limits, pagination behavior, or what 'recent' means temporally.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences: the first states the purpose, and the second adds behavioral detail about return content. It's front-loaded with the core function and avoids unnecessary verbiage, though it could be slightly more structured by explicitly separating purpose from output details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (read-only historical data retrieval), no annotations, and no output schema, the description is minimally adequate. It covers what the tool does and what it returns but lacks details on authentication, error handling, or the structure of returned transactions, which would be helpful for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter 'limit' with its type, description, and constraints. The description doesn't add any parameter-specific information beyond what the schema provides, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get transaction history for your wallet' specifies the verb (get) and resource (transaction history). It distinguishes from siblings like get_balance (which returns current balance) and transfer (which initiates transfers). However, it doesn't explicitly differentiate from compute or sign, though those are clearly different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through 'for your wallet' and 'recent transactions,' suggesting this is for viewing historical data rather than current state (get_balance) or performing actions (transfer, sign). However, it doesn't explicitly state when to use this tool versus alternatives or mention any prerequisites like authentication requirements.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_balanceA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool returns balance and ceiling information, but doesn't mention whether this requires authentication, has rate limits, is read-only, or any error conditions. For a financial tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place: the first states the purpose, the second specifies the return values. There's zero wasted text and it's front-loaded with the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a read operation with no parameters and no output schema, the description provides the minimum viable information about what it returns. However, it doesn't address authentication requirements, error handling, or format of returned data, which would be helpful given the financial context and lack of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema already fully documents the parameter situation. The description appropriately doesn't waste space discussing parameters, and the baseline for this scenario is 4 according to the scoring guidelines.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get'), the resource ('Shell balance and ceiling for your wallet'), and distinguishes from siblings like get_transactions (which returns transactions rather than balance metrics). It provides a complete picture of what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like get_transactions or compute. It doesn't mention prerequisites, timing considerations, or any explicit 'when-not' scenarios, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• whoamiA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the return values (cardId, walletId, address, balance, status) which is helpful, but doesn't cover aspects like authentication requirements, rate limits, error conditions, or whether this is a read-only operation (implied by 'Get' but not explicit).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the first sentence stating the core purpose and the second detailing return values. Every sentence earns its place by providing essential information without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is adequate but has gaps. It explains what the tool does and what it returns, but lacks context on authentication, error handling, or usage scenarios relative to siblings. For a no-parameter identity tool, this is minimally viable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema already fully documents the lack of inputs. The description appropriately adds no parameter information, maintaining focus on the tool's purpose and outputs. Baseline for 0 parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get your card and wallet identity') and resource (identity information), distinguishing it from siblings like get_balance (balance only) or get_transactions (transaction history). It precisely communicates the tool's function without restating the name/title.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. While the description implies it returns identity information, it doesn't specify scenarios where this is preferred over other tools (e.g., using get_balance for balance-only queries or sign/transfer for other operations).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• signA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool performs a cryptographic signing operation (implying mutation or sensitive action) and returns specific data, but lacks details on permissions needed, rate limits, error conditions, or whether it's idempotent. The mention of 'proving your identity' adds some context, but behavioral traits are not fully covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core action and resource, followed by return values and use case in two concise sentences. Every sentence adds value without redundancy, making it efficiently structured and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (cryptographic operation with no annotations and no output schema), the description is moderately complete. It covers purpose, returns, and use case, but lacks details on behavioral aspects like security implications or error handling. Without an output schema, it should ideally explain return values more thoroughly, though it does list them briefly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter 'challenge' as a 'Base64-encoded challenge to sign'. The description does not add any further meaning beyond this, such as examples of valid challenges or formatting details. Baseline 3 is appropriate as the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Sign a challenge') with the resource ('your card's Ed25519 keypair') and distinguishes it from siblings like get_balance or transfer by focusing on cryptographic signing for identity verification. It explicitly mentions the return values (signature, cardId, walletId) and the use case ('proving your identity to other bots').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Useful for proving your identity to other bots'), which implicitly differentiates it from siblings like compute or get_transactions. However, it does not explicitly state when not to use it or name alternatives (e.g., if other tools handle authentication differently), leaving some guidance gaps.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• computeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key traits: it's a write operation ('Execute', 'burning Shells'), involves costs based on model and tokens, and returns response content and cost. However, it misses details like rate limits, error handling, or authentication needs, which would enhance transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and followed by cost and return details. Every sentence earns its place by adding essential information without redundancy, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (LLM execution with cost), no annotations, and no output schema, the description is fairly complete but has gaps. It covers purpose, cost mechanism, and return values, but lacks details on error cases, rate limits, or output format specifics. It compensates well but could be more comprehensive for a mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters (model, messages, temperature, maxTokens). The description adds marginal value by mentioning cost calculation based on model and token usage, but does not provide additional syntax or format details beyond what the schema specifies. Baseline 3 is appropriate as the schema handles most documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Execute an LLM request'), the resource involved ('by burning Shells'), and the outcome ('Returns the LLM response content and the cost in Shells'). It distinguishes this tool from sibling tools like get_balance or transfer by focusing on LLM execution with a cost mechanism, not financial queries or transactions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for LLM requests with cost considerations, but does not explicitly state when to use this tool versus alternatives (e.g., if other tools handle LLM tasks differently) or provide exclusions. It offers some context but lacks explicit guidance on alternatives or specific scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.