Server Quality Checklist

Tool Scores

• check-balanceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Check' implies a read-only operation, it doesn't specify whether this requires network access, has rate limits, returns real-time or cached data, or what happens with invalid addresses. This leaves significant gaps for a tool interacting with blockchain data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently communicates the core functionality without any wasted words. It's perfectly front-loaded with the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a blockchain query tool with no annotations and no output schema, the description is insufficient. It doesn't explain what the balance check returns (e.g., in wei or ETH, with decimals), whether it includes pending transactions, or any error conditions. Given the complexity of blockchain interactions, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting the single 'address' parameter with format and pattern. The description adds no additional parameter information beyond what the schema provides, so it meets the baseline for high schema coverage without adding extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Check') and resource ('ETH balance of an Ethereum address'), making the tool's purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get-token-transfers' or 'get-transactions' that might also involve balance-related queries, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get-token-transfers' or 'get-transactions' that might provide related information, there's no indication of when this specific balance check is preferred or what its limitations are.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-contract-abiC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does but reveals nothing about behavioral traits: no information about rate limits, authentication requirements, error conditions, response format, or whether this is a read-only operation (though 'Get' implies reading). This leaves significant gaps for agent understanding.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core purpose without unnecessary words. It's appropriately sized for a simple retrieval tool and front-loads the essential information ('Get the ABI'). Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't address what format the ABI returns (JSON array? specific structure?), error handling, or network requirements. For a contract interaction tool with no structured metadata, more context about behavior and output would be needed for proper agent usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'address' fully documented in the schema. The description adds no additional parameter semantics beyond what the schema already provides (it doesn't explain ABI format, network context, or address validation). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('ABI for a smart contract'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'get-contract-code' (which retrieves bytecode rather than ABI), but the specificity of 'ABI' provides some implicit distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites (e.g., needing a contract address), comparison to siblings like 'get-contract-code', or context for when ABI retrieval is appropriate versus other contract-related operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-contract-codeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states what the tool does but doesn't describe how it behaves: no information on permissions needed, rate limits, error conditions (e.g., invalid addresses), output format (e.g., raw code, metadata), or whether it's a read-only operation. This leaves significant gaps for safe and effective use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It is front-loaded with the core purpose ('Get the source code'), making it easy to scan and understand quickly. Every part of the sentence contributes directly to explaining the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of retrieving smart contract code (which may involve blockchain interactions, authentication, or formatting), the description is insufficient. With no annotations and no output schema, it doesn't cover behavioral aspects like error handling, response structure, or limitations. The description alone doesn't provide enough context for reliable agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'address' parameter fully documented in the schema (including format and pattern). The description adds no additional parameter semantics beyond implying the address is for a smart contract, which is already clear from the tool name. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('source code for a smart contract'), making the purpose immediately understandable. It distinguishes from siblings like 'get-contract-abi' (which retrieves ABI) and 'get-transactions' (which retrieves transaction data). However, it doesn't specify the verb tense or scope details like whether it retrieves all source code or specific versions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a contract address), exclusions (e.g., not for non-contract addresses), or comparisons with siblings like 'get-contract-abi' for interface data. The agent must infer usage from the name and schema alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-transactionsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does but lacks critical details: it doesn't specify what 'recent' means (e.g., time range, block number), whether it includes internal transactions, the data format returned, error handling, or rate limits. This leaves significant gaps for an agent to understand operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core functionality, making it easy to parse quickly. Every word earns its place, adhering to ideal conciseness standards.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (querying blockchain data), lack of annotations, and no output schema, the description is insufficient. It doesn't explain return values (e.g., transaction list format, fields included), error cases, or behavioral constraints like rate limits or data recency. This leaves the agent under-informed for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear documentation for both parameters (address format and limit constraints). The description adds no additional parameter semantics beyond what the schema provides, such as clarifying 'recent' in relation to the limit or address validation nuances. This meets the baseline for high schema coverage but doesn't enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('recent transactions for an Ethereum address'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'get-token-transfers', but it's specific enough to avoid confusion with unrelated tools like 'check-balance' or 'get-gas-prices'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-token-transfers' for ERC-20 transfers or 'check-balance' for balance queries. It doesn't mention prerequisites, such as needing a valid Ethereum address, or exclusions, leaving the agent to infer usage context solely from the tool name and parameters.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-ens-nameB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states what the tool does but lacks details on traits like error handling (e.g., for non-ENS addresses), performance (e.g., response time), or side effects (e.g., network calls). This leaves gaps for an agent to understand operational behavior beyond the basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that front-loads the core purpose without unnecessary words. It efficiently communicates the tool's function, earning its place with zero waste, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is minimally complete. It covers the basic purpose but lacks context on usage, behavior, or output details. Without annotations or output schema, more guidance would be helpful, but it meets the minimum viable threshold for such a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'address' parameter well-documented in format and pattern. The description adds no additional parameter semantics beyond implying the address is used to fetch the ENS name. Since the schema does the heavy lifting, the baseline score of 3 is appropriate, as the description doesn't compensate but doesn't detract either.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('ENS name'), and identifies the target ('Ethereum address'). It distinguishes from siblings like 'check-balance' or 'get-transactions' by focusing on ENS name resolution. However, it doesn't explicitly differentiate from potential similar tools not present in the sibling list, keeping it at 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an address), exclusions (e.g., invalid addresses), or comparisons to siblings like 'get-contract-abi' for different data types. Usage is implied from the purpose but not explicitly stated, resulting in minimal guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-token-transfersB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states what the tool does but lacks details on permissions, rate limits, data freshness, pagination, or error handling. For a read operation with no annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded and wastes no space, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (retrieving token transfers), lack of annotations, and no output schema, the description is minimally complete. It specifies the resource and scope but omits details on behavior, output format, and usage context, which are needed for full agent guidance in this environment.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with clear documentation for both parameters ('address' and 'limit'), including formats and constraints. The description adds no additional parameter semantics beyond what the schema provides, so it meets the baseline score of 3 for adequate but not enhanced coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('ERC20 token transfers') with the scope ('for an Ethereum address'), making the purpose specific and understandable. However, it doesn't explicitly differentiate this tool from sibling tools like 'get-transactions', which might also retrieve transaction data involving addresses, leaving room for potential confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get-transactions' or 'check-balance', nor does it specify use cases, prerequisites, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-gas-pricesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'current' gas prices, implying real-time data, but doesn't specify data sources, update frequency, rate limits, or error handling. This leaves significant gaps in understanding how the tool behaves operationally.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core functionality ('Get current gas prices in Gwei') with no wasted words. Every part of the sentence contributes directly to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, zero-parameter tool with no output schema, the description is minimally adequate—it states what the tool returns. However, without annotations or output details, it lacks information on return format (e.g., numeric value, object structure), units beyond 'Gwei', or data freshness, leaving room for improvement in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the schema fully documents the lack of inputs. The description appropriately doesn't add parameter details, maintaining focus on the tool's purpose without redundancy. A baseline of 4 is applied for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('current gas prices in Gwei'), making the purpose immediately understandable. However, it doesn't distinguish this tool from potential sibling tools that might also retrieve gas prices with different parameters or scopes, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like checking balances or getting transactions, nor does it mention any prerequisites or contextual constraints. It simply states what the tool does without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.