Server Quality Checklist

Tool Scores

• faucetD
  1.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but fails completely. It doesn't indicate whether this is a read or write operation, what permissions are required, whether it makes network requests, what side effects occur, or what the typical response looks like. For a tool that likely interacts with blockchain networks, this lack of behavioral information is a critical gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single 5-word phrase. While this represents efficient communication, it's arguably under-specified rather than appropriately concise. Every word earns its place, but more words would be needed to make the description genuinely helpful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of interacting with blockchain networks, no annotations, no output schema, and 2 parameters with 0% schema description coverage, this description is completely inadequate. It doesn't explain what the tool does, how to use it, what parameters mean, what behavior to expect, or what results will be returned. For a tool in a technical domain like blockchain interaction, this level of incompleteness is unacceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and 2 parameters, the description provides no information about what 'addresses' or 'network' mean in this context. While the schema shows 'network' has an enum with values like 'testnet' and 'devnet', the description doesn't explain what these networks are, why one would choose between them, or what 'addresses' represents (wallet addresses, contract addresses, etc.). The description fails to compensate for the schema's lack of parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get faucet from sui networks' is vague and tautological - it essentially restates the tool name 'faucet' with minimal additional information. While it mentions 'sui networks' as context, it doesn't specify what 'faucet' means in this context (requesting test tokens, accessing a faucet service, etc.) or what the tool actually does beyond the generic term 'get'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling tools like 'sui-balance' or 'sui-transfer'. There's no indication of prerequisites, appropriate contexts, or alternatives. Users must infer usage from the tool name alone, which is insufficient for effective tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_account_info_by_private_keyD
  1.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states only the action without any details on permissions, security implications (e.g., handling private keys), rate limits, error handling, or what the return value includes. For a tool involving sensitive private key input, this lack of transparency is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with a single sentence, 'Get account info by private key', which is front-loaded and wastes no words. While it may be under-specified, it earns full marks for brevity and lack of redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of handling private keys, the absence of annotations, no output schema, and 0% schema description coverage, the description is severely incomplete. It does not address critical aspects like what 'account info' entails, security warnings, or usage context, making it inadequate for safe and effective tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, with one parameter 'privateKey' undocumented in both schema and description. The description does not add any meaning beyond the parameter name, such as explaining the format (e.g., hex string, mnemonic), validation rules, or security considerations. This fails to compensate for the low coverage, leaving the parameter semantics unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get account info by private key' is essentially a tautology that restates the tool name with minimal elaboration. It specifies the verb 'Get' and resource 'account info', but lacks specificity about what 'account info' includes or distinguishes this tool from siblings like 'sui-balance' or 'random-sui-account'. The purpose is vague beyond the basic action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description does not mention prerequisites, context, or exclusions, and it fails to differentiate from sibling tools such as 'sui-balance' (which might retrieve balance info) or 'gen_sui_accounts_by_mnemonic' (which generates accounts). This leaves the agent without direction on appropriate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gen_mnemonicC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions a security warning ('Not recommended for production'), which hints at potential risks, but doesn't describe what the tool actually does (e.g., generates a BIP39 mnemonic, returns a string, requires no authentication, has no side effects). This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two words and a parenthetical warning, making it front-loaded and free of unnecessary details. Every element ('Generate mnemonic' and the warning) serves a clear purpose without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a tool that likely generates cryptographic mnemonics), lack of annotations, no output schema, and poor parameter documentation, the description is incomplete. It doesn't explain what a mnemonic is, how it's used, what the output looks like, or the implications of the security warning, leaving the agent with insufficient context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter ('num') with 0% description coverage, and the tool description provides no information about parameters. The description doesn't mention 'num' or explain what it controls (e.g., number of mnemonics to generate), failing to compensate for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate mnemonic'. It uses a specific verb ('Generate') and identifies the resource ('mnemonic'), making the action clear. However, it doesn't differentiate from sibling tools like 'gen_sui_accounts_by_mnemonic' or explain what type of mnemonic is generated (e.g., cryptographic seed phrase).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a warning ('Not recommended for production') which offers some usage context about security concerns. However, it doesn't specify when to use this tool versus alternatives like 'gen_sui_accounts_by_mnemonic' or 'random-sui-account', nor does it explain prerequisites or typical use cases beyond the warning.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sui-transferC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'transfer SUI' which implies a write/mutation operation, but doesn't mention critical behaviors like transaction fees, irreversible nature, required permissions (e.g., private key), rate limits, or error handling. This is a significant gap for a financial tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at one sentence with zero wasted words, front-loading the core action. It efficiently conveys the basic purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a financial transfer tool with 3 parameters (0% schema coverage), no annotations, and no output schema, the description is incomplete. It lacks details on behavior (e.g., transaction confirmation), parameter usage, error cases, and output format, making it inadequate for safe and effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate but fails to do so. It mentions 'amounts' and 'recipients' implicitly via 'transfer...to addresses', but doesn't explain parameter meanings (e.g., amounts in mist, recipients as addresses), relationships (arrays must match), or the optional 'network' parameter with its enum values. This leaves key semantics undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'transfer' and resource 'SUI(in mist)' with scope 'to single or multiple addresses', which is specific and actionable. However, it doesn't distinguish this from potential sibling tools like 'faucet' (which might give SUI) or 'sui-balance' (which checks SUI), so it lacks explicit differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing SUI balance), exclusions (e.g., not for testing), or comparisons to siblings like 'faucet' for obtaining SUI or 'sui-balance' for checking funds.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gen_sui_accounts_by_mnemonicC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the tool creates accounts from a mnemonic and warns against production use, but it doesn't cover critical aspects like whether this is a read-only or write operation, what permissions are needed, rate limits, or what the output looks like. The warning adds some value but leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that conveys the core purpose and a key warning without unnecessary words. It's appropriately sized and front-loaded with the main action, though it could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (creating accounts, which implies a write operation), no annotations, no output schema, and low schema description coverage, the description is incomplete. It misses details on behavioral traits, output format, and full parameter semantics, making it inadequate for safe and effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 2 parameters with 0% description coverage, so the schema provides no semantic details. The description implies the 'mnemonic' parameter is used to generate accounts and hints at 'num' through 'accounts' (plural), but it doesn't explicitly explain what 'num' does or provide format details for 'mnemonic'. This adds marginal value but doesn't fully compensate for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Create' and the resource 'SUI accounts from mnemonic', making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'gen_mnemonic' or 'random-sui-account', which might also generate accounts or mnemonics in different ways.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a warning 'Not recommended for production', which provides some context on when not to use it, but it lacks explicit guidance on when to use this tool versus alternatives like 'random-sui-account' or 'gen_mnemonic'. No prerequisites or specific scenarios are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sui-balanceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a read operation ('Get'), which implies it's non-destructive, but doesn't cover other important aspects like network latency, rate limits, authentication requirements, error conditions, or what happens with invalid addresses. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose ('Get balance'), making it immediately clear. Every word earns its place by contributing to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, no output schema), the description is incomplete. It lacks details on return values (e.g., balance format, units), error handling, and behavioral constraints. For a financial tool querying blockchain networks, this omission is significant, as agents need to understand what to expect from the output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter descriptions. The description mentions 'address' and 'sui networks', which loosely map to the 'addresses' and 'network' parameters, but doesn't explain their semantics (e.g., that 'addresses' is an array, 'network' has enum values with defaults). It adds minimal value beyond the schema's structure, resulting in a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get balance') and target resource ('of an address from sui networks'), making the purpose immediately understandable. It distinguishes itself from siblings like 'sui-transfer' (which moves funds) and 'faucet' (which distributes funds), though it doesn't explicitly name these alternatives. The verb 'Get' is specific enough for a read operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid address), exclusions, or comparisons to sibling tools like 'get_account_info_by_private_key' (which might provide similar balance information). Usage is implied by the purpose but not explicitly defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• random-sui-accountB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the production warning but lacks critical details: what exactly gets created (e.g., key pair, address, mnemonic?), whether this consumes resources or has rate limits, what the output format is, or if there are any security implications beyond the production warning.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (two brief clauses) with zero wasted words. It front-loads the core purpose and follows with critical usage guidance. Every sentence earns its place by providing essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 0% schema description coverage, and no output schema, the description is incomplete for a tool that creates financial/cryptographic assets. It misses details about what constitutes an 'account' in this context, output format, security considerations, and how this differs from sibling tools. The production warning is helpful but insufficient for full understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, so the description must compensate. While it doesn't explicitly mention the 'num' parameter, the tool name and description imply creation of one or more accounts, and the default value of 1 in the schema provides some context. The description adds value by clarifying the random nature of generation, though it could explicitly note the parameter controls quantity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create random SUI account') and specifies the resource type. It distinguishes from siblings like 'gen_sui_accounts_by_mnemonic' by emphasizing randomness rather than mnemonic-based generation, though it doesn't explicitly contrast with all alternatives like 'faucet' or 'get_account_info_by_private_key'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear warning about when NOT to use this tool ('do not use it in production'), which is valuable guidance. However, it doesn't specify when TO use it (e.g., for testing, development, or demonstration purposes) or mention alternatives like 'gen_sui_accounts_by_mnemonic' for more controlled account generation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.