Server Quality Checklist

Tool Scores

• scan_vulnsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It only states basic scanning capability without disclosing accuracy, output format, or whether it modifies state. Minimal behavioral context is provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single well-formed sentence, front-loaded with the main action. It is concise, though could benefit from additional details without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool scans for vulnerabilities but the description does not explain return values, output structure, or limitations. Without an output schema, more context is needed for completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds a list of vulnerability types but does not enhance parameter understanding beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Scan' and specifies the resource 'Solidity source code', listing example vulnerabilities (reentrancy, overflow, access control). It is clear but does not explicitly distinguish from sibling 'audit_contract', which may perform broader analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives (e.g., audit_contract). It lacks exclusions or context about prerequisites or limitations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_holdersB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states 'analyze', which implies read-only, but does not confirm whether it is a safe read operation, whether it makes external calls, or any potential side effects. No mention of data freshness, auth needs, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that conveys the core purpose without any extraneous words. It is front-loaded and earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of analyzing token holder distribution for risk, the description lacks details about what constitutes a risk signal, how the analysis is performed, or the output format. With no output schema, the agent is left guessing about return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the schema already provides clear descriptions for both parameters ('Token contract address (0x...)' and 'Chain name: "ethereum", "base", "bsc", etc.'). The tool description adds no further meaning beyond the schema, so baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Analyze' and clearly identifies the resource 'token holder distribution' with the explicit purpose of assessing 'centralization and rug-pull risk'. This distinguishes it from sibling tools like 'audit_contract' (security audit) and 'scan_vulns' (vulnerability scanning).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, nor are prerequisites or exclusions mentioned. The description merely states the tool's function without contextualizing its use relative to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_contractA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; the description only says 'fetch', implying a read operation. It lacks disclosure of potential rate limits, authentication needs, or whether the data is always up-to-date. Minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that efficiently communicates the core purpose. No filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description states the output (source code and ABI) but does not specify the format (e.g., JSON) or potential error conditions. For a simple fetch tool, it is mostly adequate but could include a hint about the response structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters have clear descriptions in the input schema (address and chain). The description does not add any extra meaning beyond what the schema already provides, such as format or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'fetch', the resource 'verified smart contract source code and ABI', and the data source 'Blockscout', distinguishing it from sibling tools like audit_contract.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving smart contract code and ABI, but does not explicitly state when to use this tool versus alternatives (e.g., for audits use audit_contract). No when-not-to-use advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_contractA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It lists the steps but does not disclose behavioral traits like time requirements, destructive actions, authentication needs, or rate limits. For a 'full audit', more transparency is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with emoji, efficiently conveying the tool's purpose. It is front-loaded and brief, though it could benefit from structured bullet points for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema or annotations, the description should detail what the report contains. It only mentions 'generate report' without specifying format or how results are presented, leaving gaps for a complex multi-step tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear descriptions (address, chain list, include_holder_analysis default). The description adds context but not new parameter-level info beyond schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it's a full smart contract audit combining multiple sub-tasks (fetch source, scan vulnerabilities, analyze holders, generate report). It distinguishes from siblings like check_holders, fetch_contract, scan_vulns by being the one-stop solution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for a comprehensive audit, with sibling tools for individual steps. However, it doesn't explicitly state when to use alternatives or provide exclusions, leaving some inference to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_chainsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided. The description discloses that the tool lists chains, which is a safe read operation, but does not mention any potential behaviors like authentication needs or limitations. Given the simplicity, it is adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear 7-word sentence with no superfluous information. It is optimally concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description sufficiently explains its purpose. It could hint that listing chains is a prerequisite for other chain-specific tools, but this is not essential.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and schema coverage is 100%. The description does not need to add parameter details, meeting the baseline for no-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'list', the resource 'all supported EVM chains', and the context 'for contract auditing'. It effectively distinguishes from sibling tools like audit_contract or scan_vulns, which operate on specific chains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives. Usage is implied but not clarified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.