Server Quality Checklist

Tool Scores

• build_contractA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds value by specifying that output is structured compiler diagnostics rather than raw rustc text, which is a meaningful behavioral trait. However, it does not disclose potential side effects such as file system changes, network usage, or permission requirements, leaving notable gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, with the first stating the action and the second clarifying the output format. Every sentence provides essential information, and there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately explains the tool's core purpose and output type. For a build tool with only two parameters and no output schema, it covers the essential usage context. Minor gaps exist (e.g., side effects or success behavior) but the tool is sufficiently described given its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides full descriptions for both parameters (network enum/default and project_path with Cargo.toml context), achieving 100% coverage. The description does not add additional parameter semantics beyond mentioning the Stellar CLI, so it aligns with the baseline of 3 for fully covered schemas.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Builds' and identifies the resource as 'a Soroban smart contract project using the Stellar CLI.' It clearly distinguishes itself from sibling tools like deploy_contract or invoke_contract by focusing on the build stage. The note about returning structured diagnostics further clarifies its unique purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus its siblings. It implies its role as the build step (before deployment) but lacks direct guidance on prerequisites or exclusion criteria. Users must infer usage from the tool name and sibling context, making it decent yet not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_contract_stateA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context by noting keys and values are decoded to human-readable strings where possible. However, it does not mention pagination behavior, network-specific nuances, or potential side effects, leaving gaps given the lack of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action and resource, and contains no unnecessary details. It is concise and well-structured for a read tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool and complete schema coverage for all parameters, the description adequately covers the tool's purpose and the key behavior of decoding values. It does not describe the return format in detail, but the phrase 'returns storage entries' is sufficient for a straightforward read operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameters are fully described in the schema. The description adds no additional parameter-specific meaning beyond what is already in the schema, such as the meaning of limit or network.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads and returns storage entries for a Soroban contract, using a specific verb and resource. It distinguishes from sibling tools like invoke_contract or deploy_contract by focusing on reading state rather than executing or deploying.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to inspect contract storage, but it does not explicitly state when to use it versus alternatives or provide exclusions. No comparison is made with sibling tools such as get_account_info or invoke_contract.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deploy_contractA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description adds some context by noting the returned values and mainnet unsupported, but it omits critical behavioral traits: deployment is a paid, state-changing transaction requiring a funded source account, and it may incur fees. The description does not disclose these side effects or prerequisites, leaving a significant transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, using only two sentences: the first states the primary action, and the second covers return values and an important network restriction. Every word earns its place, and it is front-loaded with the deploy action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters and no output schema, the description covers the main action, returns, and network exclusion. However, it lacks explicit prerequisites like 'wasm must be compiled' and 'source account must be funded', which are important for correct invocation. The schema partially compensates, but the description itself is not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% parameter descriptions, so the description does not need to elaborate on parameters. The description's mention of 'using the Stellar CLI' and 'source account' is redundant with the schema, but the schema already covers each parameter's meaning, warranting the baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool deploys a compiled .wasm file to testnet or futurenet, using a specific verb and resource. It distinguishes itself from sibling tools like build_contract or invoke_contract, and also mentions the return values (contract ID and transaction hash).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates the target networks and explicitly excludes mainnet, providing clear usage boundaries. It implies usage after building a wasm file, but does not explicitly name alternatives or when not to use this tool beyond the mainnet exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_account_infoA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavior disclosure. It adds a notable behavior (testnet funding link when account is not found) but omits other potential behaviors like rate limits, error formats, or permission requirements. For a read-only getter, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with the primary purpose stated first and the special behavior second. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple account query tool with two parameters, the description covers the returned fields and the not-found behavior. Since there is no output schema, listing the return contents is helpful. Minor gaps exist (e.g., network-specific behavior details), but overall it is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with clear explanations for both 'public_key' and 'network'. The tool description adds no new parameter semantics beyond what the schema already provides, resulting in the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states what the tool returns: balance, sequence number, and signers for a Stellar account. The specific verb 'Returns' combined with the resource and result set clearly distinguishes it from sibling contract operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies usage for retrieving Stellar account information, and the sibling tools are contract-related, so the context is clear. However, it does not explicitly mention alternatives or exclusions, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_testsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the return format (pass/fail report with test name, status, duration, failure message), but it does not mention potential side effects like compilation, dependency resolution, or behavior on test failure. With no annotations, the description carries the full burden and leaves some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that includes the command, purpose, and return value. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple test-runner tool with documented parameters and an explicit return format in the description, the definition is complete. No output schema is needed, and the tool's behavior is adequately scoped.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes both parameters, including the optional filter. The description adds no additional parameter semantics, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs the Soroban contract test suite via `cargo test` and returns a structured report, distinguishing it from build, deploy, invoke, and get-state siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context that the tool is for running tests, but it does not explicitly mention when not to use it or name alternative tools. The purpose and sibling names imply the right usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• invoke_contractA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses key behaviors: simulation via RPC, return of function result and resource costs (CPU, memory, min fee), and signing only needed for non-read-only calls. This goes beyond a simple 'invokes a function' and addresses important operational aspects, though it does not explicitly rule out on-chain side effects or detail error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the central purpose, and every clause earns its place. It efficiently communicates the simulation nature, return value composition, and a key usage condition without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simulation tool with no output schema, the description sufficiently explains what the tool returns and the signing context. However, it omits prerequisites such as the contract being deployed or the need for a network connection, and it could clarify that simulation does not submit an on-chain transaction. These gaps prevent a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the input schema fully documents each parameter. The description adds minimal parameter-specific meaning beyond mentioning signing in the context of read-only calls, which is also covered by the source_account parameter description. At the baseline for full schema coverage, this score is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Simulates') and resource ('Soroban contract function call via RPC'), making the tool's purpose immediately clear. It distinguishes from siblings like deploy_contract or get_contract_state by focusing on function invocation simulation, not deployment or direct state reads.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context that this is a simulation, not actual execution, and notes the signing requirement for read-only calls. However, it offers no explicit comparison to alternatives (e.g., when to use get_contract_state instead) or exclusions, so it slightly misses the highest bar.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.