Server Quality Checklist

Tool Scores

• directory_statsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, and the description adds content details (services, verification breakdown, data freshness) beyond the annotations. It does not contradict annotations, but it omits any potential side effects, performance characteristics, or required permissions—though these are less critical for a read-only stats endpoint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the main function and lists key data categories. Every word adds value, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no input parameters and no output schema, the description covers the essential information: it returns index-level stats about services, verification, and freshness. It could be enhanced by specifying the return format or exact fields, but the simplicity of the tool makes this adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters in the schema, so the description has no burden to explain them. With 0 parameters, the baseline is 4, and the description appropriately focuses on what the tool returns rather than parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides index-level statistics covering services, verification breakdown, and data freshness. The resource (index) and data types are specified, and it distinguishes from sibling tools that focus on individual services or wallets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for aggregate statistics ('index-level'), which contrasts with sibling tools for specific lookups, but it does not explicitly advise when to choose this tool over alternatives or mention exclusions. The writer expects the agent to infer from the context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_paid_serviceA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the description does not need to restate that. It adds useful behavioral details: results are unified per host, each rail carries separate reputation/price/retention, and 'coming_soon' lists unmeasured chains. This goes beyond annotations and helps the agent anticipate output semantics. No contradiction detected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is roughly 100 words and each sentence earns its place. It front-loads the core purpose, then adds critical result-structure context, and ends with a practical recommendation. While slightly dense, it is well-organized and not bloated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description compensates by explaining the result structure (unified per host, per-rail metrics, coming_soon). It also covers the important caveat about verified status. Given the schema covers all parameters and the tool is a search, the description is sufficiently complete for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with each parameter having a description. The description adds extra meaning for 'chain' (explains matched_rail and per-rail reputation) and for 'verified_only' (advisory to prefer verified true). This exceeds the schema baseline, providing richer guidance for key parameters, though not all 11 are individually expanded.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Search' and the resource 'Verantis's verified directory of machine-payable services'. It distinguishes from siblings: get_service likely retrieves a specific service, directory_stats likely provides statistics, and check_wallet likely checks wallet info. The purpose is unambiguous and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers practical guidance: 'Prefer verified=true before paying anyone' and explains how to use 'chain' to see reputation for the actual payment rail. It implies this is the directory search tool but does not explicitly contrast with siblings (e.g., when to use get_service for a known host). Clear usage context is present, but exclusionary guidance is missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_serviceA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already indicates a safe read operation, and the description aligns with that by mentioning returning a record. It adds detail about the content but no new behavioral caveats.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct, two sentences, with no redundant information. It efficiently conveys the purpose and parameter usage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers what the tool does and the required parameter. It does not mention return format, errors, or pagination, but for a simple record retrieval this is likely sufficient given the domain is well-defined.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'domain' is clearly explained as the host/domain to look up, and the description reinforces this with an example (api.nansen.ai). The schema also provides a description, so meaning is fully covered.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a full unified record for a service, listing specific details like settlement rails, reputation, price, buyers, volume, and coming-soon chains. This distinguishes it from sibling tools like find_paid_service or directory_stats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides parameter usage ('Pass the host (domain)') but does not explicitly state when to use this tool versus alternatives. However, the purpose is specific enough that an agent would infer when to call it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_walletA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With readOnlyHint and openWorldHint already present, the description expands substantially on what the agent can expect: reputation tier, human-readable reasons, settlement rail, host, buyer retention, other rails, and the shared_wallet caveat. It also implicitly explains the tool's informational, non-mutating nature by framing it as a guard before payment.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average but front-loads the core purpose and then efficiently packs relevant behavioral details. Every sentence earns its place: pre-payment use, return content, rail semantics, shared_wallet hazard, and the risk warning. It could be tightened slightly, but all content is useful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description gives a thorough mental model of what will be returned, including reputation, human-readable reasons, settlement rail, host, customer-retention metrics, other settlement rails, and the shared_wallet meaning. It is complete for an agent to know exactly when this tool answers and how to interpret its result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single pay_to parameter, including format for EVM and Solana addresses. The description adds crucial semantic detail beyond the schema: pay_to is the service-provided settlement address, and the tool will return wallet reputation across settlement rails. This is more than the schema alone provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear, specific purpose: 'Check a recipient wallet BEFORE your agent pays it — a pre-payment guard.' This distinguishes it from sibling tools like get_service and find_paid_service by targeting the wallet of the service, not the service itself. It also states exactly what it returns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage context: pass the pay-to address a service asked you to pay and do this before payment. It also gives decision guidance, such as treating shared_wallet results with care and pausing for unknown/low-reputation recipients. It does not explicitly rule out alternative tools, but the sibling tools are not relevant alternatives for wallet checks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.