Server Quality Checklist

Tool Scores

• wais_executeA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses that tokens and DPoP proofs are handled internally and that requests route based on api_base_url, but does not mention side effects, error handling, or rate limits, leaving gaps in behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement, mechanism explanation, parameter mapping, and examples. It is concise, with no wasted words, and front-loaded for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description need not explain return values. It covers dependencies, parameter semantics, and provides examples. However, without annotations, it could elaborate on edge cases or error behavior, but overall it is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It explains each parameter: site_url is from site.url in agents.json, action_id is the action id, params should match input_schema. Examples clarify usage, adding significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Execute an action at a WAIS-compatible site,' clearly identifying the verb (execute) and resource (action at a WAIS site). It differentiates from sibling tools by explaining it uses agents.json from wais_discover to resolve endpoints, methods, and scopes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains prerequisites (agents.json from wais_discover) and parameter mapping, providing context for when to use the tool. However, it does not explicitly state when not to use it or name alternatives, so there is room for improvement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wais_registerA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the use of SD-JWT selective disclosure, the sharing of only required claims, and the optional claims parameter behavior. It lacks details on error scenarios or potential side effects, but the behavioral traits are sufficiently transparent for a registration tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, at roughly 8 lines, with a clear structure: introduction, behavioral note, prerequisite, and parameter explanations. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no enums, no nested objects) and the presence of an output schema, the description covers all necessary aspects: purpose, usage order (wais_discover first), parameter semantics, and default behavior. It is complete for the intended use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains site_url as 'The site.url from agents.json' and claims as 'Claims to disclose (e.g. ["email"]). If not provided, uses required_claims from agents.json automatically.' This adds meaningful context beyond the bare schema titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Register the user at a WAIS-compatible site using their WAIS identity' with specific verb and resource. It distinguishes from sibling tools like wais_discover by explaining the registration action and the use of SD-JWT selective disclosure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises 'Call wais_discover first to see what claims the site needs,' providing a clear prerequisite. It also explains the default behavior when claims are omitted, offering good context for when to use the tool versus relying on defaults.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wais_confirmA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool polls for completion automatically if the challenge includes a resolution object. It implies a write/confirmation action but does not detail failure modes or side effects. The polling behavior adds transparency, raising the score above baseline.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is fairly concise, with a clear header sentence followed by conditions and parameter explanations. It could be slightly more streamlined by integrating the 'Call after' condition more tightly, but overall it is well-structured and front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, the description covers the basic behavior, preconditions, and parameter sources. It mentions automatic polling, which is a key behavioral detail. The presence of an output schema (not shown but indicated) supplements the completeness. Could mention error handling or what to do if the challenge is already confirmed, but the description is largely adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description provides detailed explanations for both parameters in the Args section: site_url comes from agents.json, challenge_id from the 402 response. This adds significant meaning beyond the raw schema, which only lists names and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool confirms a high-risk action or completes a payment challenge, specifying the context: after wais_execute returns a 402 confirmation challenge and user approval. The verb 'confirm' is specific and distinguishes it from sibling tools like wais_execute (preceding step) and wais_discover/wais_register/wais_status (different purposes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to call: after wais_execute returns a 402 confirmation challenge and the user has approved. Also mentions that if there was a payment link, payment must be completed first, and describes automatic polling if a resolution object exists. This provides clear context and conditions for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wais_discoverA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully informs the agent that it fetches /.well-known/agents.json and returns a summary including site URLs, actions, parameters, and registration info. It discloses that the agents.json is the source of truth, though it does not cover potential failure cases or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: it starts with the purpose and usage, then briefly describes the fetch action and returned fields, and ends with the parameter description. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity with one parameter and an output schema, the description is fully complete. It explains what the tool does, what it returns, and how to use the results with sibling tools, leaving no gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description compensates by explaining the site_url parameter: 'The site's URL (e.g. 'https://serphub.deeger.io').' This adds essential meaning beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Discover what a WAIS-compatible site offers. Call this first for any site.' It uses a specific verb (discover) and resource (WAIS site capabilities), and distinguishes itself from sibling tools like wais_execute by directing the agent to use this tool first.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Call this first for any site' and explains that the agents.json is the source of truth for action IDs used in wais_execute. While it does not mention when not to use it, the context implies this is the initial step, providing clear usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wais_statusA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must suffice. It discloses that the tool is read-only (check status), details its return (plan, credits, usage stats), and reveals an internal behavior (looking for specific actions). It doesn't mention idempotency or rate limits, but the core behavior is well-covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with a clear first sentence stating the main action, followed by essential details in two more sentences. No redundant or ambiguous phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (single parameter, no nested objects) and presence of an output schema, the description sufficiently covers prerequisites, behavior, and parameter guidance. It leaves no critical gaps for a standard status check.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has only a title for site_url; the description adds full meaning by explaining it should be the site.url from agents.json, which completely compensates for the 0% schema coverage and clarifies the expected value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks user account status at a WAIS-compatible site, returning specific data (plan, credits, usage stats). It distinguishes itself from sibling tools like wais_register (registration) and wais_execute (execution), making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a prerequisite ('user must be registered first') and explains the internal lookup in agents.json, providing practical context. While it doesn't explicitly list alternatives or when to avoid usage, the purpose is distinct enough from siblings to guide appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.