Server Quality Checklist

Tool Scores

• setup_cli_credentialsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the core side effects (creates an IAM user, writes credentials), but does not reveal whether repeated calls overwrite existing credentials, or what happens if called prematurely. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two focused sentences: the first states the verb and resource, the second gives the usage trigger. Every word earns its place, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the tool's purpose, the effect on the CLI, and the correct timing relative to check_claim_status. It lacks details on return values or failure modes, but for a one-time setup tool with strong contextual anchor, it is adequately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with each parameter already carrying descriptive meaning. The description adds no new parameter-level detail beyond what the schema provides, maintaining the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Provision') and names exact resources (S3 IAM credentials, ~/.xns/credentials), immediately distinguishing this tool from siblings like check_claim_status or install_relayer. It clearly states the outcome (S3 verbs work without further configuration).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides an explicit trigger condition: 'Call once after check_claim_status reaches STATE_3.' This is clear contextual guidance on when to invoke the tool, though it does not mention alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_claim_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes on the responsibility of disclosing behavior. It reveals that the tool polls every 10 seconds, lists the meaning of each state, and states that it automatically proceeds upon reaching STATE_3. This goes beyond the basic 'poll status' phrasing, though it does not detail timeout error behavior or return format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded. The first sentence states the purpose, and the second sentence efficiently enumerates states and the auto-proceed behavior. Every sentence contributes necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the polling interval, state meanings, and auto-proceed behavior, providing a good overall context. However, it does not explain what happens if the timeout_ms expires (e.g., whether it returns an error or a timeout state), which is a minor gap for an agent invoking a long-polling operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear descriptions for claim_id and timeout_ms. The description itself adds no direct parameter semantics but the schema already provides sufficient information, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Poll the status of a claim session.' It further distinguishes the tool by detailing the session states (STATE_1, STATE_2, STATE_3), which ties directly to sibling tools like start_claim and check_relayer_health, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after start_claim by referencing claim sessions and the 'claim_id returned by start_claim' in the schema. It does not explicitly mention alternatives or exclusions, but the polling context and state definitions make it clear when this tool should be used (to wait for claim completion).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• install_relayerA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral disclosure. It honestly details side effects: fetching from releases.scpri.me, writing a .env, running docker compose up -d, and falling back to a bundled copy if the fetch fails. It also notes the user doesn't need to author files, which is a useful behavioral cue.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the primary action, and uses a few sentences to pack essential operational details without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex installation tool, the description covers the source, the installation mechanism, fallback behavior, and the override path. It lacks explicit mention of port defaults or prerequisites, but the schema supplies port defaults and no output schema is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes all four parameters with 100% coverage, so the baseline is 3. The description adds value by clarifying compose_url as an override, but does not deepen the meaning of s3_port, ui_port, or install_path beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Install and start the XNS Relayer' using a specific verb and resource, distinguishing it from sibling health/restart/status tools. It also specifies the default bundle and monitoring stack, making the tool's scope unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explains the default behavior (canonical beta bundle, anonymous pull) and when to use the override ('Pass compose_url only to override with a custom compose'). However, it does not explicitly mention any exclusions or alternatives among the sibling tools, such as checking prerequisites with check_prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_backupsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly warns that restore 'OVERWRITES current state and restarts services' and advises confirming with the operator, which is critical for a potentially destructive operation. It does not mention deletion permanence, but the restore warning is a strong disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with each sentence adding value: the first enumerates actions, the second provides a critical safety warning, and the third states a prerequisite. It is well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity and lack of output schema, the description covers the main operations, key safety warnings, and prerequisites. It doesn't explain return values, but this is not essential for a backup management tool. The description is adequate for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides full descriptions for all parameters (action, file, components) with 100% coverage. The description adds a bit of practical guidance (e.g., 'state which archive and components first'), but it doesn't substantially augment the schema's parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs: 'list archives, start a backup, restore from an archive, or delete one.' It identifies the resource (Relayer configuration backups) and distinguishes the tool from siblings that handle other operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: it is used for managing backups, and it explicitly notes a prerequisite ('Backups must be enabled'). However, it doesn't explicitly contrast with alternative tools or state when not to use it, so it falls short of full usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_email_verifiedA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the polling interval, timeout duration, early return condition, and the no-account case. These are meaningful behavioral traits that go well beyond the tool's name and cannot be inferred from the schema alone.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the primary purpose. Every sentence contributes essential information: polling mechanics, immediate return condition, and the no-account edge case. No filler, repetition, or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple polling tool with no output schema and no annotations, the description covers the key scenarios: polling, early verification, timeout, and no-account. It does not specify the exact return format, but given the simplicity and lack of an output schema, this is a minor gap. It is adequately complete for an AI agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters. The description adds context about the polling behavior (every 15 seconds, up to 30 minutes), which elaborates the 'poll' parameter's meaning and behavior. This adds value beyond the schema without redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: polling to check email verification status. It uses a specific verb ('Poll to check') and resource ('user's email address'), and it explains distinct outcomes (returns immediately if verified, indicates registration if no account). However, it does not explicitly contrast with sibling tools like check_prerequisites or check_claim_status, though its unique subject matter makes the purpose understandable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: it polls every 15 seconds up to 30 minutes, returns immediately if already verified, and indicates when registration is needed. This implies when to use it and hints at an alternative (registration). It stops short of explicitly naming alternative tools or stating when not to use it, which would have earned a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• restart_serviceA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of safety disclosure. It explicitly warns that the operation is disruptive and that in-flight S3 requests will fail, which is critical behavioral context. It does not detail every possible side effect, but the primary risk is clearly communicated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main action and followed by essential risk and recovery guidance. Every sentence earns its place without redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter restart tool with no output schema, the description covers the action, target options, disruptive impact, operator confirmation, and verification step via a sibling tool. It is complete within its context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the only parameter, service, is already fully documented with an enum and default. The description repeats the service list but does not add new parameter-level detail beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (restart) and the resource (Relayer services), lists the valid service targets, and distinguishes from sibling tools by focusing on restart rather than health checks or installation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit preconditions: confirm with the operator before calling, and directs verification with check_relayer_health afterward. It does not explicitly state when not to use the tool or name alternatives, but the usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_storageA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behavioral traits: it creates a test bucket, uploads/downloads an object, and compares. It also reveals a practical requirement for fullaccess credentials and explains endpoint auto-detection with remote host support. However, it doesn't disclose whether the test bucket is cleaned up after the test, which is a notable omission for a tool with side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, with no redundant sentences. It front-loads the purpose, then adds credential and endpoint details in a logical order. Every clause contributes useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main purpose, credentials, and endpoint behavior but omits cleanup behavior and failure semantics. Given the tool creates a test bucket, the lack of cleanup information leaves an important gap. Overall, it's adequate but not fully complete for a side-effectful verification tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents all three parameters with descriptions (100% coverage). The description adds critical context: endpoint default and override conditions, and the exact credential type ('fullaccess' from Relayer UI IAM, not read-only/bucket-scoped). This enriches the schema meaning and helps agents pick appropriate values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Verify the S3-compatible storage gateway is working by performing a round-trip test...' It specifies the verb, resource, and method, distinguishing it from sibling tools like check_relayer_health.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context: use this when you need to verify the storage gateway, not just service health. It also specifies when to override the endpoint ('when auto-detection cannot reach the host') and explicitly states the required credential type. However, it doesn't name alternative tools for different checks, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• configure_vpdA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses important behavioral traits: dry_run preview does not apply changes, requires OIDC sign-in, and enforces minimum host thresholds. It does not detail side effects such as persistence or service restart, but covers key behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the main action, and no redundant information. Each sentence adds meaning: purpose, configuration syntax, and prerequisites/constraints.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a configuration tool with no output schema, the description covers the core aspects: purpose, usage, constraints, and auth. It lacks explicit return value or post-conditions, but is sufficient for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds practical value by showing default values ('true') and explaining CEL expressions filter by tags, plus ties host count constraints to the expression parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool configures VPD host selection for the Relayer, with an optional dry_run preview. It distinguishes itself from sibling tools by focusing on host selection, a unique function not covered by other listed tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context on when to use: configuring host selection with defaults or CEL expressions, and notes the minimum host count requirement to guide criteria. It mentions the OIDC sign-in prerequisite but does not explicitly contrast with alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_host_tagsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the OIDC sign-in requirement on first use and the output transformation rule ('never show raw CEL expressions'), which are important behavioral traits. It could mention whether the operation is strictly read-only, but the verb 'Get' and the nature of the tool imply it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences: purpose, output behavior, and authentication note. Each sentence is necessary and informative, with no fluff. The most important information (what the tool does) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no params, no output schema), the description covers purpose, output format, and auth. It could specify the return structure more explicitly, but it adequately explains what the operator will receive and how to handle it. The sibling context (configure_vpd) reinforces its role as a prerequisite helper.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty, so there are no parameter semantics to clarify. The description adds value by explaining how the results are presented (plain-language options), which is more about output than parameters. The baseline for 0 params is 4, and the description does not need to do more.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action and resource: 'Get the available host tags for VPD configuration, plus the currently applied VPD host selection.' This is a specific verb + resource and distinguishes it from configure_vpd and other siblings by focusing on retrieval and current state.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context by tying the tool to VPD configuration and indicating it retrieves both available tags and the current selection. It does not explicitly name alternatives or exclusions, but the purpose is positioned as a prerequisite step, which is sufficient for a zero-parameter informational tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• register_accountA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so description carries the burden. It discloses that registration triggers a verification email, requires user action, and suggests polling. It also notes the existing-account condition. It could mention error responses or idempotency, but the workflow is well covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each serving a distinct purpose: purpose, post-registration requirement, and alternative paths. No filler or redundancy; information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-param tool with no output schema, the description covers the registration flow, verification step, and conditional skip. It lacks explicit error messages or return values, but is complete enough for an agent to act correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage for both parameters with descriptions (email format, password min length). Description only restates 'email and password' without adding extra meaning beyond what schema provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Register' with the resource 'new XNS account' and explicitly mentions email/password. It distinguishes itself from sibling tools by directing to check_email_verified for polling and install_relayer for existing accounts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says when to use the tool (to register), what to do after (verify email, poll with check_email_verified), and when to skip (if email already has an account, go to install_relayer). This provides clear context and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_claimA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior, and it does: it notes the tool returns a claim URL that requires browser action, mentions the expiration time, and points to the monitoring tool. This gives the agent a clear model of the tool's side effects and follow-up actions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four short sentences, with the main action first and no filler. Every sentence adds essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter tool without an output schema, the description covers the core flow: start, get URL, user action, expiry, and monitoring. It doesn't discuss failure handling, but the absence of parameters and the straightforward session flow make the description sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema already reflects this. The description adds no parameter-specific details, but with 0 params, a baseline of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Start a claim session to link this Relayer installation to an XNS account,' clearly stating the verb, resource, and goal. It distinguishes itself from the sibling check_claim_status by indicating it's the initiation step that returns a URL.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly directs the user to 'use check_claim_status to monitor claim progress' after calling, providing a clear sequential guideline. It doesn't explicitly state exclusions or prerequisites, but the context makes the primary use case obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_settingsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the return structure (current value, default, type, guidance per setting) and the authentication requirement (OIDC sign-in). While it does not explicitly state that the operation is read-only, the verb 'List' and the context of inspecting settings before updating imply a safe, non-mutating behavior. This is adequate transparency for a listing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long and front-loaded with the core purpose. The first sentence states what the tool lists and returns; the second provides usage timing and auth requirements. Every word earns its place—no fluff, no repetition of schema (which is empty).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description covers the essential aspects: what the tool does, what it returns, when to use it, and what prerequisites exist. It is sufficiently complete for a simple listing tool with no parameters. The inclusion of return detail (value, default, type, guidance) compensates for the missing output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4 per the rubric. The description adds value by naming the settings covered, which helps the agent understand the scope of the output, even though no input is needed. Since there are no parameters to document, the description's mention of the settings is a bonus.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and clearly identifies the resource as 'adjustable Relayer settings', enumerating categories (worker/concurrency tuning, backup schedule, cost center). It distinguishes itself from siblings by explicitly stating 'Use before update_settings', making its role as a read-only precursor clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance to use 'before update_settings', directly linking this tool to its sibling. It also notes the OIDC sign-in requirement. However, it does not mention scenarios where this tool should not be used or alternatives beyond update_settings, so it lacks exclusions but provides clear use context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_prerequisitesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly discloses a read-only action ('Check') and lists all the checks performed, including network endpoints and ports. However, it does not state the return format or whether the check may require authentication (e.g., Docker credentials), leaving some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the primary purpose, followed by a detailed enumeration of checks and a clear usage directive. Every sentence earns its place; the first sentence is dense but informative, the second provides critical workflow guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 params, no output schema, no annotations), the description provides the essential context: what is checked and when to run it. It does not describe the output or success criteria, which the agent would need to interpret results, so it is not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing to explain. The schema coverage is 100% because the schema is empty. The description adds no parameter information since none exist, which is appropriate; the baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Check') and resource ('system prerequisites for XNS Relayer installation'), and enumerates specific checks (Docker, ports, existing installation, disk space, network connectivity). This distinguishes it from sibling tools, as it is the prerequisite check before any other relayer operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Run this first before any other relayer tool,' providing a direct and unambiguous usage directive. This tells the agent exactly when to use this tool in the workflow, though it doesn't need to list alternatives because this is the entry point tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_relayer_healthA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses key behaviors: polling interval/timeout, per-component status reporting, degraded state handling, Docker context auto-detection with host override, and the OIDC caveat for HostIO. This gives the agent a complete understanding of side effects and edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, front-loaded with the core purpose, and every clause adds value—polling details, status reporting, degraded state, and host detection. No fluff or redundancy is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains what the tool returns (per-component status, unhealthy components named) and important caveats (degraded monitoring stack, HostIO unknown until OIDC). It covers the essential information an agent needs to invoke and interpret the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed parameter descriptions, so the baseline is 3. The description adds meaningful context beyond the schema by specifying the polling interval (10s) and timeout (300s), and explaining the impact of the polling behavior on the 'poll' parameter. It also reinforces the default ports and host override, adding nuance not fully captured in schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific action ('Check the health') and enumerates the exact resources (UI on port 8888, S3 gateway on 9000, HostIO, and monitoring sidecars). This clearly distinguishes it from sibling tools like check_prerequisites or check_claim_status, which target different concerns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context on when to use the tool (health checks during/after installation) and important conditional behavior (degraded monitoring stack does not block install flow). However, it does not explicitly mention alternatives or when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_settingsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses the require_restart return flag, the need for restart_service, and the re-billing side effect of CostCenter changes. This is strong, though it doesn't detail all potential side effects or the timing of changes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no fluff: purpose, example, return behavior, and a warning. Information is front-loaded and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description covers the return value (require_restart), the follow-up restart_service action, and a special-case confirmation for CostCenter. It is self-sufficient when paired with describe_settings, and the sibling list reinforces the workflow context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds a concrete example (HOSTIO_UPLOAD_WORKERS: 20) and clarifies the map format, which enriches the schema's generic definition and gives the agent a clearer invocation pattern.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Update' with the resource 'Relayer settings' and explicitly directs users to describe_settings for the allowed set. This clearly distinguishes it from sibling tools like describe_settings (read-only) and restart_service.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit when-to-use guidance: consult describe_settings for allowed settings, follow up with restart_service if require_restart is true, and always confirm before changing CostCenter. This gives clear context and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.