Server Quality Checklist

Tool Scores

• check_custom_domainA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, destructiveHint, idempotentHint, and openWorldHint. The description adds value by specifying the return condition (ready:true) and the need for authentication, without contradicting the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences that are succinct and front-loaded. Every sentence provides essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple tool with 2 parameters, no output schema, and good annotations, the description covers the main purpose, return indicator, and authentication requirement. It could mention potential error conditions but is generally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with detailed parameter descriptions. The description adds minimal extra meaning beyond the schema, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Check' and the resource 'custom domain provisioning', specifying the return condition 'ready:true once the site is serving'. It distinguishes itself from siblings like add_custom_domain and site_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates the requirement for an API key and mentions fallback to environment variable, providing clear context for usage. Although no explicit exclusions are given, the purpose is specific enough to guide when to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rollback_siteA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds value beyond annotations: it states the operation is instant, requires authentication via API key, and is limited to owned sites. Annotations include idempotentHint=true, and the description aligns with this by implying the result is a deterministic state. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences, no extraneous words. It front-loads the primary purpose and follows with key details. Every sentence contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the schema fully documents parameters and the output schema is absent, the description adequately covers the tool's behavior, prerequisites, and effects. It explains the version concept and the instant nature of the rollback.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% description coverage for all parameters. The description echoes the versionId parameter but does not add new semantic meaning beyond the schema. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('roll back'), the resource ('previously-published version of a site'), and the effect ('live again, instantly'). It distinguishes from sibling tools like publish_site by specifying it's a rollback operation, not a new publish.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use this tool ('to roll back to a previously-published version') and mentions a prerequisite ('requires an API key (owned sites only)'). While it doesn't explicitly say when not to use, the sibling tools provide context for alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• site_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint true and destructiveHint false. The description adds context by specifying the exact status fields returned and repeating 'Read-only', which reinforces the safety profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. Front-loaded with the verb 'Look up' and immediately specifies the resource and input. Ideal conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all key information for a simple status lookup: what it does, what it returns, and the input. Could mention error behavior if slug not found, but not a major gap given simplicity and annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the slug parameter already has a detailed description with example. The tool description adds minimal extra meaning beyond explaining the context of the parameter in the tool's purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it looks up a published site by slug and lists specific output fields (live, anonymous/owned, expiration, URL). It distinguishes from siblings like add_custom_domain and publish_site which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It tells when to use (to look up site status by slug) and notes the tool is read-only. However, it does not explicitly state when not to use or mention alternatives like the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_custom_domainA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds value beyond annotations by explaining the return value, required DNS pointing, auto-issuing certificate, and the need for polling with check_custom_domain. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with no fluff, each providing essential information: action, follow-up, and requirement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 simple parameters and no output schema, the description covers the return value, next steps, and prerequisites adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description repeats the API key requirement already in the schema. No additional parameter details are provided beyond what the schema offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Attach' and resource 'custom domain', distinguishing it from sibling tools like check_custom_domain which is mentioned for polling readiness.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on when to use (attaching a domain) and mentions a follow-up action (poll with check_custom_domain), but lacks explicit when-not or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• publish_siteA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate non-read-only and non-idempotent; the description adds lifecycle details (24-hour lifetime, claimUrl for permanence) and update behavior, though it doesn't explicitly warn that repeated calls without identifiers create new anonymous sites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences deliver maximum information: purpose, anonymous workflow, and update rule—no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 7-parameter tool with no output schema, the description covers key workflows (publish, update, anonymous vs owned) and mentions the response includes a claimUrl, but could detail other response fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters (100% coverage), but the description adds logical grouping ('files or dir'), usage context for claimToken/slug, and environment fallback for apiKey, improving understanding beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Publish a website to shiply.now') and the resource (website), with explicit differentiation from sibling tools via the publishing scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use (publishing a site) and when not to ('NEVER create a new site to update an existing one'), including how to update with claimToken or slug.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.