Server Quality Checklist

Tool Scores

• list-projectsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the agent knows it is a safe, non-destructive read operation. The description goes beyond by providing extensive context about what projects are (sota.io, GDPR, auto-URL, PostgreSQL, etc.), which helps the agent understand the domain. It does not disclose any hidden behavioral traits beyond what annotations cover, but it adds useful context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence for the core purpose, followed by a long sentence detailing sota.io features. The core action is front-loaded and clear. The additional information, while somewhat lengthy, provides valuable context for the agent. It is not overly verbose for the value it adds.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, no output schema, and rich annotations, the description is complete. It explains what the tool does, the domain context, and the nature of the projects listed. There are no gaps in understanding what the tool returns or requires.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with 0 parameters, so the description has nothing to add beyond what the schema already shows. Baseline 3 is appropriate here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'list' and the resource 'projects on your sota.io account', which is specific. It distinguishes itself from siblings like create-project and delete-project, but it does not explicitly distinguish from other list tools like list-domains, so it is not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is a straightforward listing tool with no parameters, suitable for when you need an overview of all projects. However, it provides no guidance on when to use it versus other tools (e.g., get-status for a specific project), nor when not to use it. The context is clear but lacks explicit usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-domainA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds value by explaining the domain status lifecycle (pending→verified→active) and actionable advice for pending domains ('configure the DNS record as shown in the instructions'), which goes beyond the annotation structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two well-structured sentences with status enumeration. The first sentence front-loads purpose, the second provides actionable status-specific guidance. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description mentions 'details... status and DNS setup instructions' but is vague about the full response structure. It adequately covers the main use but could be more specific about what fields are returned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add new meaning to parameters beyond what the schema already provides (UUID, use list-domains/projects to find IDs). No additional semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get details of a custom domain including its current status and DNS setup instructions,' specifying the verb, resource, and scope. It differentiates from siblings like 'add-domain', 'remove-domain', and 'list-domains' by focusing on a single domain's details and statuses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (to get domain details) and provides parameter sourcing hints ('Use list-domains to find the ID'), but does not explicitly contrast with alternatives like 'list-domains' or mention when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-envA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive hints. The description adds transparency about values being masked for security and the fact that auto-injected variables may not appear but are available at runtime, which is useful behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (two sentences), front-loaded with the primary action, and every sentence adds value. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter read tool, the description adequately covers purpose, security, and a caveat. However, it does not describe the output format (e.g., list of key-value pairs), which would be helpful given the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with the single parameter 'project_id' described as 'Project ID (UUID)'. The description adds no new parameter details beyond mentioning 'for a project', so it meets the baseline expectation without improvement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists environment variables for a project, using a specific verb ('List') and indicating the resource (project). It distinguishes itself from siblings like 'set-env' (write) and other unrelated tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for reading environment variables but does not explicitly state when to use it versus alternatives like 'set-env' or 'get-logs'. The caveat about auto-injected variables provides context but no exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list-domainsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, non-destructive, idempotent behavior. The description adds that the tool shows domain name, status, and ID, and implies a list output. This is adequate but does not disclose potential pagination or limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and no extraneous words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one parameter and strong annotations, the description covers the main purpose and return fields. It could mention the output format (e.g., array of objects) but is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage for the single parameter project_id, with a clear description. The description does not add further parameter-level detail, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List all custom domains for a sota.io project' with a specific verb and resource, and mentions the information shown (domain name, status, ID). It distinguishes from sibling tool get-domain by directing users there for details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides guidance by stating 'Use get-domain with a domain ID to see DNS instructions and full details,' which tells the agent when to use an alternative. It does not explicitly list when not to use this tool, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create-projectA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite minimal annotations (all false), the description provides rich behavioral details: auto-provisioning of database, connection pooling, backups, immutable slug, framework support, custom domain setup, and DNS instructions. No contradiction with annotations; the description fully compensates for the lack of annotation hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average but every sentence adds value (provisions, constraints, post-creation actions). It is front-loaded with the main purpose. Could be slightly tightened, but the verbosity is justified by the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers creation behavior, automatic provisioning, constraints, and post-creation steps (custom domains via API). However, it lacks explicit details about the tool's return value or response structure, which is notable since no output schema exists. This gap slightly reduces completeness for an agent that needs to chain actions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers 'name' with a description of slug generation. The description adds meaningful constraints: slug is lowercase, hyphens, max 63 chars, and immutable after creation. This extra detail helps the agent generate appropriate values beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Create a new project on sota.io.' with a specific verb and resource. It goes into extensive detail about what is automatically provisioned, which distinguishes it from sibling tools like deploy or list-projects that serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for creating projects but does not explicitly state when to use it versus alternatives. There is no mention of prerequisites, exclusions, or when not to use it. Compared to siblings like add-domain, the usage context is clear only by name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rollbackA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (destructiveHint=true, idempotentHint=true), the description reveals that the rollback swaps the container image without rebuilding, uses a blue-green strategy for zero downtime, and does not affect the database. These details help the agent understand operational impact and recovery guarantees.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences: purpose, mechanism, and usage guidance. Every sentence adds distinct value without redundancy. It is front-loaded and highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple input schema (one well-documented parameter), no output schema, and the tool's complexity, the description covers purpose, behavioral traits, usage conditions, and a critical caveat (no database rollback). No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage and only one parameter (project_id), the schema already explains the parameter well. The description adds no further semantics beyond what is in the schema, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a clear verb+resource pair, 'Rollback a project to its previous deployment.' This distinctively separates it from sibling tools like 'deploy' (which creates a new deployment) and 'delete-project' (which removes the project entirely).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the use case: 'Use this when a deployment introduces bugs or breaks the app.' It also clarifies a key limitation: 'The database is NOT rolled back,' helping agents avoid misuse. While it does not name alternative tools, the purpose and context make the decision clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deployA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate destructiveHint=true, and the description goes far beyond that by explaining the exact behavior: creates a tar.gz archive, uploads (max 50 MB), auto-detects framework, builds Docker image, uses blue-green deployment, keeps old container until health checks pass, and excludes files matching .gitignore/.sotaignore. It also details the auto-injected database environment variables. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections and bullet points for framework detection. It is front-loaded with the purpose and then dives into details. However, it is somewhat verbose, especially the section on database environment variables which could be shortened. Still, every sentence earns its place by providing necessary operational context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, and the description does not mention what the response or return value of the deploy call is. It also does not explicitly state error conditions, failure modes, or what happens if the upload fails. Given the complexity of the tool (deployment with blue-green, health checks, etc.), the description should include what the agent can expect back (e.g., a deployment ID, status, or confirmation). The absence of this information makes it slightly incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters described. The description adds context about the directory needing to contain app source code and mentions that project_id can be found via list-projects, but this mostly repeats schema descriptions. It does not add new semantic constraints beyond what is in the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Deploy an application to sota.io' and explains the entire process: creating an archive, uploading, auto-detecting framework, and building a Docker image. This is a specific verb-resource pair that distinguishes it from all sibling tools (none of which deploy).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides extensive guidance on when to use this tool, including framework detection, the requirement to listen on the PORT environment variable, and the blue-green deployment strategy. It also explicitly mentions using 'get-logs' to monitor build progress, steering the agent to an alternative tool for monitoring. The description covers prerequisites, supported frameworks, and custom Dockerfile usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete-projectA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already signal destructiveHint=true (alerting to damage) and idempotentHint=true (unusual for deletion but noted). The description adds crucial behavioral context beyond annotations: it explicitly lists what gets destroyed ('all deployments, the managed PostgreSQL database, environment variables, and webhooks'), states the action is 'PERMANENT and irreversible', and mentions that 'The project slug will become available again after deletion.' This goes well beyond what annotations alone convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the action and permanence, and lists the destroyed resources concisely. Every sentence serves a purpose: first sentence states the action and its finality, second sentence details what is removed and the side effect. Zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's high complexity (destructive, irreversible, many side effects), the description is complete: it specifies what is deleted, that it's permanent, and that the slug becomes available again. There is no output schema, and the description does not need to explain return values for a deletion tool—the important information is the outcome, which is clearly stated.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a single parameter (project_id) already documented as 'Project ID (UUID) to delete. Use list-projects to find the ID.' The description does not add new parameter semantics, but since coverage is high and the parameter is simple, the baseline is 3. The description earns a 4 because it provides important usage context (where to get the ID and what happens upon deletion) that complements the schema without being redundant.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Delete a project and all its deployments from sota.io' with a specific verb ('Delete'), resource ('project and all its deployments'), and platform ('sota.io'). It distinguishes itself from siblings like list-projects, create-project, and set-env by describing the permanent and irreversible nature of the action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use it (to delete a project) and highlights its permanence and irreversibility, guiding the agent to be cautious. However, it does not explicitly state when not to use it or provide alternatives among sibling tools (e.g., consider archiving or using rollback instead). The strong emphasis on permanence implies rare use, but no exclusions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description's burden is lower. The description adds value by listing possible statuses (pending, building, built, deploying, running, failed, stopped) and mentioning the URL format ({slug}.sota.io), which helps the agent understand the lifecycle. It does not discuss rate limits or potential staleness of status, but these are acceptable omissions for a simple read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (3 sentences) and front-loaded: the first sentence states the purpose, the second lists statuses, and the third gives usage guidance. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, simple schema, clear annotations, and no output schema), the description is complete. It explains what the tool returns (URL, framework, history), defines statuses, and clarifies the expected use case. The agent has everything needed to invoke and interpret the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with the schema already describing the parameter 'project_id' as a UUID and referencing 'list-projects' to find it. The description adds no further parameter semantics beyond what the schema provides. Baseline 3 is correct since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'current deployment status for a project'. It distinguishes itself from siblings like 'deploy' (which triggers deployment) and 'get-logs' (which retrieves logs) by specifying the exact output: live URL, detected framework, and deployment history. The description also lists deployment statuses, adding precision.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Use this to verify a deployment succeeded after calling deploy.' This directly helps the agent choose between this and sibling tools like 'get-logs' (for troubleshooting) or 'deploy' (to initiate). No alternative exclusions are needed since the use case is singular.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remove-domainA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide destructiveHint=true and idempotentHint=true. The description adds significant detail: it removes SSL certificate, is immediate and irreversible, and clarifies that DNS records are not automatically removed. This enriches the behavioral profile beyond what annotations alone convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the primary action. Every sentence adds value: purpose, scope of removal, and important caveats. No fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple removal tool with no output schema, the description covers the action, what is destroyed, what is not destroyed, and the irreversible nature. It also mentions a post-action step. No missing information given the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and both parameters already include usage guidance (e.g., 'Use list-domains to find the ID'). The description does not add any additional parameter semantics, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Remove a custom domain from a sota.io project' – a clear verb+resource. It explicitly states what is removed (domain mapping + SSL certificate) and distinguishes from siblings like add-domain by noting what it does NOT handle (DNS records). This provides full purpose clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description is clear about the action's irreversibility and the need to manually clean up DNS records. It implicitly tells the user when to use this tool (when you want to unmap a domain) and what to do after. However, it does not explicitly compare to alternatives like add-domain or list-domains, which would be a minor improvement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add-domainA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral context beyond the annotations: it explains the 5-domain limit, automatic HTTPS via Let's Encrypt, the specific DNS record types and IP/CNAME values, and the domain status lifecycle. Annotations (idempotentHint=true, destructiveHint=false) are consistent, and the description enriches the agent's understanding of side effects, return values, and provisioning behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient: a single paragraph of four sentences, each adding unique value. It front-loads the purpose, then concisely covers constraints, DNS instructions, and status lifecycle. No redundancy or filler; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema, straightforward mutation), the description covers all critical aspects: what it does, constraints (5 domains, automatic HTTPS), return value (DNS instructions), and status progression. The agent can reliably decide to invoke and understand the outcome. The only minor gap is error handling, but the idempotentHint annotation covers safety.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description adds significant meaning beyond the parameter descriptions. It explains how the domain parameter determines whether an A record (for apex) or CNAME record (for subdomain) is required, provides the exact IP address and CNAME format, and clarifies the return value contains DNS setup instructions. This goes well beyond the schema's example values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Add a custom domain to a sota.io project' with a specific verb and resource. It distinguishes from sibling tools like list-domains, get-domain, and remove-domain by clearly indicating this is the creation operation, and provides additional context about constraints and return value.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives, nor does it provide exclusions or prerequisites. The context 'Each project supports up to 5 custom domains' implies a limitation but does not directly guide the agent on when to choose add-domain over other domain-related tools. Usage is implied by the name and description, but not explicitly guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set-envA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare idempotentHint=true and destructiveHint=true, meaning the tool is safe to retry but changes take effect. The description adds critical context: variables are encrypted at rest (AES-256-GCM), they are injected at runtime, the auto-managed variable list, and the non-redeploy behavior. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is substantial but front-loaded with the core action ('Set an environment variable for a project.') and then covers encryption, auto-managed vars, redeploy requirement, and Next.js specifics. Every sentence provides necessary information, though it could be slightly tighter by merging the auto-managed variable list into a single sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (3 required params, mutation tool with deployment lifecycle dependency, encryption info, and sibling environment tools), the description covers all relevant aspects: what it does, what not to do, side effects, and special framework considerations. No output schema exists, but the description's behavioral coverage compensates adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and already provides descriptions for all three parameters (key, value, project_id). The description adds value beyond the schema by clarifying which keys are auto-managed (DATABASE_URL, PORT) and why, effectively enriching the semantics of the key parameter without needing to repeat schema text.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb-resource pairing: "Set an environment variable for a project." It also distinguishes itself from sibling tools like get-env and deploy by explaining the relationship between setting env vars and redeployment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use this tool, listing auto-managed variables (DATABASE_URL, PORT) that should NOT be set manually. It also explains that env var changes do not auto-redeploy and instructs the agent to call deploy or redeploy API to apply changes, plus special guidance for Next.js NEXT_PUBLIC_* variables.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-logsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the safety profile is clear. The description adds significant behavioral context: default behavior (latest deployment if no deployment_id), content of logs (framework detection, dependency installation, build steps, etc.), and common failure scenarios (missing dependencies, build errors, port issues). This goes well beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (about 5 sentences) and well-structured: it starts with the core purpose, then gives usage context, lists log contents, and ends with troubleshooting tips. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema, no nested objects), the description is complete. It covers the purpose, default behavior, log contents, common issues, and parameter hints. No gaps are apparent for an agent to correctly select and invoke this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (both parameters have descriptions). The description adds semantic meaning beyond the schema: it explains the default behavior of deployment_id (optional, defaults to latest) and provides a helpful hint to use get-status to see recent deployment IDs. This enriches the parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb and resource: 'Get build and runtime logs for a deployment.' It also specifies the default behavior (returns logs for latest deployment if no deployment_id provided) and distinguishes itself from siblings by positioning it as a tool to use after deploy to monitor progress and diagnose failures.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Use this after calling deploy to monitor build progress and diagnose failures.' It also provides troubleshooting context (common issues) and references another tool ('Use get-status to see recent deployment IDs') in the parameter description, offering clear guidance on alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.