Server Quality Checklist

Tool Scores

• varity_dev_serverA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It mentions return of localhost URL but does not disclose behavioral traits like auto-selection of port, persistence to config file, or any side effects. Parameter description covers some details, but main description lacks behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose. No extraneous information. Efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Description mentions return of localhost URL, but lacks details for stop and status actions (e.g., what they return). No output schema exists, so description should cover return values for all actions. Missing error handling or state change information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and parameter descriptions (especially port) are detailed. The main description adds no additional parameter meaning beyond what the schema provides, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool manages a local development server with start, stop, and check actions, and specifies it returns a localhost URL. This distinguishes it from sibling tools like deploy or build.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (for local dev server management) but does not explicitly mention when not to use or alternatives. Implicit differentiation from siblings is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_install_depsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description indicates the tool installs dependencies, which is a read-modify operation. The annotations include destructiveHint: true, confirming modification of files. The description does not add further behavioral context (e.g., that it modifies package-lock.json or may overwrite existing node_modules), but given the annotation coverage, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two short sentences. The first sentence immediately states the purpose, and the second gives usage guidance. Every word is necessary, and no extraneous information is included.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 optional parameters, no output schema, destructive annotation), the description covers the essential usage context. It could be improved by mentioning potential side effects (e.g., file system modifications) or prerequisites (e.g., requires Node.js/npm), but these are implied by the tool's nature and annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters have descriptions in the schema (100% coverage). The description adds a small extra: for the 'packages' parameter, it notes 'If omitted, runs npm install for all dependencies,' which is implied by the schema. Overall, the description does not significantly add beyond the schema, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool installs npm dependencies in a Varity project. It uses a specific verb ('Install') and resource ('npm dependencies'), and the context 'Use after creating a project or when adding new packages' distinguishes it from sibling tools like varity_init (which creates the project) and varity_build (which builds the project).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'after creating a project or when adding new packages.' This provides clear context. However, it does not mention when not to use it or direct the agent to alternative tools for related tasks, such as initial project setup.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_open_browserA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description confirms the tool opens a URL in the default browser. Without annotations, it carries the full burden and could benefit from disclosing that it does not wait for the browser, potential for multiple tabs, or error behavior, but the basic behavior is stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two succinct sentences: the first states the primary action, the second gives usage context. No unnecessary words, perfectly front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema, no nested objects), the description adequately covers purpose and typical usage. Minor omission: no mention of error handling or security implications, but acceptable for a straightforward tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a single parameter 'url' that is well-described in the schema. The description does not add new semantic information beyond what the schema already provides, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Open a URL') and the resource ('in the user's default browser'). It unequivocally distinguishes this tool from siblings, as no other sibling tool opens a browser.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides specific scenarios for use: after deploying to show the live app or after starting the dev server. Although it does not explicitly state when not to use it or mention alternatives, the context is clear and directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_initA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool scaffolds a Next.js project and includes specific components. The force parameter warns about overwriting existing files. However, it does not mention potential side effects like network downloads, dependency installation, or execution time. The behavioral disclosure is adequate but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with front-loaded purpose: two sentences for what it does, then a list of features, then a usage guideline. The path parameter description is long but necessary. Overall, it is concise without wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description covers what the tool creates and includes usage considerations. It lacks detail on return values or success output, but the project contents are well described. With 4 parameters and no nested objects, the description provides sufficient context for an agent to understand the tool's scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining what the resulting project includes, and the path parameter includes a concrete example and a warning about default behavior. The force parameter reinforces the safety warning. These details go beyond the schema's property descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a new production-ready app with specific features like auth, database, and payments. It lists the resulting project contents (dashboard, authentication, settings, etc.). This distinguishes it from sibling tools like varity_build or varity_deploy, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use the tool: 'when a developer wants to start a new project, create an app, or scaffold something.' Additionally, the path parameter description provides critical usage guidance by instructing to always pass an absolute path and ask the user if unsure. However, it does not explicitly state when not to use it or list alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_search_docsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the agent knows it's safe. Description does not add further behavioral context beyond stating the content types searched. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with clear hierarchy: purpose, exclusion, inclusion. Efficient and front-loaded, though could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple search nature with read-only annotation, the description covers the main points. It doesn't detail result format or pagination, but this is acceptable for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and both parameters have clear descriptions. The tool description adds context but does not enhance parameter semantics beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool searches tutorials, troubleshooting guides, and deployment docs, and distinguishes from SDK API references. The verb 'search' is specific and the resource scope is well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly provides when to use (how-to guides, getting started, FAQ, troubleshooting) and when not to (SDK API references), offering an alternative (varity://sdk/* resources). This is excellent guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_migrateA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructiveHint=true. The description adds context by detailing the transformation steps (removing vercel.json, @vercel/* packages, etc.) and the output (deployment URL + report). No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the main action, and contains no redundant information. Every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the key aspects: what it does, which projects it works with (Next.js), and the return value (URL and report). No output schema exists, but the return information is provided. Could mention prerequisites (e.g., GitHub access) but not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description provides context for the parameters (e.g., 'The repository will be cloned to a temporary directory') but does not add significant meaning beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Migrate a Vercel project to Varity in one step'. It details the specific actions (clone, remove artifacts, deploy) and distinguishes from siblings like varity_deploy and varity_create_repo by focusing on the migration workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this when a developer wants to move their Vercel app to Varity' and notes it works with Next.js projects. While it doesn't list when not to use or alternative tools, the context is clear enough for an AI agent to select this over other sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_agent_infoA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. It enumerates exactly what information is returned (image, ports, resources, env vars, cost, access mode, operator notes). Implies read-only operation. Could explicitly state no side effects, but overall clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff. First sentence lists contents, second provides usage guidance. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simple tool with one param, no output schema, no annotations, description sufficiently covers what it returns. Could mention prerequisites like authentication, but likely implicit.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Single parameter 'name' has enum values fully described in schema (100% coverage). Description adds context that the slug should come from varity_list_agents, providing extra value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool shows full details for a single AI agent template, listing specific items. It differentiates from sibling tool varity_list_agents by specifying the input comes from that list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: when developer has picked an agent and wants specifics before deploying. Also tells to pass slug from varity_list_agents. Could add when not to use, but current guidance is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_buildA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate destructiveHint=true, and description adds that it auto-detects framework and verifies compilation. However, it lacks details on side effects like cleaning previous builds or dependency requirements. Description adds value but could be more thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with clear front-loading. First sentence states purpose, second adds usage context. Every sentence earns its place without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one optional parameter and no output schema, the description covers purpose, usage timing, and auto-detection. Could mention output artifacts or prerequisites, but sufficient for a straightforward build tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter 'path'. Description adds meaning by noting auto-detection from package.json, implying the path should contain package.json. This enhances understanding beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool builds a project for production, with specific verb 'Build' and resource 'project'. It distinguishes from siblings like varity_dev_server by specifying production context and pre-deployment usage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'Run before deploying or to verify the project compiles.' Provides clear context, though does not explicitly list alternatives or when-not scenarios. Still strong guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_doctorA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true. The description adds valuable context by enumerating what is verified (Node.js, npm, CLI, auth), which goes beyond the annotation. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with purpose, no wasted words. Perfectly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter check tool, the description covers purpose, usage timing, and what is checked. Does not detail output format, but for a simple diagnostic tool this is acceptable and not a critical gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, and schema coverage is 100% (empty). Baseline score of 4 is appropriate; description adds no param info but none needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks if the developer's environment is ready, specifying exact resources (Node.js, npm, varitykit CLI, authentication). It distinguishes from siblings by positioning as a prerequisite check before varity_init or varity_deploy.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to run before varity_init or varity_deploy, providing clear usage context. Does not explicitly state when not to use, but the positive guidance is strong and sufficient for a zero-parameter tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_deployA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true, so the description adds value by explaining the automatic framework detection, build process, and zero-config nature. However, it does not disclose potential side effects like overwriting previous deployments or billing start, but the mention of billing in the alt tool reference provides partial context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a focused paragraph with front-loaded main action. Each sentence serves a purpose, though it could be slightly trimmed without losing meaning. Overall efficient given the information conveyed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and good annotations, the description covers purpose, usage context, alternatives, and basic behavior. It mentions the URL pattern but does not explain error handling or status checks. Still, it is sufficiently complete for a deploy tool with well-documented parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds cross-reference value by mentioning that repo_url can be taken from varity_create_repo and app_name controls the deployment URL, which is additional context beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Deploy the current project to production on Varity' with specific verb and resource. It distinguishes from sibling tools varity_deploy_agent and varity_delete_deployment by naming them explicitly, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: 'Use this when a developer wants to deploy, publish, ship, or make their app live' and directs to alternatives for other use cases. It also mentions when to use varity_deploy_agent for curated AI agents and varity_delete_deployment to stop a deployment.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_deploy_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description adds behavioral details about the output (URL, status, framework, size, creation time). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. First sentence defines purpose and output, second sentence provides usage guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 3 optional parameters, readOnlyHint annotation, and no output schema, the description fully covers the tool's behavior and usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, baseline 3. Description adds context for each parameter: path 'only show deployments for this project', limit 'default 10, max 50', deployment_id 'optional, omit to list recent'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the purpose: 'List deployments or get status of a specific deployment' and lists the output fields. It distinguishes from sibling tools like varity_deploy, varity_delete_deployment, and varity_deploy_logs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance: 'Use this when a developer asks about their deployments, wants to check status, or needs to find a deployment URL.' No explicit when-not-to-use, but the context of sibling tools implies alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_cost_calculatorA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, so the tool is safe. The description adds that it computes a comparison and highlights the flat pricing model, which is transparent. However, it does not mention error handling or authorization requirements, but given the safety annotations, this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (4 sentences), front-loads the main purpose, and follows with usage context and parameter guidance. Every sentence adds necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description does not need to explain return values. It covers the main functionality, parameter modes, and comparison details. Missing are possible edge cases or error handling, but for a simple calculator tool it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining the difference between subdomain and app_profile, and by listing the enum presets with specific cost for static-site, which is not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool compares deployment costs between Varity and competitors, with a specific verb 'compare' and resource 'cost'. It distinguishes itself from sibling tools (all other tools are for deployment, building, etc.) as the only cost calculator.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use whenever a developer asks about pricing, cost, hosting bills, traffic costs, or platform comparison.' It also explains the two modes (subdomain for real cost, app_profile for estimate), providing clear context for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_delete_deploymentA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses behavioral traits: stops billing immediately, closes compute lease, removes deployment record, and notes static deployments also stop billing. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is informative but slightly verbose; however, it is front-loaded with key purpose and each sentence adds value. Could be trimmed slightly but still effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description covers all necessary context: behavior (deletion, billing stop), prerequisites (confirm name if unsure), and impact. Complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with good parameter description for 'name'. The tool description adds example names but does not significantly enhance understanding beyond schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action (delete deployment), the resource (Varity deployment), and key consequence (stop billing). It distinguishes from sibling tools by focusing on deletion and provides example user phrases. The verb 'delete' and noun 'deployment' are specific, and the text adds 'stop billing immediately' for clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly lists when to use: when developer says 'stop my <name>', 'shut down', etc. Also advises when not to use: suggests using varity_deploy_status or listing deployments to confirm name if unsure. Provides clear context for appropriate invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_deploy_agentA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that deployment makes the agent accessible at a URL and that env variables vary per agent. Since no annotations are present, it carries the full burden and adequately describes the mutation, though it lacks info on auth or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with core action, no wasted words. Every sentence adds meaningful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 params and no output schema, the description covers purpose, usage guidance, and parameter details. It references sibling tools and setup steps, though it could mention error scenarios or default behaviors.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline is 3. The description adds value by explaining the env parameter with example, name as URL slug, and agent as the list, going beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it deploys a curated AI agent from 5 templates and provides the URL. It distinguishes itself from siblings like varity_delete_deployment by specifying when to use each.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use this tool (e.g., 'deploy hermes', 'I want a Telegram bot') and advises calling varity_agent_info first to get required env vars. It also references the alternative varity_delete_deployment for stopping.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_list_agentsA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It details what the tool returns (name, cost, env vars, ports, resource footprint) but does not explicitly state it is read-only or has no side effects. However, it implies a safe list operation, so it is fairly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph that front-loads the main purpose, then provides agent examples, usage context, and return details. It is informative without being verbose, and each sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description covers all relevant information: agent names, costs, return fields, and usage scenario. It also links to a sibling tool for deployment, making it complete for the user's needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema coverage is 100%. The description does not need to add parameter info, and it correctly omits any. The baseline for 0 parameters is 4, but the description fully satisfies the need, earning a 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists curated AI agent templates, provides specific examples with details, and distinguishes from sibling tools like varity_agent_info and varity_deploy_agent by stating when to use it and what follow-up action to take.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this when a developer asks what AI agents can I deploy?' and directs to deploy with varity_deploy_agent after selection, providing clear context and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_loginA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully bears the transparency burden: it explains the two behavioral paths (with/without deploy_key), the browser interaction, and the outcome of enabling deployments. No hidden side effects are implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no redundancy, front-loaded with purpose. Every sentence adds distinct, valuable information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one optional param, no output schema). The description covers both usage scenarios completely, leaving no ambiguity about what the tool does.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the description adds significant behavioral context beyond the schema: it ties the parameter value to the immediate ability to deploy, and explains the fallback behavior when omitted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb-resource combination ('Log in to Varity') and explains its purpose: enable deployments. It distinguishes itself from sibling tools like varity_deploy by being the prerequisite login step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: use with deploy_key for immediate deployment readiness, or omit to get the key via browser. No alternative tools are mentioned, but the usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_create_repoA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the destructiveHint annotation, the description discloses that the tool creates an empty repo and pushes actual code, or creates from a SaaS template if no path is given. It also mentions the need for a GitHub token and the requirement to call before varity_deploy, providing critical behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a concise opening sentence, then a logical breakdown of two use cases, followed by a deployment ordering note, and finally a token requirement. Every sentence is informative and front-loaded, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers preconditions, use cases, and side effects well for a creation tool with no output schema. However, it does not explicitly state what the tool returns (e.g., the repository URL), which would be beneficial for the agent to know that it can capture the output for use in subsequent commands.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds significant meaning: it explains the dual behavior of the 'path' parameter (push code vs. template), clarifies that 'name' must be lowercase and hyphens allowed, and notes that 'github_token' can come from an environment variable. This enriches the schema's descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: creating a GitHub repository and pushing code. It distinguishes two primary use cases (custom apps with path vs. template without path), and differentiates from sibling tools by specifying its role in dynamic deployments before varity_deploy.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidelines: when to use path (custom apps) vs. omit path (template quick-start), and when to call before varity_deploy. It also states the prerequisite of a GitHub token with repo scope, covering when and how to use the tool effectively.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• varity_deploy_logsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses dual behavior: returns actual log lines if full logs exist, otherwise returns structured summary receipt. Adds context beyond readOnlyHint annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with verb and resource, no fluff. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Explains return types, references sibling for edge case, and covers all agent needs given no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3. Description adds context by linking parameters to behavior (limit used when logs exist) and implying deployment_id value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb and resource: 'Get build logs or deployment summary for a specific deployment.' It distinguishes from sibling varity_build by specifying what each tool retrieves.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly explains when to use: to get live URL, check status, confirm build metrics. Also provides a when-not: for detailed build errors, use varity_build.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.