Server Quality Checklist

Tool Scores

• power_off_vmB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description should disclose more behavioral details. It states 'hard power off' implying forceful shutdown but does not mention risks (data loss), prerequisites, or behavior if VM is already off or not found.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no wasted words, but lacks structure such as separate sections for usage or parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with an output schema, the description is adequate for the core action, but fails to explain the optional parameter and lacks behavioral context, leaving gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters. It explains name_or_id (display name or moref ID) but completely ignores the 'target' parameter, leaving its purpose unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (hard power off) and the resource (VM), and specifies the identification methods (display name or moref ID), distinguishing it from siblings like power_on_vm or delete_vm.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (to hard power off a VM) but provides no explicit guidance on when not to use, such as preferring a soft shutdown or prerequisites like the VM being powered on.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_vmA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden but only mentions 'network boots first' as a behavioral trait. It does not disclose whether the operation is idempotent, requires authentication, or what happens on conflict.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise, using four short lines with no unnecessary words. Each sentence adds value and is front-loaded with the primary purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 parameters and no output schema details, the description is incomplete: it omits required fields, resource constraints, and 5 parameters entirely. An output schema exists but does not compensate for missing parameter explanations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description only explains three parameters (vm_type, disk_provisioning, network_profile) out of eight. Critical parameters like name, cpu, ram_mb, and disk_gb are left undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Create' and the resource 'VM', and adds specificity with 'network boots first'. Sibling tools are management operations, so there is no confusion with alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives examples for vm_type and disk_provisioning defaults, but does not provide explicit guidance on when to use this tool versus alternatives, nor does it mention prerequisites or constraints.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• power_on_vmA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description only states the action without disclosing side effects (e.g., idempotency), required permissions, or error conditions. With no annotations, the agent lacks critical behavioral context for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no wasted words. Every piece of information earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two parameters and an output schema, the description covers the primary parameter but misses behavioral details (e.g., what happens if VM is already on) and the purpose of the 'target' parameter. Adequate but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explains that name_or_id accepts a display name or moref ID, adding value beyond the schema structure. However, the optional 'target' parameter is not explained, and schema coverage is 0%, so the description only partially compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Power on a VM') and the resource identifiers ('by display name or moref ID'), which distinguishes it from siblings like power_off_vm.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies how to identify the VM (name or moref ID) with an example, but does not provide guidance on when to choose one identifier over the other or mention when not to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_vmA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description shoulders full transparency. It discloses the irreversible destructive action, the pre-step (power off), and the input format. Missing details like permission requirements or error behavior, but core behavioral traits are present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no extraneous information. Action and input format are front-loaded, making it efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the action and key parameter but lacks explanation for the optional 'target' parameter and does not mention output schema or return behavior, leaving gaps for a complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain all parameters. It clarifies that name_or_id accepts a display name or moref ID, but completely omits the 'target' parameter, leaving users without guidance on its purpose or usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Permanently delete a VM' and details the process ('power off if running, then destroy from disk'), clearly differentiating it from sibling tools like list_vms or power_on_vm.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains how to identify the VM (name or moref ID) but does not provide explicit guidance on when to use this tool versus alternatives, such as powering off or creating a snapshot.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_vmsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses significant behavioral differences between standalone ESXi and vCenter, including optional datacenter grouping. Without annotations, it carries the full burden and does so well. It implies read-only operation (list) but does not explicitly state lack of side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two bullet points, front-loading the main action. Every sentence provides value, no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's core behavior and parameter roles. With an output schema present (context signal), return values need not be described. It is fairly complete for a list tool, though explicit mention of read-only nature would strengthen it further.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate. It explains the 'datacenter' parameter defaults to the target's configured datacenter but lacks detail on the 'target' parameter (e.g., format, valid values). Some meaning added, but insufficient given the gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists VMs on a target, with specific behavior for standalone ESXi vs vCenter. It uniquely identifies the resource (VMs) and action (list), distinguishing it from sibling tools like create_vm, delete_vm, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (listing VMs on a target) and mentions key parameters (target, datacenter). However, it does not explicitly state when not to use or suggest alternative tools, though siblings are distinct actions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.