Server Quality Checklist

Tool Scores

• get_vm_infoB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool gets 'detailed information' but doesn't specify what that includes (e.g., configuration, status, metadata), whether it's read-only, or any prerequisites like authentication. This leaves significant gaps in understanding the tool's behavior beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any fluff. It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter) and the presence of an output schema (which handles return values), the description is somewhat complete. However, with no annotations and minimal parameter guidance, it lacks details on behavioral traits and usage context, making it adequate but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, so the schema provides no semantic information. The description adds minimal context by implying 'vm_id' identifies a specific VM, but doesn't explain what format the ID should be (e.g., UUID, name) or where to find it. This partially compensates for the schema gap but remains vague.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('detailed information about a specific VM'), making the purpose unambiguous. It distinguishes from siblings like 'list_vms' (which lists multiple VMs) and 'power_vm' (which controls power state), though it doesn't explicitly mention 'get_vm_power_state' which might overlap in scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention that 'list_vms' might be better for discovering VMs, or that 'get_vm_power_state' might be sufficient if only power state is needed. The description implies usage for a specific VM but offers no exclusions or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_vm_power_stateB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool gets power state, implying a read-only operation, but doesn't specify if it requires authentication, has rate limits, returns specific error conditions, or details the output format. This leaves significant gaps in understanding the tool's behavior beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It's front-loaded with the core purpose and efficiently conveys the essential information without unnecessary elaboration, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and the presence of an output schema (which should document return values), the description is minimally adequate. However, it lacks context on usage guidelines and behavioral details, which are important for a tool that interacts with system resources like VMs, leaving room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. It mentions 'a specific VM', which implies the 'vm_id' parameter identifies the VM, but doesn't clarify the ID format, source, or constraints. This adds minimal semantic value beyond what the schema's title ('Vm Id') suggests, meeting the baseline for low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('power state of a specific VM'), making the purpose unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'get_vm_info' (which might include power state among other details) or 'list_vms' (which lists VMs rather than querying a specific one's state), so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get_vm_info' (which might retrieve power state along with other info) or 'power_vm' (which changes power state), leaving the agent to infer usage context without explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_vmsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action but doesn't describe what 'list' entails—e.g., whether it returns names, IDs, statuses, or pagination details. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior and output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It is front-loaded and appropriately sized for a simple listing operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, output schema exists), the description is minimally adequate. However, with no annotations and an output schema present, it doesn't explain what the listing includes or how to interpret results, leaving some context gaps despite the structured output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, meaning no parameters are documented in the schema. The description doesn't add parameter details, which is appropriate since there are none. A baseline of 4 is applied for zero parameters, as no compensation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all VMs in VMware Fusion'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like get_vm_info (which retrieves detailed info for a specific VM) or power_vm (which controls power state), but the basic action is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention that list_vms returns a broad overview while get_vm_info provides detailed data for a specific VM, or that it might be a prerequisite for operations like power_vm. Usage context is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• power_vmA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states what actions are valid. It doesn't disclose behavioral traits like whether actions are immediate or queued, permission requirements, whether 'shutdown' is graceful or forced, side effects on other VMs, or error handling. The description is minimal beyond stating valid actions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and follows with essential action details, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (power operations on VMs) and no annotations, the description is incomplete. It covers valid actions but lacks crucial context like permissions, side effects, or differences between actions. The presence of an output schema helps, but the description doesn't reference it or explain return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant value beyond the 0% schema coverage by explicitly listing all valid action values ('on', 'off', 'shutdown', 'suspend', 'pause', 'unpause'), which the schema doesn't document. However, it doesn't explain the 'vm_id' parameter or provide context about action differences.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('perform a power action') and resource ('on a VM'), specifying the exact scope of operations. It distinguishes from sibling tools like 'get_vm_info' or 'list_vms' by focusing on power state changes rather than information retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for power state changes but doesn't explicitly state when to use this tool versus alternatives like 'get_vm_power_state' for checking status. It provides valid action values but no guidance on prerequisites, error conditions, or when certain actions are appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.