Server Quality Checklist

Tool Scores

• control_plugC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool turns the plug on/off but doesn't describe what happens on failure (offline device, invalid ID/nickname), whether the operation is synchronous or returns confirmation, or any side effects. For a state-changing tool with zero annotation coverage, this is a notable gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single efficient sentence that packs the action, target resource, and the two parameters into minimal words. Zero waste, appropriately front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations, no output schema, and a mutation action (on/off), the tool should disclose more about behavior like error handling and confirmation semantics. Sibling tools include control_switch and control_purifier, which raises ambiguity about which tool controls which device type. The description is technically complete for a simple two-param binary switch but lacks operational detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% — both deviceId and state are already described in the schema with clear descriptions and an enum for state. The description notes that deviceId accepts either a MAC or nickname, which does add minor value beyond the schema. This meets the baseline-3 threshold given full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Turn a Wyze plug on or off.' The verb 'turn on/off' plus the resource 'Wyze plug' is specific. It doesn't explicitly distinguish from siblings like control_switch or control_purifier, but the resource (plug) provides differentiation, earning a 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes that either device ID or nickname can be used to identify the plug, which is mildly helpful for usage. However, it provides no guidance on when to use this tool vs alternatives like control_switch or control_thermostat, nor does it mention prerequisites like the plug needing to be online or paired.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• control_switchB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description implies a state mutation (on/off) but doesn't mention confirmation of the action, error behavior when the device is offline/unreachable, latency expectations, or whether the tool returns a status result. For a control tool with zero annotation coverage, more behavioral detail is warranted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One concise sentence carries the core purpose, and a second sentence covers identification. No wasted words. Could be slightly extended to include behavioral notes given the lack of annotations, but as written it is efficiently structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a device-control tool with no annotations and no output schema, the description is minimal. It doesn't explain what happens after the command (return value, confirmation), how to discover device IDs (though list_devices sibling exists), or handle failures. Among the sibling tools, this is a straightforward toggle, but the description still leaves the agent guessing about result semantics and failure modes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters (deviceId and state) are fully documented in the schema. The description adds minor value by clarifying the dual identification method (device ID or nickname) for deviceId, but doesn't add semantic depth beyond the schema's enum and descriptions. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb+resource construct: 'Turn a Wyze wall switch on or off.' It identifies the resource (wall switch) and the action (on/off toggle). However, it doesn't explicitly distinguish itself from the sibling control_plug or control_purifier beyond the word 'switch,' which creates minor ambiguity about which device type it controls.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying it controls a wall switch and that device ID or nickname can identify the switch. However, it provides no guidance on when to choose this tool over control_plug or other siblings, nor any exclusions or prerequisites (e.g., device must be online).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_devicesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The 'refresh' parameter implies a network call to the Wyze API, but the description doesn't disclose auth requirements, rate limits, potential latency of the refresh operation, or what 'current status' means (cached vs live). For a tool that hits an external API, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences that are concise and front-loaded with the primary action. The description is efficiently sized with zero filler. It loses a point only because mentioning the optional type filter in the description is somewhat redundant with the schema's enum documentation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a listing tool with no output schema and no annotations, the description is reasonably adequate but could do more. It doesn't describe the return format/structure of device data, doesn't mention pagination or volume concerns when listing 'all' devices, and doesn't explain what the refresh parameter actually changes behaviorally. Given it's a simple listing tool, this is minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters (type and refresh) well documented in the schema. The description adds the 'filter by device type' context that maps to the type parameter, but doesn't add meaning beyond what the schema provides. Baseline 3 is appropriate given full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (List), the resource (all discovered Wyze devices), and what's returned (current status), with optional filtering by device type. It distinguishes the listing scope from the sibling control/status tools since none of the siblings list all devices. A 5 isn't earned because it doesn't explicitly differentiate from get_device_status, which is a related status-focused tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is the entry-point tool for discovering devices, which differentiates it from the control_* and get_device_status siblings. However, it doesn't explicitly say WHEN to use this vs get_device_status (which also reports device status) or when-not to use it. The filter option hints at selective use but no explicit guidance on choosing alternatives is present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• control_purifierB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. 'Control' implies mutation/destructive state changes, and the description is honest that it sets state. However, it doesn't disclose potential side effects, whether partial updates are allowed (e.g., setting only fan mode), or any connectivity/error behavior. It adequately signals it's a write operation but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that efficiently convey purpose and key usage detail (device identification methods). No wasted words. Slightly more explicit behavior detail could be added, but as written it's tight and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description could be more complete. It explains the core action and device identification but doesn't note whether state and fanMode can be combined, whether omitting fanMode preserves current fan setting, or what the return/response looks like. Adequate but leaves behavioral gaps for a 3-parameter control tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter having its own description in the schema. The description adds marginal value by noting devices can be identified by deviceId, MAC, or nickname, which clarifies the deviceId parameter's dual nature beyond the schema's 'MAC or nickname' text. However, this largely repeats schema content.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource ('Control a Wyze air purifier') and identifies the actions (set power state and/or fan mode), distinguishing it from sibling tools like control_plug and control_switch which target different device types. However, it doesn't explicitly name sibling alternatives for differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating what can be controlled (power state and fan mode) and how to identify the device (device ID or nickname). It does not provide explicit when-to-use vs alternatives or exclusions, though the device-type-specific language in sibling names provides context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• control_thermostatA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses the dual-behavior nature of combined devices (plug vs thermostat), which is the key subtlety. However, it doesn't disclose the effect of turn_on/turn_off on non-combined thermostats, error behavior, or whether set_mode with 'off' disables the thermostat. Decent but not rich behavioral transparency for a mutating tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero waste. Every clause earns its place: the first states the tool's scope, the second clarifies the critical nuance about combined devices. Front-loaded with the primary purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  A mutating tool with no annotations and no output schema has gaps. The 100% schema coverage and enum-rich parameters help, and the combined-device clarification is valuable. However, missing guidance on what happens for turn_on/turn_off on non-combined devices, and no behavioral contract about setpoints (ranges, units like Celsius/Fahrenheit) leaves room for the agent to make wrong calls. Adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters thoroughly, including enums and per-parameter purpose. The description adds the plug-vs-thermostat semantic distinction for turn_on/turn_off, which is genuinely additive. Baseline 3 is appropriate since the schema does the heavy lifting and the description adds one useful clarification.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool controls a Wyze thermostat with a specific verb (control) and resource (thermostat). It usefully differentiates the plug vs thermostat behavior for combined devices, which distinguishes this from control_plug/control_switch siblings. However, the purpose is somewhat generic ('Control a Wyze thermostat') without enumerating the specific actions supported, though those are in the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly clarifies when turn_on/turn_off applies (only combined devices, controlling the plug/heater power) versus temperature actions controlling the thermostat. This provides clear context on how to select actions. It doesn't explicitly name sibling tools for exclusion, but the wiring of action-to-device-type behavior is helpful guidance that reduces misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_device_statusA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral burden. It states the tool reads status (a read operation) but does not explicitly disclose non-mutating behavior, permission/auth requirements, or what happens when the deviceId is invalid/unreachable. The device-type-specific output enumeration adds useful context, but for a tool without annotations there is room to more explicitly signal that this is a safe read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, dense sentence that efficiently packs the tool's purpose and device-type-specific outputs. Every clause adds value—the device-type enumeration distinguishes result shapes without waste. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one parameter, 100% schema coverage, and a good output enumeration, the description is mostly adequate. However, there is no output schema, and for a read tool the description could have disclosed the response format or behavior on invalid device IDs. Given the device-type diversity (thermostats, plugs, purifiers), a note on which device types are unsupported or the error behavior for unknown IDs would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%—the single parameter (deviceId) is well documented in the schema as 'Device ID (MAC) or nickname of the device.' The description adds context about what the tool returns but doesn't add parameter-level meaning beyond the schema. With full schema coverage, baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource ('Get detailed status') and clearly identifies the resource (Wyze device). It distinguishes itself from siblings by enumerating device-type-specific return data: temperature/humidity/setpoints for thermostats, on/off for plugs/switches, AQI and fan mode for air purifiers. This clearly separates it from the control_* sibling tools in the list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies it should be used to inspect device status, and the device-type-specific output lists hint at when each result would be relevant. However, it lacks explicit when-to-use guidance relative to alternatives—such as clarifying that this is the read-only counterpart to the control_* tools, or when get_api_status (a sibling API-level tool) would be more appropriate versus this device-level tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_api_statusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided (no readOnlyHint, no destructiveHint), so the description carries the burden of behavioral disclosure. The tool is clearly a read-only status query by nature, but the description doesn't explicitly state this or note whether it consumes API rate-limit quota itself, which would be useful context for a tool that reports rate limits. It does enumerate the returned fields, which is helpful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, dense, well-structured sentence that front-loads the action and then lists the specific returned information. Zero wasted words; every element contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only status tool, this is complete. It enumerates all returned categories (rate limit calls, reset time, cache info, key expiration). No output schema exists, so describing return values in prose is appropriate and adequate. A minor gap: it doesn't clarify whether calling this tool consumes rate-limit quota or how to interpret values, but for a status dashboard this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are 0 parameters, and schema coverage is 100%, so the baseline of 4 applies for a zero-parameter tool. The description correctly explains what the tool reports (rate limits and key expiration), giving context for what this no-input tool does, which is all that's needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and specific resources (Wyze API rate limit status and key expiration). It also enumerates the returned values (remaining calls, reset time, cache info, API key expiration), which clearly distinguishes it from siblings like get_device_status and the control_* tools that deal with device operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys what the tool returns but does not give explicit guidance on when to use it versus alternatives. However, for a diagnostic/status tool, usage context is reasonably implied — checking rate limits before making API calls. There's no stated exclusion or alternative mention.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.