Server Quality Checklist

Tool Scores

• tuya_set_colorB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral disclosure burden, but it only states the action and value ranges. It does not mention side effects (e.g., whether the light is turned on), required permissions, error responses, or any other operational consequences, leaving the agent without important safety context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no filler. It front-loads the main action and then succinctly provides parameter ranges, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a mutating tool with no annotations, and the description lacks critical operational details such as expected output, error cases, or whether the action is idempotent. Even though an output schema exists, the description does not explain what the agent should expect in return, leaving the context incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining parameters. It provides useful ranges for hue, saturation, and value, but does not clarify device_id beyond an ambiguous reference to 'name', and it fails to specify that device_id is a unique identifier string. The 'name' mention is misleading given the schema only accepts device_id.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Set the color') and target resource ('Tuya RGB light'), distinguishing it from sibling tools like set_brightness and set_color_temperature. However, the mention of 'by device_id or name' is ambiguous because the schema only includes device_id, creating slight confusion about the intended input.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for colored lights using HSV values, which differentiates it from brightness or color temperature tools. However, it does not explicitly state when to use this tool versus alternatives, nor does it provide exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tuya_toggle_deviceA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It explains the toggle behavior, but it also claims the tool works 'by name' ('Living Room Light') while the input schema only has a device_id parameter, which is misleading. It also does not disclose how the toggle is performed (e.g., state read or raw command), potential failure modes, or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the primary action. The two-sentence structure is efficient, but the unsupported 'or name' claim adds a bit of unnecessary and potentially confusing text, preventing a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and an output schema exists, so return values do not need explanation. However, the description toggles the device but does not explain what switch_code does, how name-based lookup works, or how an unknown device state is handled. These gaps make the description only minimally adequate for reliable invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds some context for device_id by implying it may be a name, but this conflicts with the schema's field name. The switch_code parameter is not mentioned at all, leaving its purpose and usage undefined.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (toggle) and the resource (Tuya device), and specifies the behavior: it turns the device on if off and off if on. This distinct behavior separates it from the sibling tools tuya_turn_on_device and tuya_turn_off_device.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when you want the device state to flip. It does not explicitly mention alternatives or 'when-not to use', but the toggle semantics are self-evident, and no ambiguous conditions are raised.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tuya_turn_off_deviceA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden. It discloses the action and multi-switch behavior but does not mention side effects, errors, reversibility, prerequisites, or behavior when the device is already off. For a mutating tool, this is a significant transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action and immediately useful details. Every word earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a straightforward device-control tool, the description covers the essential selection/invocation context: identifying the device, multi-switch handling, and the default switch_code. An output schema exists to document return values, so the lack of return-value detail is acceptable. It only misses broader operational context like error or safety behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions cover 0% of parameters, so the description must add meaning. It explains device_id can be a name (with example) and specifies switch_code values ('switch_1, switch_2, etc.'), going beyond the raw schema. This compensates well for the two-parameter schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb ('Turn off') and resource ('a Tuya device'), specifies how to identify the device (device_id or name) with an example, and indicates multi-switch handling. This clearly distinguishes it from siblings like tuya_turn_on_device and tuya_toggle_device.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as tuya_toggle_device or tuya_turn_on_device. It only gives context about multi-switch devices, but no explicit when/when-not or alternative recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tuya_send_commandA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior but provides scant detail. It mentions the commands format and that device_id or name can be used, but does not disclose side effects (e.g., state changes), error handling, or any prerequisites like permissions. This is a significant gap for a tool that can send arbitrary commands, potentially altering device behavior in unexpected ways.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core purpose, followed by the essential parameter detail and example. No wasted words; every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple (two parameters) and has an output schema, so return values need not be described. However, the description lacks any mention of when to use this tool over the many specialized siblings, and it does not address potential failure modes or the scope of 'custom commands'. These gaps make it only minimally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by explaining the commands parameter with a concrete example and clarifying that device_id can be a friendly name. This adds meaning beyond the bare 'string' type in the schema, though it could go further by describing command syntax details or available codes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Send custom commands to a Tuya device'. It specifies a specific verb ('Send'), a resource ('Tuya device'), and a scope ('custom commands'). This distinguishes it from the more specific sibling tools (e.g., turn_on, set_brightness) which handle predefined actions, making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through an example ('commands must be a JSON string like...') but does not explicitly state when to use this tool versus the many sibling tools. It lacks guidance on when custom commands are appropriate or when alternative specific tools should be preferred, leaving the agent to infer from context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tuya_get_device_infoA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It indicates a read operation ('Get detailed information') with no side effects, but it does not explicitly state read-only status or error behavior. More importantly, the description says lookup can be by 'device_id or name', yet the schema only accepts a device_id parameter, creating a potential mismatch in expected input; this ambiguity is not fully disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence of about 26 words. It leads with the action and object, then specifies the lookup method and example, then lists output attributes. No unnecessary words or filler; every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter read tool with an output schema provided, the description is largely complete. It covers the core purpose, parameter flexibility, and a representative example. It lacks explicit sibling differentiation and error cases, but these are non-critical given the tool's simplicity and the presence of an output schema for return value details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, device_id, has no schema description (0% coverage). The description compensates by explaining that the parameter accepts either a device ID or a friendly name, providing a concrete example ('Living Room Light'). This adds critical meaning beyond the bare schema, clarifying that the string parameter is flexible despite its name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to get detailed information about a specific Tuya device. It lists concrete data fields (capabilities, category, model, firmware version), making it distinct from sibling tools like get_device_status, which focuses on real-time state. The inclusion of an example name ('Living Room Light') further clarifies the resource type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving device metadata, but it does not explicitly differentiate from alternatives such as get_device_status or tuya_list_devices. There is no when-to-use guidance or exclusion note. The mention of specific fields (model, firmware) hints at a configuration/info use case, but this is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tuya_list_devicesA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden. It states the scope ('all available') and return fields, but does not mention pagination, error behavior, or authentication needs. The read-only nature is implied but not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the verb and resource. Every word adds value, with no unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list tool that already has an output schema, the description sufficiently captures the tool's purpose and result. It is complete for the simplicity of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the input schema is an empty object with 100% coverage. The description does not need to add parameter details, so the baseline of 4 for 0-parameter tools applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get a list') and the resource ('all available Tuya devices'), specifying the included fields (IDs, names, categories, online status). This distinguishes it from sibling tools that target individual device actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing devices but lacks explicit guidance on when to prefer this over alternatives or any exclusions. No alternatives are mentioned, though sibling names make the distinction obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tuya_get_device_statusA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It conveys that the tool returns live status data including power, brightness, color, temperature, and other properties, which is useful. However, it does not disclose error behavior (e.g., device offline, invalid name) or any consequences, though the read-only nature is self-evident.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One concise sentence with the verb and resource front-loaded, followed by a parenthetical example. No redundant words, every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a simple single-parameter getter, and an output schema exists, so the description need not enumerate return fields. It covers purpose and parameter semantics adequately. The only gap is the lack of explicit usage exclusions, but overall the tool is well-defined.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only lists device_id as a string with zero description coverage. The description adds crucial semantics by stating the parameter accepts either a device_id or a device name (e.g., 'Living Room Light'), which is essential for correct usage and goes far beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Get the current status') and the target ('a specific Tuya device') with an example name. This distinguishes it from sibling control tools like tuya_turn_on_device or tuya_set_brightness, which perform mutations rather than reads.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It implies the tool is for reading the current state of a device (as opposed to the setter/toggle siblings) and clarifies the parameter can be a device_id or a device name. However, it lacks an explicit 'use this when you need to check state rather than change it' exclusion or mention of alternatives, so it falls short of full guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tuya_set_brightnessA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description provides one important behavioral constraint: brightness must be between 0 and 1000. However, it does not disclose side effects like whether the light will be turned on if off, or the reversibility of the action, leaving some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the purpose and includes essential constraints. No redundant information or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the key aspects for a simple tool: what it does, how to identify the device, and the valid brightness range. Given the presence of an output schema and the simplicity of the tool, it is mostly complete, though it lacks details on error handling or behavior when the device is unreachable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no parameter descriptions, but the description compensates fully by explaining that device_id can be a name (e.g., 'Living Room Light') and that brightness is an integer with a valid range. This adds significant meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Set the brightness of a Tuya light'. It identifies the resource (Tuya light) and the specific action (set brightness), distinguishing it from siblings like set_color_temperature and set_color.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context by specifying that the device can be identified by device_id or name, but it does not explicitly mention when to use this tool versus alternatives like set_color. This is adequate but not explicit about exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tuya_set_color_temperatureA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the valid temperature range and warm/cool mapping, which is useful. However, it does not mention any side effects, prerequisites (e.g., device must be online), or what happens on invalid input.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One succinct sentence front-loads the action, includes an example, and states the valid range. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter setter, the description covers the essential purpose, device identification, and value constraints. Output schema exists, so return values need no explanation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description adds key semantics: device_id can be a name (example), and temperature has a meaningful scale (0-1000 with warm/cool mapping). This goes beyond the bare field types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Set') and the specific resource ('color temperature of a Tuya light'), with an example device identifier. It distinguishes from sibling tools like set_brightness and set_color.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use the tool (when adjusting color temperature) and provides a concrete example. It does not explicitly name alternatives or exclusions, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tuya_turn_on_deviceA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It reveals that the device can be identified by ID or name, and that switch_code controls which switch is activated. It doesn't address edge cases like already-on behavior, but the core behavior is transparent enough for a simple turn-on action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and target, and efficiently packs the switch_code nuance. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists and there are only two parameters, the description covers purpose, parameter semantics, and multi-switch handling adequately. It doesn't mention error scenarios, but that is not critical for a straightforward command.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% with no property descriptions, so the description must compensate. It explains that device_id can be either an ID or a friendly name, and adds meaning to switch_code by explaining its purpose for multi-switch devices with examples (switch_1, switch_2). This goes well beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states "Turn on a Tuya device" with specific targeting by device_id or name, distinguishing it from siblings like tuya_turn_off_device or tuya_toggle_device. The switch_code detail further clarifies the tool's specific role for multi-switch devices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when to use the tool (to turn on a device) and offers explicit guidance for multi-switch devices by specifying switch_code. It doesn't explicitly exclude alternatives like toggle, but the action is singular and well-separated from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.