Server Quality Checklist

Tool Scores

• set_temperature_offsetC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but lacks critical behavioral details. It mentions 'adjusts the measured temperature by a fixed offset' but doesn't disclose permissions required, whether the change is reversible, rate limits, or what happens if the device is offline. For a mutation tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with zero waste, front-loaded with the core action and efficiently explaining the effect. Every sentence earns its place by adding useful context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is incomplete. It lacks information on permissions, error handling, return values, and how it differs from sibling tools, leaving significant gaps for an AI agent to understand full context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('device_id' and 'offset'). The description adds minimal value by clarifying that the offset is for calibration and can be positive or negative, but doesn't provide additional syntax or format details beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Set') and resource ('temperature offset for a thermostat device'), explaining it adjusts measured temperature by a fixed offset. However, it doesn't explicitly differentiate from sibling tools like 'set_temperature' or 'get_temperature_offset', which would require a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as 'set_temperature' or 'get_temperature_offset'. The description only states what it does, not when it's appropriate or what prerequisites might be needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_presetB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a 'Set' operation (implying mutation) and that presets apply 'predefined temperature settings and behaviors,' but lacks critical details: required permissions, whether changes are reversible, effects on other settings, error conditions, or response format. For a mutation tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states the core action with examples, and the second explains what presets do. Every word earns its place, with no redundancy or fluff. It's appropriately sized for a simple tool with two parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (mutation with 2 parameters), no annotations, and no output schema, the description is minimally adequate. It covers the basic purpose and parameter examples but lacks usage guidelines, behavioral details, and output information. It meets the bare minimum for understanding what the tool does but leaves the agent guessing about practical application.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters (location_id and preset). The description adds minimal value beyond the schema: it provides examples of preset values ('home, away, sleep, vacation') and clarifies that location_id targets 'a thermostat or zone,' but doesn't explain parameter interactions or constraints. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Set the preset mode') and target resource ('on a thermostat or zone'), with examples of preset values. It distinguishes the tool's purpose from siblings like set_temperature or set_dhw_mode by focusing on predefined behavior modes rather than direct temperature control. However, it doesn't explicitly contrast with all siblings, missing some differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like set_temperature or set_regulation_mode. It mentions that presets apply 'predefined temperature settings and behaviors' but doesn't clarify use cases, prerequisites, or exclusions. The agent receives no help in choosing between this and other control tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_temperature_offsetB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool returns the current offset value and its valid range, which is useful context beyond basic purpose. However, it doesn't cover other important aspects like whether this is a read-only operation, potential rate limits, authentication needs, or error conditions, leaving gaps for a mutation-free query tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured in two sentences: the first states the purpose, and the second explains the return value. There's no wasted text, and information is front-loaded. It could be slightly improved by integrating usage hints, but it's efficient as is.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (single parameter, no output schema, no annotations), the description is moderately complete. It covers the purpose and return values adequately, but lacks usage guidelines and full behavioral context. For a simple query tool, this is acceptable but not comprehensive, aligning with a minimum viable score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'device_id' clearly documented. The description adds no additional parameter semantics beyond what the schema provides, such as format examples or constraints. According to the rules, with high schema coverage, the baseline is 3, which is appropriate here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the temperature offset (calibration) for a thermostat device.' It specifies the verb ('Get'), resource ('temperature offset'), and clarifies that this is for calibration. However, it doesn't explicitly differentiate from sibling tools like 'get_temperature' or 'get_all_temperatures,' which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions the offset is for calibration if the sensor reads incorrectly, but doesn't specify prerequisites, when to avoid it, or compare it to related tools like 'set_temperature_offset' or 'get_temperature.' This leaves the agent with insufficient context for optimal selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• control_switchB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the tool 'works with' specific device types and controls 'switchable devices', but lacks details on behavioral traits like required permissions, error handling, rate limits, or whether the action is reversible. For a mutation tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with zero waste. It front-loads the core purpose ('Turn a switch or relay on or off') and efficiently adds context about device compatibility and usage scope, making it easy to understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (mutation with 3 parameters), no annotations, and no output schema, the description is moderately complete. It covers the purpose and device scope but lacks details on behavioral aspects and output expectations. It's adequate as a minimum viable description but has clear gaps for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the three parameters (appliance_id, state, model). The description adds no additional meaning beyond what's in the schema, such as explaining parameter interactions or usage examples. Baseline score of 3 is appropriate as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('turn on or off'), the target ('switch or relay'), and the scope ('Plugwise switches, relays, and smart plugs'). It specifies the resource type but doesn't explicitly differentiate from sibling tools like 'set_gateway_mode' or 'set_preset', which also control devices but for different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating 'Works with Plugwise switches, relays, and smart plugs' and 'Use this to control any switchable device', providing context for when to use it. However, it doesn't explicitly mention when not to use it or name alternatives among siblings, such as distinguishing from 'set_preset' for temperature-related controls.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_regulation_modeB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that the tool 'Controls the overall heating system behavior' and describes the modes, but it does not disclose critical behavioral traits such as whether this requires specific permissions, if changes are reversible, potential side effects (e.g., system downtime during bleeding modes), or rate limits. For a mutation tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded. The first sentence states the core purpose, and the second sentence elaborates on the modes with clear examples. Every sentence adds value without redundancy, making it efficient and easy to understand.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation tool with one parameter but significant behavioral implications), no annotations, and no output schema, the description is partially complete. It covers the purpose and parameter semantics well, but lacks details on behavioral aspects like permissions, reversibility, or system impact. For a tool that controls heating system behavior, more context on safety or operational constraints would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context beyond the input schema. The schema has 100% coverage with an enum for 'mode' and a description that mirrors the tool description. However, the tool description provides additional semantics by explaining the purpose of each mode: 'heating' for normal operation, 'off' to disable heating, and bleeding modes for maintenance. This clarifies the practical use of the parameters, compensating for the lack of detailed schema annotations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Set the heating regulation mode. Controls the overall heating system behavior.' It specifies the verb ('Set') and resource ('heating regulation mode'), but does not explicitly differentiate it from sibling tools like 'set_dhw_mode' or 'set_preset', which likely control different aspects of the heating system. This makes it clear but not fully distinguished from alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidelines by listing the modes and their purposes: 'heating' for normal operation, 'off' to disable heating, or bleeding modes for system maintenance. However, it does not explicitly state when to use this tool versus alternatives like 'set_dhw_mode' or 'set_preset', nor does it mention any prerequisites or exclusions. The guidance is useful but not comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_hubsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the data sources ('/hubs folder and in-memory registry') and return format ('hub information including name, IP address, model, and firmware version'), which adds useful context. However, it doesn't cover potential issues like network timeouts, authentication needs, or error handling, leaving gaps for a tool that interacts with external resources.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently conveys the action, sources, and return data. Every part earns its place without redundancy, making it easy to parse and front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is moderately complete. It explains what the tool does and what it returns, but lacks details on behavioral aspects like error conditions or performance. For a read-only listing tool, this is adequate but has clear gaps in operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the schema fully documents the absence of inputs. The description adds no parameter information, which is appropriate here. A baseline of 4 is applied because no parameters exist, and the description doesn't need to compensate for any gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all registered Plugwise hubs') and the resources involved ('from the /hubs folder and in-memory registry'). It distinguishes from siblings like 'add_hub' (which creates) or 'get_devices' (which might return different data), making the purpose unambiguous and well-differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., whether hubs must be registered first), exclusions, or comparisons to siblings like 'get_devices' (which might list devices rather than hubs). Usage context is implied but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_dhw_modeA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It describes what each mode does (e.g., 'boost mode provides immediate hot water heating'), which adds behavioral context, but doesn't disclose critical traits like required permissions, whether changes are reversible, system state effects, or error conditions. For a mutation tool with zero annotation coverage, this is a significant gap in behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose and then efficiently explaining each mode option in a single, well-structured sentence. Every sentence earns its place by adding specific value without redundancy or waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (single parameter mutation), no annotations, and no output schema, the description is partially complete. It explains the purpose and parameter semantics well, but lacks details on behavioral aspects like permissions, reversibility, or system impacts. It's adequate for basic use but has clear gaps for safe agent operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter 'mode' with enum values and descriptions. The description adds value by providing expanded semantics for each mode option (e.g., 'comfort mode maintains higher temperature'), which enhances understanding beyond the schema's basic enum descriptions. With only one parameter, the baseline is high, and the description compensates well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'set' and the resource 'domestic hot water (DHW) heating mode' with specific scope 'for systems with hot water control'. It distinguishes from siblings like set_temperature or set_preset by focusing exclusively on DHW mode control.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by explaining when to use each mode option (e.g., 'Auto mode follows the schedule', 'boost mode provides immediate hot water heating'), but doesn't explicitly state when to use this tool versus alternatives like set_temperature or set_preset for similar heating controls. It implies usage through mode explanations but lacks explicit exclusions or comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• connectA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the core behavior (connection and retrieval of gateway details) and the fallback mechanism for missing host. However, it lacks details on potential side effects (e.g., authentication requirements beyond password hints, network timeouts, or error handling) and does not specify the return format beyond listing fields like model and MAC address.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the main purpose in the first sentence, followed by conditional behavior and return details in a second sentence. Every sentence adds value without redundancy, making it efficient and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (connection with optional parameters) and no output schema, the description is mostly complete: it covers the purpose, usage context, and return information. However, it could be more complete by detailing error cases or the structure of the returned gateway information, which would help an agent handle responses better.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters thoroughly. The description does not add any additional meaning or examples beyond what the schema provides (e.g., it mentions 'If no host is provided' but the schema already states this). Baseline 3 is appropriate as the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('connect to a Plugwise gateway'), identifies the resource types (Adam, Anna, Smile P1, or Stretch), and distinguishes this from sibling tools like 'list_hubs' or 'get_devices' by focusing on establishing a connection and retrieving gateway metadata rather than listing or controlling devices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool (to connect to a gateway and get its info) and includes a fallback behavior ('If no host is provided, automatically connects to the first discovered hub'). However, it does not explicitly state when not to use it or name specific alternatives among the siblings, such as 'list_hubs' for discovery without connection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_devicesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what the tool returns (comprehensive device information including states, sensors, capabilities) and implies it's a read operation ('Retrieve'), but doesn't mention potential limitations like rate limits, authentication requirements, or data freshness. The description adds useful context about scope but lacks operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences that each add value: the first states the core action and scope, the second elaborates on the comprehensiveness of returned data. It's appropriately front-loaded with the main purpose. Minor improvement could be made by combining related concepts more tightly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read tool with no annotations and no output schema, the description provides adequate coverage of what the tool does and returns. However, it could be more complete by mentioning the format of returned data, whether it includes pagination for large device sets, or any prerequisites for successful execution.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema already fully documents the parameter situation. The description appropriately doesn't discuss parameters since none exist, maintaining focus on what the tool does rather than what inputs it accepts.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Retrieve', 'Returns') and resources ('Plugwise devices', 'current states, sensors, and capabilities'). It distinguishes from siblings by specifying it returns comprehensive information for all connected devices, unlike more specific tools like get_temperature or get_all_temperatures.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by stating it retrieves 'all Plugwise devices' with 'comprehensive information', suggesting it should be used when broad device overview is needed rather than specific data points. However, it doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_temperatureA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the return values (measured temperature and target setpoint), which is useful, but does not mention behavioral traits like error handling, rate limits, or authentication needs. The description adds some context but lacks comprehensive behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and followed by return details. Every sentence earns its place with zero waste, making it appropriately sized and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a simple input schema, the description is mostly complete for a read-only tool. It covers purpose and returns, but could improve by addressing potential errors or usage nuances. It's adequate but has minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'device_id' parameter. The description adds no additional meaning beyond what the schema provides, such as examples or constraints. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get'), the resource ('current room temperature and setpoint'), and the target ('for a specific thermostat or zone'). It distinguishes from siblings like 'get_all_temperatures' (which lacks specificity) and 'set_temperature' (which is a write operation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing temperature data for a specific device/zone, but does not explicitly state when not to use it or name alternatives. It provides clear context but lacks exclusions like 'use get_all_temperatures for bulk reads'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_gateway_modeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key behavioral traits: it's a system-wide setting affecting all zones and presets, and it explains the operational impact of each mode (e.g., 'away mode uses reduced temperatures'). However, it doesn't mention permissions, rate limits, or reversibility.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the purpose and followed by explanatory details. Every sentence earns its place by clarifying scope and mode behaviors without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (system-wide setting), no annotations, and no output schema, the description is fairly complete—it explains the effect of each mode. However, it lacks details on response format or error conditions, which would be helpful for a mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'mode' parameter with enum values and descriptions. The description adds minimal value by restating the modes but doesn't provide additional syntax or format details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Set') and resource ('gateway mode'), with specific options (home, away, vacation) that distinguish it from siblings like set_temperature or set_preset. It's not a tautology of the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by explaining what each mode does (e.g., 'Home mode uses normal schedules'), but it doesn't explicitly state when to use this tool versus alternatives like set_preset or set_regulation_mode, nor does it provide exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_temperatureA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key traits: it's a write operation ('Set'), supports multiple system types, and specifies that 'Changes take effect immediately', which is crucial for understanding impact. However, it does not cover aspects like error conditions, permissions needed, or rate limits, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core action and efficiently covers key details in two sentences without any wasted words. Each sentence adds value: the first specifies the tool's function and system support, and the second clarifies the immediate effect, making it appropriately sized and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a write operation with 4 parameters) and no annotations or output schema, the description does a good job by explaining the action, system types, and immediate effect. However, it could be more complete by addressing potential side effects, error handling, or response format, which are important for a mutation tool without structured output documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds marginal value by clarifying that setpoint is for single-setpoint systems and setpoint_low/high are for heat pump systems, but this mostly reiterates schema details without providing additional syntax, constraints, or usage examples beyond what's in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Set') and resource ('temperature setpoint on a thermostat or zone'), specifying the exact action. It distinguishes from siblings like 'get_temperature' (read vs. write) and 'set_temperature_offset' (setpoint vs. offset adjustment), making the purpose specific and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning support for different system types (single vs. heat pump) and that changes take effect immediately, which provides some context. However, it lacks explicit guidance on when to use this tool versus alternatives like 'set_preset' or 'set_regulation_mode', and does not state any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_hubA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses key behaviors: network scanning, file storage location (/hubs folder), and file format (JSON). However, it lacks details on error handling, network requirements, or what happens if the hub already exists. It adequately describes the core operation but misses some behavioral nuances.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states the action and key parameter, the second explains the process and storage. Every sentence adds essential information with zero waste, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no annotations and no output schema, the description covers the purpose, process, and outcome well. It explains what the tool does, how it works (network scan, file storage), and the parameter's real-world meaning. The main gap is lack of output details, but given the tool's simplicity, this is a minor omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by explaining that the hub name is 'used as password' and is 'the unique identifier printed on the back of your Plugwise device,' providing practical context beyond the schema's technical description. This elevates the score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Add'), resource ('new Plugwise hub'), and specific action ('by providing its name'). It distinguishes from siblings like 'list_hubs' by specifying creation vs. listing, and from 'connect' by focusing on initial setup rather than ongoing connection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('Scans the network to find the hub') and storage outcome ('stores it in the /hubs folder as a JSON file for future use'), but does not explicitly state when NOT to use it or name alternatives. It suggests this is for initial hub setup, which provides clear guidance but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_all_temperaturesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the tool's behavior by describing the return data (measured values, setpoints, control states, climate modes) and scope (all devices), but does not mention potential limitations like rate limits, authentication needs, or data freshness. It adds useful context but lacks operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and followed by details on return data. Every sentence adds value: the first defines the action and scope, the second elaborates on the comprehensive output. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (retrieving data for all devices), no annotations, and no output schema, the description is fairly complete—it explains what data is returned and for which devices. However, it could improve by mentioning output format or pagination, but the absence of an output schema is partially compensated by the descriptive return details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately does not discuss parameters, focusing on the tool's purpose and output. Baseline is 4 for zero parameters, as it avoids unnecessary details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('current temperatures and setpoints for all thermostats and zones'), specifying it returns comprehensive data for every temperature-capable device. It distinguishes from siblings like 'get_temperature' (likely single device) and 'get_devices' (broader device info).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving temperature data across all devices, but does not explicitly state when to use this versus alternatives like 'get_temperature' (single device) or 'get_devices' (general device list). It provides clear context but lacks explicit exclusions or named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_notificationA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that this is a destructive operation ('Delete all gateway notifications') and specifies the target ('error messages or warnings'), but it does not mention potential side effects, permissions required, or confirmation prompts. The description adds some behavioral context but lacks completeness for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with zero waste. The first sentence states the action and resource, and the second provides usage guidance, making it front-loaded and efficiently structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation with 0 parameters, no output schema, and no annotations), the description is reasonably complete. It explains what the tool does and when to use it, but it could benefit from additional details on behavioral aspects like irreversible effects or error handling, which are not covered by other structured fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description does not add parameter details beyond the schema, but since there are no parameters, a baseline of 4 is appropriate as it does not need to compensate for any gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Delete all gateway notifications') and the resource ('gateway notifications'), distinguishing it from siblings like 'reboot_gateway' or 'get_devices'. It also specifies the purpose ('clear error messages or warnings'), making the verb+resource combination precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Use this to clear error messages or warnings from the Plugwise gateway.' This provides clear context for its application, distinguishing it from other tools that might manage devices or settings rather than notifications.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reboot_gatewayA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's impact ('temporarily disconnect all devices', 'interrupt heating/cooling control'), timing ('1-2 minutes to fully restart'), and caution needed, providing comprehensive behavioral context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each serve distinct purposes: the first states the action and caution, the second provides timing information. There's no wasted language or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter destructive operation with no annotations or output schema, the description provides excellent context about impacts and timing. It could slightly improve by mentioning whether user confirmation is needed or what happens after reboot completes, but it's largely complete for this tool type.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so no parameter documentation is needed. The description appropriately focuses on behavioral aspects rather than repeating parameter information that doesn't exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('reboot') and target resource ('Plugwise gateway'), distinguishing it from sibling tools like 'set_gateway_mode' or 'control_switch'. It precisely communicates the tool's function without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear contextual guidance ('Use with caution as this will temporarily disconnect all devices and interrupt heating/cooling control'), indicating when this tool should be used carefully. However, it doesn't explicitly mention when to choose alternatives like 'set_gateway_mode' for non-reboot adjustments.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.