Server Quality Checklist

Tool Scores

• get_outletA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does add a useful behavioral detail: 'freshly read from the hub' indicates the data is not cached. However, it does not explicitly state that this is a read-only operation, nor does it describe error handling or what happens if the outlet does not exist. For a simple getter, this is adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences plus a focused Args block. Every element earns its place—the first sentence states purpose, the second adds a key behavioral nuance, and the Args block adds necessary parameter context. No superfluous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter getter, the description is largely complete. It clearly defines the tool's purpose and parameter. The output schema (though not shown) likely covers return format, so the description need not explain that. The main gap is the lack of error/edge-case behavior, but given the low complexity, the description is sufficient to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It does so effectively for the only parameter, outlet_id, by specifying that it is 'as returned by list_outlets.' This adds provenance and usage context beyond the schema's generic string type, significantly aiding correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get the current state of a single outlet, freshly read from the hub.' This specifies a precise verb (get), resource (single outlet), and distinguishes it from sibling tools like list_outlets (multiple outlets) and set_outlet/toggle_outlet/power_cycle (mutations).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: this tool is for retrieving the current state of one specific outlet. It implicitly indicates that list_outlets is the source for the outlet_id, but it does not explicitly name alternatives or state when not to use this tool. The context is clear, but exclusions are not stated, so it does not receive a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• toggle_outletA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the core behavior (changing to the opposite state) and the return value ('return the new state'). It also provides a useful constraint that outlet_id must be 'as returned by list_outlets.' This goes beyond a generic 'toggle' statement, though it does not cover error cases or side effects beyond the state change.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a clear one-sentence purpose, followed by a concise Args section. No unnecessary words or repetition. The front-loaded first sentence immediately conveys the tool's action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required parameter) and presence of an output schema, the description covers the essential aspects: what it does, how to supply the ID, and that it returns the new state. It could have mentioned alternatives for completeness, but for a toggle operation this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so by clarifying the single parameter: 'outlet_id: The outlet's id, as returned by list_outlets.' This adds meaning beyond the schema's title 'Outlet Id' by specifying the source and expected format, which is valuable for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Flip an outlet to the opposite of its current state and return the new state.' This uses a specific verb ('flip') and resource ('outlet'), and distinguishes it from siblings like set_outlet (which sets a specific state) and get_outlet (which reads state).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by describing the toggle behavior, but does not explicitly state when to use this tool versus alternatives. It lacks any mention of 'use this instead of set_outlet when you want to flip state' or exclusions. Sibling tools exist, but no contrast is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_outletA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses the key side effect (switching) and adds the behavioral detail that it returns the state read back from the hub. It could mention failure modes or prerequisites, but for a simple control tool this is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, front-loads the action, and includes only necessary details. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with an output schema, the description adequately covers action and parameters. It lacks explicit comparison to siblings, but the low complexity and output schema reduce the need for additional detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description fully compensates by explaining both parameters: outlet_id is the id from list_outlets, and on is a boolean to switch on/off. This adds clear meaning beyond the bare schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Switch an outlet on or off and return its state as read back from the hub.' The verb 'switch' and resource 'outlet' are specific, and the read-back phrase distinguishes it from toggle_outlet and power_cycle.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for setting a specific on/off state but does not explicitly mention when to use this tool over alternatives like toggle_outlet or power_cycle. No exclusions or alternative references are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• power_cycleA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses blocking behavior, timeout constraint, and guarantees power restoration even if the wait is interrupted. It also notes off_seconds range. These are meaningful behavioral traits beyond the schema; however it omits error conditions or what happens on failure, but output schema may cover returns.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a clear one-line definition, followed by use-case context, a timeout warning, and a compact Args list. Every sentence adds information; no filler, though it is longer than the minimal example.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and minimal schema, the description covers purpose, usage context, blocking/timeout behavior, interruption guarantee, and parameter semantics. An output schema exists, so return values needn't be explained. It is sufficiently complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The Args section explains both parameters: outlet_id sourced from list_outlets, and off_seconds as a range (0.5 to 300) and as the powered-off duration. This adds meaning the schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Power-cycle whatever is plugged into an outlet: switch it off, wait, switch it back on.' This clearly distinguishes it from sibling tools like toggle_outlet or set_outlet by naming the off-wait-on sequence and intended use for rebooting hardware.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states the intended scenario: 'Intended for rebooting hardware attached to a smart plug, such as a development board.' This gives clear context, though it does not explicitly contrast with alternatives like toggle_outlet or set_outlet. Thus clear context but no explicit exclusionary guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_outletsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full transparency burden. It clearly states the tool lists all outlets and discloses the exact return fields (id, name, room, on/off state, reachability), which is the core behavioral trait. While it does not mention error handling or empty results, the description adequately communicates the tool's read-only enumeration behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the core purpose ('List every outlet'). The second sentence provides return fields and usage guidance without redundancy. Every word earns its place, making this an exemplary concise description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter list tool, the description is fully complete: it states scope, output fields, and how the ids integrate with sibling tools. The presence of an output schema reduces the need for detailed return descriptions, and the description provides enough context for an agent to invoke the tool correctly and use its results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is an empty object with zero parameters, so parameter semantics are fully covered by the schema. Per the rubric, 0 parameters earns a baseline of 4. The description adds context by confirming the tool lists all outlets without filters, which enriches the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and the resource 'outlets (smart plug) paired to the DIRIGERA hub', immediately establishing the tool's scope. The phrase 'every outlet' distinguishes it from sibling tools like get_outlet, set_outlet, toggle_outlet, and power_cycle, which operate on individual outlets. Purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs 'Use the id with the other outlet tools', clearly positioning this tool as the enumeration/selection step before applying actions via siblings. This gives strong guidance on when to use the tool and how its output feeds into other operations, effectively distinguishing its role from the alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.