Server Quality Checklist

Tool Scores

• diagnose_deviceC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is clear. The description adds no additional behavioral context (e.g., permissions, response format, side effects) beyond what annotations convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence listing four items, which is concise but lacks structure (e.g., bullets or categorization). Every word serves a purpose, but the listing style could be clearer.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description omits what the tool returns or how diagnostics are presented. The tool's role relative to siblings is implied but not explained, leaving gaps for an agent to fully understand its usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not clarify the device_id parameter—no format, constraints, or relationship to the listed checks is explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks bindings, encrypted credential presence, route policy, and LAN read. It uses a specific verb ('Check') and lists distinct diagnostic areas, distinguishing it from sibling tools like get_device or refresh_device, though not explicitly contrasting them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use diagnose_device versus alternatives such as get_device or discover_lan_devices. No prerequisites, context, or exclusion criteria are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discover_lan_devicesB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide idempotentHint true and destructiveHint false. The description adds 'reconcile identities,' implying state changes, but lacks further context on side effects or prerequisites. Adequate given annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no fluff, front-loaded with action. Could be clearer but efficient for a single-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema covers return values, annotations cover safety. Description omits usage scenarios and parameter constraints, but overall adequate for a simple discovery tool with a single optional parameter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates by noting target may be an IP/broadcast address. However, it omits format, required vs optional, and default behavior, leaving gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool scans LAN ACs and reconciles identities, providing a specific verb and resource. It distinguishes from siblings like list_devices (known devices) by focusing on discovery, though 'LAN ACs' is somewhat cryptic.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like list_devices or get_device. The context of target being an IP/broadcast address is implied but not elaborated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_devicesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds 'cached state,' implying the list may not be real-time, but lacks other behavioral details like pagination or response size.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with verb and resources, no redundant words, perfectly concise for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (no params, output schema exists, annotations cover safety), the description sufficiently explains the tool's purpose and output scope. It could mention that the list is complete or lacks filtering, but it's adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the description adds value by listing what is included (air conditioners, bindings, routes, cached state), which helps the agent understand the output. The baseline is 4 for 0 params.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'list' with specific resources ('registered logical air conditioners, bindings, routes, and cached state'), clearly distinguishing it from sibling tools like get_device (single device) or discover_lan_devices (discovery).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus siblings (e.g., get_device for details, refresh_device for updates). The agent must infer from tool names alone without explicit context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_deviceA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds that it 'includes capabilities', which is a minor behavioral detail. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single 10-word sentence that is front-loaded with the verb and resource. Every word is necessary and no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with one parameter and an output schema, the description conveys the core purpose and parameter meaning. It could optionally mention that the output is a device object, but the output schema covers that. Slightly more context about when to use this over siblings would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage for device_id. The description explains that the parameter is a 'midea-mcp internal UUID', providing essential context that the schema lacks. This fully compensates for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' and resource 'one AC', clearly distinguishing from sibling tools like list_devices (plural) and diagnose_device. It mentions the identifier type (midea-mcp internal UUID) and that capabilities are included.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives (e.g., list_devices for all devices, diagnose_device for diagnostics). The description only states what it does, not when to choose it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_deviceA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds context beyond annotations: it discloses the side effect of updating local cache and the LAN mechanism, matching the readOnlyHint and idempotentHint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given few parameters, presence of annotations and output schema, the description provides sufficient context for the tool's purpose and behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description does not explain the device_id parameter, though its purpose is implicitly clear from context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads the actual AC state via LAN and updates the local cache, specifying verb and resource, and distinguishes from siblings like set_device_state.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when fresh data is needed (vs cached), but does not explicitly state when to use vs alternatives or provide exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_device_stateA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate idempotentHint=true and destructiveHint=false. The description adds valuable context with 'never retry an uncertain LAN write via Cloud', clarifying the one-shot nature and retry policy beyond what annotations provide. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, short sentence that conveys the core purpose and a critical behavioral note. No wasted words; front-loaded with the action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 params, no enums) but has an output schema and nested objects. The description covers the main behavior and retry policy, but lacks details about the 'changes' parameter, prerequisites (e.g., device must exist), and expected outcomes. Basic but missing some context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain parameters. It only mentions 'AC desired state' but does not define the 'changes' object or its allowed keys. The 'device_id' is also not explained. This leaves the agent with insufficient information for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Set AC desired state') and the resource ('AC desired state'), using a specific verb. It distinguishes itself from sibling tools (all reads or diagnostics) by indicating a state-setting operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The instruction 'never retry an uncertain LAN write via Cloud' provides a clear behavioral guideline for when to avoid retrying. It implies usage for setting state without retry, but does not explicitly state when to use this tool versus alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.