Server Quality Checklist

Tool Scores

• lock_doorA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden for behavioral disclosure. It states the action but does not mention side effects beyond locking, required permissions, failure modes, idempotency, or what response to expect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences front-load the action and then give the essential prerequisite. There is no filler or redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one well-documented parameter and a clear action, so the description is minimally viable. However, with no annotations and no output schema, it leaves return/error behavior and side-effect expectations unexplained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the schema already says device_id comes from list_locks. The description repeats that prerequisite but adds no additional parameter semantics beyond what the schema provides, so the baseline score applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('Lock a Kwikset door') and the prerequisite ('Get device_id from list_locks first') further clarifies its role. It is clearly distinct from unlock_door and list_locks without needing to read their schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear usage context by instructing the agent to obtain device_id from list_locks first. It does not explicitly mention when not to use it or name unlock_door as the alternative, but the command's action is self-explanatory enough for basic routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_lock_statusA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the disclosure burden. It indicates a read-only retrieve operation by the word 'Get' and lists the data returned, but it does not explicitly state side-effect-free behavior, failure/error behavior for a bad device_id, or whether status is live vs cached.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences contain only useful information: what the tool returns and the required prerequisite. The action and scope are front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with no output schema, the description covers what will be returned and how to get the parameter. It omits edge-case behavior, but the essential calling context is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, giving baseline 3. The description adds value beyond the schema by telling the agent where to obtain device_id and that it must be obtained first from list_locks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb ('Get'), names the exact resource ('current status ... for one lock'), and enumerates the returned fields (status, battery level, model/serial). This clearly differentiates it from sibling tools like list_locks (listing) and lock_door/unlock_door (actuation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear prerequisite and data source: 'Get device_id from list_locks first.' It does not explicitly say when not to use this tool, but the operation is unique among the siblings, so the workflow instruction is the main needed guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_locksA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It clearly states what the tool returns (locks with status, battery, and home) and implies that the device_id is obtainable from the result, which is essential for subsequent operations. It does not explicitly mention side effects (likely read-only) or edge cases, but for a simple listing tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, with the core purpose and returned content front-loaded and the usage guidance provided second. There is no wasted wording or off-topic detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately covers what the tool does, what it returns, and when to use it relative to siblings. It does not mention pagination or ordering, but given the simple nature of the tool and absence of parameters or output schema, this is a minor gap and does not hinder correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so no parameter documentation is needed. The baseline of 4 for zero-parameter tools applies because there is nothing to explain, and the description does not omit any required parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and a specific resource ('every Kwikset lock on the account'), and enumerates the returned fields (status, battery, home). This clearly distinguishes it from sibling tools that target individual locks (lock_door, unlock_door, get_lock_status).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Call this first to find a lock's device_id before locking/unlocking/checking it.' This gives a direct and clear usage directive that routes the agent to this tool before the sibling operations, and implies when not to use it (after you have the device_id).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• debug_raw_devicesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full disclosure burden. It clearly states this is a diagnostic read-only dump and explains the failure mode it addresses. It doesn't explicitly state side effects or caveats, but 'dumps' combined with 'Diagnostic tool' implies no mutation, and the scope is well defined.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences, with the most essential fact ('Diagnostic tool') front-loaded. Every clause earns its place: the source of data, the triggering condition, and the reason for the null-field symptom. No filler or repetition exists.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully covers what an agent needs: when to invoke this tool, what it returns, and why the output matters. No parameters or output schema are required, and the diagnostic purpose is unlikely to require further detail about return values or error handling for correct selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and a 100% schema description coverage (vacuously, since the schema is empty). The description explains the nature of the output ('raw, unprocessed... JSON') and the diagnostic trigger, which is sufficient because no parameter guidance is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('dumps the raw, unprocessed home/device JSON') and a clear resource ('straight from Kwikset's API'). It identifies itself as a 'Diagnostic tool,' which immediately distinguishes it from the sibling commands that perform listing, status, and locking actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit condition for when to use it: 'Use this only when another tool returns null fields.' It also explains the reasoning (Kwikset renamed a field) and the expected value (shows real field names), leaving no ambiguity about when to choose this tool over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• unlock_doorA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility, and it delivers. It flags the action as physical-security-sensitive, states the confirmation requirement, and discloses that calling with confirm=false is a no-op that returns a reminder instead of unlocking.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. It front-loads the core action, then packs the security warning, caller precondition, and confirmation behavior into compact, high-value language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter physical-security action with no output schema, the description covers everything needed to invoke it correctly: the requirement for explicit user consent, the needed confirmation flag, and the no-op fallback. Nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are already documented. The description adds meaningful behavioral context beyond the schema by specifying that confirm=true is required and that false leads to a no-op reminder, which is not fully captured in the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Unlock a Kwikset door.' It is immediately distinguishable from sibling tools like lock_door and list_locks, leaving no ambiguity about the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: only after the user has explicitly asked to unlock this specific door. It also requires confirm=true and explains that confirm=false is a deliberate no-op, providing clear behavior boundaries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.