Server Quality Checklist

Tool Scores

• get_work_stateC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a query operation but doesn't mention whether it's read-only, requires authentication, has rate limits, or what specific status information is returned. The description provides minimal behavioral context beyond the basic operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise with clear sections for Args and Returns, but the 'Query robot working status' statement is somewhat vague. The structure is functional but not optimally front-loaded with the most critical information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a query tool with no annotations and no output schema, the description is insufficient. It doesn't explain what specific status information is returned, what format the dictionary contains, or any error conditions. Given the complexity of querying device status, more context is needed for effective tool use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single 'nickname' parameter. The description repeats the parameter documentation verbatim without adding any additional meaning or context beyond what's in the schema, meeting the baseline expectation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Query' and resource 'robot working status', making the purpose understandable. However, it doesn't differentiate from sibling tools like 'get_device_list' which might also provide status information, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'get_device_list' or when not to use it. The description lacks any context about prerequisites or appropriate scenarios for querying robot status.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_device_listC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool returns a dictionary of robot nicknames, which implies a read-only operation, but doesn't disclose behavioral traits like authentication needs, rate limits, error handling, or whether it's a safe operation. The description is minimal and lacks crucial context for an agent to understand how it behaves beyond basic output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise with two short sentences: 'Query robot list' and 'Returns: Dict: Dictionary containing list of robot nicknames'. It's front-loaded with the purpose and efficiently states the return type. However, it could be slightly more structured by clarifying the tool's role relative to siblings, but it earns high marks for brevity and lack of waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 0 parameters, no annotations, and no output schema, the description provides minimal but adequate context for a simple query tool. It states what the tool does and the return format, but lacks details on behavioral aspects like safety or usage guidelines. For a tool with no structured data, it meets the minimum viable threshold but has clear gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter info, which is appropriate here. Baseline is 4 for zero parameters, as the schema fully covers the absence of inputs, and the description doesn't need to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Query robot list' which indicates the tool retrieves robot information, but it's vague about what specific data is returned beyond 'list of robot nicknames'. It doesn't distinguish from sibling tools like get_work_state, set_charging, or set_cleaning, which have different purposes. The description provides a basic purpose but lacks specificity about scope or differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The description doesn't mention prerequisites, context, or exclusions. It fails to explain how this tool relates to siblings like get_work_state (which might provide status info) or set_charging/set_cleaning (which are mutation tools). No explicit or implied usage scenarios are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_cleaningC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the action types (start, resume, pause, stop) but lacks critical details such as required permissions, error conditions, or side effects (e.g., whether the robot moves immediately). This is insufficient for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the main purpose. The structured 'Args' and 'Returns' sections are clear, though the formatting could be slightly more polished (e.g., using bullet points). Overall, it's efficient with little waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of controlling a robot (a mutation tool with no annotations and no output schema), the description is incomplete. It lacks information on return values (only states 'Dict' without details), error handling, and how it integrates with sibling tools, leaving significant gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters fully. The description adds minimal value by restating the 'act' parameter's options in a slightly different format but doesn't provide additional context like examples or edge cases beyond what the schema specifies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Start robot cleaning') and resource (robot), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'set_charging' or 'get_work_state', which would require a more specific comparison to achieve a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. For example, it doesn't mention prerequisites like needing a robot device from 'get_device_list' or how it relates to checking status with 'get_work_state'. This leaves the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_chargingB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the robot action types but doesn't cover important aspects like whether this requires specific permissions, what happens if the robot is already charging, whether the action is reversible, or potential error conditions. The return value description ('Dictionary containing execution results') is too vague.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately brief with a clear purpose statement followed by structured Args and Returns sections. However, the parameter descriptions in the Args section are redundant with the schema and could be more concise. The overall structure is logical and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what the execution results dictionary contains, what success/failure looks like, or important behavioral constraints. Given the complexity of controlling physical robot behavior, more context about limitations, safety considerations, or state dependencies would be needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters completely. The description repeats the parameter information verbatim without adding any additional context, syntax examples, or format details beyond what the schema provides. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Make robot return to charging station') and identifies the resource (robot). It distinguishes from sibling tools like 'set_cleaning' by focusing on charging behavior rather than cleaning. However, it doesn't explicitly contrast with 'get_device_list' or 'get_work_state' which are read-only operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the parameter descriptions ('go-start begin returning', 'stopGo stop returning'), suggesting when to use different actions. However, it doesn't provide explicit guidance on when to choose this tool over alternatives like 'set_cleaning' or mention prerequisites such as robot availability.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.