Server Quality Checklist

Tool Scores

• control_deviceD
  1.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. 'デバイスを制御します' only states the action without revealing any behavioral traits: it doesn't indicate whether this is a read or write operation (though 'control' implies mutation), what permissions are required, whether changes are reversible, what side effects occur, or what the response looks like. For a device control tool with zero annotation coverage, this is completely inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While the description is technically concise (a single phrase), this represents under-specification rather than effective brevity. The single sentence 'デバイスを制御します' doesn't earn its place by providing meaningful information beyond what's already implied by the tool name. True conciseness balances brevity with information density, which this description fails to achieve.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a device control tool (implying mutation) with no annotations, no output schema, and a minimal description, the description is completely inadequate. The agent needs to understand what 'control' means operationally, what happens when commands are executed, what the response contains, and how this differs from sibling tools. The current description provides none of this essential context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with both parameters ('deviceId' and 'command') fully documented in the schema. The description adds no additional meaning about parameters beyond what the schema already provides. According to the scoring rules, when schema coverage is high (>80%), the baseline score is 3 even with no parameter information in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'デバイスを制御します' (controls devices) is a tautology that essentially restates the tool name 'control_device' in Japanese. It doesn't specify what kind of control is performed or differentiate this tool from its siblings 'get_device_status' and 'list_devices'. While it indicates the general domain (device control), it lacks specificity about the verb and resource scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of when to use 'control_device' instead of 'get_device_status' or 'list_devices', nor any context about prerequisites, appropriate scenarios, or exclusions. The agent receives no usage direction beyond the tool name itself.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_device_statusC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description only states what the tool does ('get device status') without mentioning any behavioral traits such as whether it's read-only, requires authentication, has rate limits, returns specific data formats, or handles errors. This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is appropriately sized and front-loaded, with zero waste, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what the status includes (e.g., online/offline, metrics), how results are returned, or any error conditions. For a tool with no structured behavioral or output information, the description should provide more context to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'deviceId' documented as 'デバイスID' (device ID). The description adds no additional meaning beyond what the schema provides, such as explaining what constitutes a valid device ID or how to obtain it. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's purpose ('デバイスのステータスを取得します' translates to 'Get device status'), which is a clear verb+resource combination. However, it doesn't differentiate from sibling tools like 'list_devices' (which might list multiple devices) or 'control_device' (which performs actions rather than retrieving status). The purpose is understandable but lacks sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There are no explicit instructions about when to use 'get_device_status' instead of 'list_devices' or 'control_device', nor any context about prerequisites or exclusions. This leaves the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_devicesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. The description only states what the tool does ('get device list') but doesn't disclose behavioral traits like whether this requires authentication, returns paginated results, has rate limits, or what format the output takes. For a tool with zero annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise single sentence that directly states the tool's purpose. No wasted words or unnecessary elaboration. Perfectly front-loaded with the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero annotations, no output schema, and a simple list operation, the description is incomplete. It doesn't explain what the output looks like (list format, fields included), whether there are any constraints or prerequisites, or how this differs meaningfully from sibling tools. The description provides minimal context beyond the basic action.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage (empty schema). The description doesn't need to explain parameters since there are none. Baseline for 0 parameters is 4, as the description correctly doesn't attempt to document nonexistent parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'デバイス一覧を取得します' (Get device list) clearly states the action (取得/retrieve) and resource (デバイス一覧/device list). It distinguishes from siblings like 'control_device' (which modifies) and 'get_device_status' (which gets status of individual devices). However, it doesn't specify scope (all devices? filtered?) which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The description doesn't mention when to use list_devices versus get_device_status (for individual device status) or control_device (for device control). Usage context is implied but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.