Server Quality Checklist

Tool Scores

• connect_deviceB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the 'Unlocks app lifecycle tools' behavioral trait, which is useful state-changing context, but doesn't describe connection limitations, timeout behavior, platform requirements beyond schema, or any failure modes. For a connection tool with no annotation coverage, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two crisp sentences—one stating the action and identification modes, one stating the unlock side effect. Zero fluff, but 'Unlocks app lifecycle tools' is somewhat vague jargon that could benefit from elaboration, and the connection logic is packed into a single sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No annotations and no output schema, so the description must carry the behavioral burden. It covers the invocation modes and a key side effect, but doesn't mention what a successful connection returns, what errors look like (host unreachable, unknown name), or whether the tool is idempotent. Adequate but with notable gaps for a connection tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so all three parameters (host, name, platform) are documented in the schema itself. The description adds the relational constraint that platform is 'Required with host' (which the schema enum hint also implies) and that name comes from devices.yaml. It adds slight value by connecting parameters to the two invocation modes, but largely leans on the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource ('Connect to a TV') with explicit modes of identification (configured name or host+platform). It also notes a side effect ('Unlocks app lifecycle tools'), which adds purpose context, though it doesn't explicitly distinguish itself from siblings like list_devices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states the two identification modes (name or host+platform) which implicitly tells the agent how to invoke it, and the sibling list_devices suggests a workflow. However, there's no explicit guidance about when to choose name vs host+platform, or when NOT to use this tool. The 'Unlocks app lifecycle tools' hint implies it's a prerequisite step but doesn't spell out a decision rule.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_devicesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It notes the tool handles both configured and live-discovered TVs, and includes reachability info, which adds some behavioral context. However, it doesn't disclose whether this performs network discovery, how long it may take, whether it requires prior configuration, or what 'reachability' entails (does it ping each TV?). The lack of annotations makes these gaps more significant.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that fully captures the tool's purpose with zero wasted words. It front-loads the key information (what is listed) and adds the technical brand/platform detail (Samsung Tizen / LG webOS) plus the meaningful differentiator (reachability) that helps an agent understand the result set. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, no-output-schema listing tool, the description is reasonably complete. It identifies the device brands/platforms and what the listing includes (reachability, configured and live-discovered). Minor gaps exist around what fields the returned list contains and whether discovery is active, but given the tool's simplicity, the description covers the essential context an agent needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing for the description to explain about arguments. By the rubric, 0 params earns a baseline of 4. The description appropriately needs no parameter elaboration since the schema is empty and coverage is 100%.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') plus a clear resource ('configured and live-discovered Samsung (Tizen) / LG (webOS) TVs with reachability'). It clearly communicates what is returned. It distinguishes itself from the sibling connect_device (which would be connection-oriented) by focusing on listing existing devices rather than establishing connections.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is used to enumerate TVs before connecting to one, but doesn't explicitly state when to use it vs connect_device or when not to use it. No explicit alternative is named or usage context (e.g., 'call this before connect_device') is given. The purpose is clear enough that usage is largely implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• docsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. The description does disclose that calling without a topic lists topics, which is a behavioral trait not derivable from the schema. However, it doesn't disclose the return format, whether content is paginated, or the depth/format of the documentation returned. For a read-only docs tool this is acceptable but could be richer.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero waste. The first sentence establishes purpose and topic domains; the second sentence gives the key usage behavior for the optional parameter. Every word earns its place and the most important operational detail (call without topic to list) is front-loaded into the second sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple single-optional-parameter tool with 100% schema coverage and no output schema, the description is fairly complete. It identifies the pain-point domains and the zero-argument behavior. A minor gap: it doesn't specify what the documentation content looks like when called with a topic, but the schema enumerates topics clearly. For the complexity level, this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the topic parameter is fully documented in the schema with its allowed enum values. The description adds the behavioral detail that omitting the topic lists available topics, which complements the schema. Since the schema does the heavy lifting on parameter meaning, baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states this is a docs tool for platform pain points (signing, dev mode, packaging, pairing) and distinguishes its purpose by noting it covers deep documentation topics. However, it doesn't explicitly differentiate from sibling tools list_devices and connect_device, though the context makes it fairly obvious docs is informational. The verb 'deep docs' and specific pain points give a clear, specific purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Call without a topic to list topics,' which is a clear usage guideline for the zero-parameter case. It gives some guidance that topics map to specific pain points (signing, dev mode, packaging, pairing), though it doesn't explicitly state when NOT to use this tool versus alternatives like connect_device. The sibling tools are operational (list/connect), so context implies docs is for reference, but that contrast is not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.