Server Quality Checklist

Tool Scores

• appC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description fails to disclose behavioral traits like side effects, required permissions, or what happens when actions conflict (e.g., launching an already running app). Only lists actions without behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely short and front-loaded with the key actions. No wasted words, though could benefit from slightly more detail without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 parameters and no output schema, the description is too minimal. It omits overall purpose, platform context, and return value behavior, leaving gaps despite high schema coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 86%, and most parameters have descriptions. The description adds little beyond listing actions already in the enum, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description lists key actions (launch, stop, install, list) indicating it manages app lifecycle, clearly distinguishing from sibling tools like device or flow. However, it could be more specific about the platform scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives or under what conditions. The actions imply usage but exclusions or context are missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inputC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full burden. It mentions actions and associated parameters but discloses no behavioral traits (e.g., side effects, permissions, error behavior). For a complex input tool, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: one sentence that front-loads the purpose and lists action-parameter mappings. No redundancy, though it sacrifices completeness for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 19 parameters, 6 sibling tools, and no output schema, the description is too incomplete. It omits return values, platform-specific behavior (e.g., iOS vs Android), and when to use each action. Much context is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 95%, so baseline is 3. The description adds value by grouping parameters per action (e.g., 'coords or text/id/label/index' for tap), but does not cover all 19 parameters (e.g., hints, platform). It improves on the schema but not extensively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool handles input actions and lists specific actions (tap, double_tap, etc.), making it distinct from sibling tools like 'device' or 'ui'. However, it lacks a strong verb indicating what the tool does (e.g., 'performs' or 'simulates').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'ui' or 'device'. The description merely lists actions without explaining contexts or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• systemC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It hints at file transfer, shell execution, clipboard manipulation – all potentially destructive – but does not disclose side effects, required permissions, or safety considerations. For a tool that runs shell commands, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded, listing categories in a clear bullet-like fashion. However, it uses periods and line breaks inefficiently, and could be more structured. Still, it respects conciseness without excessive verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 14 parameters and many distinct actions, the description is insufficient. It doesn't explain return values (no output schema), error conditions, or how different actions behave across platforms. The complexity is high, and the description fails to cover most of it beyond a high-level list.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is high (93%) and many parameters have descriptions in the schema. The description adds little beyond listing action categories; it does not explain how parameters interrelate or provide additional context for the 'action' enum. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description lists 'System operations' and breaks down into categories like clipboard, permissions, files, shell, logs, which gives a good overview. However, it doesn't explicitly state the verb-resource pattern for each action, leaving some ambiguity about what 'system' does exactly. It distinguishes from siblings like 'app' by being system-level, but could be more precise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs. alternatives. There is no mention of context, prerequisites, or exclusions. The description simply enumerates sub-categories without advising when each is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• repl_expectB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses blocking behavior and the four termination conditions, which is helpful. However, it does not mention whether the tool is destructive, requires any specific session state, or what happens if the session is already closed. A 3 is appropriate as it provides some behavioral context but is not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words. It front-loads the core behavior ('Block until...'), making it easy to scan. Every part of the sentence carries essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters (1 required), no output schema, and no annotations, the description should explain what the tool returns, how to interpret the result (which event fired), and any state changes. It provides none of that. The tool has a potentially complex return behavior (return different data for each termination condition), which is left entirely unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all 4 parameters. It only names the parameter 'regex' implicitly ('prompt regex'), but does not explain 'id', 'idleMs', or 'timeoutMs' meanings or formats. The default values in the schema help slightly, but the description adds almost no parameter-level detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool blocks until one of several events occurs (prompt regex match, session idle, child exit, timeout). This distinguishes it from siblings like repl_send (which sends input) or repl_snapshot (which captures state), though it could be more specific about the relationship to repl sessions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explain when to use this tool versus alternatives like repl_send or repl_snapshot. For example, if a prompt is expected after sending input, an agent might not know to call repl_expect instead of just waiting. No prerequisites or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• repl_sendB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries full burden. It discloses the newline default behavior, but omits important traits like whether input is echoed, session readiness requirements, or error handling. The single behavioral detail is insufficient for a mutation tool with no other sources.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero waste. The first sentence front-loads the purpose, the second adds a critical default behavior. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 3 parameters, the description should provide more context (e.g., how to obtain session id, what happens after writing, error cases). It covers only the basic action and one default, leaving significant gaps for a reliable agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must explain all parameters. It adds meaning for 'text' and 'newline', but entirely omits 'id' (the session identifier), which is required. This gap leaves the agent unable to understand which session receives the text.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Write' and resource 'text to a REPL session', making the tool's function specific and unambiguous. This contrasts with sibling tools like repl_key or repl_expect, which handle different REPL interactions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives (e.g., repl_key for key presses, repl_expect for pattern matching). It only implies use for sending text, but lacks exclusions or contextual boundaries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• uiB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description reveals some behavioral traits, such as platform-specific actions ('fuzzy tap (Android)' and 'tap by text (Desktop)'), and lists various action types. However, it lacks details on caching, destructive operations, permissions, or return behavior, which are important given no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, listing actions with one-line explanations. It is well-structured and front-loaded with the general purpose, then each action. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (20 parameters, multiple actions), the description is too brief. It omits important context like when to use each action, return value structure, and interaction with sibling tools. No output schema exacerbates the gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 95% schema description coverage, the schema already provides thorough parameter explanations. The description adds no additional meaning beyond what is in the schema, so it meets the baseline expectation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'UI inspection and interaction' and lists specific actions (tree, find, find_tap, etc.), making the tool's purpose evident. However, it does not explicitly differentiate from sibling tools like 'input' or 'screen', which could cause confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It simply enumerates actions without context on appropriate scenarios, prerequisites, or comparisons to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• repl_snapshotB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description mentions that output is redacted for common secret patterns unless disabled, which is a useful behavioral trait not covered by annotations (none provided). However, it does not disclose other important aspects like whether the snapshot is destructive, performance implications, or if it requires special permissions. With no annotations, the description carries the full burden and provides only partial transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, both earning their place: the first states the core purpose, the second adds a critical behavioral detail. It is efficient and front-loaded with the primary action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 params, no output schema, no annotations), the description provides the basic purpose and a key behavioral note. However, it lacks details on return format, error conditions, or how to handle the output, leaving gaps for an agent. It is minimally complete but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%, with 'tail' having a description ('Trailing lines to return') in the schema. The description adds no additional meaning beyond the schema for either parameter. The 'id' parameter lacks schema description, but the tool description implies it's a session identifier, making it interpretable. The baseline is 3 due to partial coverage and no extra value added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads the current emulated terminal screen for a session, using a specific verb ('Read') and resource ('terminal screen for a session'). It distinguishes itself from siblings like 'repl_send' and 'repl_key' by focusing on snapshot reading, though it could differentiate more explicitly from 'repl_list' which lists sessions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool over alternatives. There is no mention of prerequisites, such as needing a session from 'repl_spawn', or when to prefer 'repl_expect' for waiting. The description lacks context for proper selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• screenB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions compression/diff for capture and bounding boxes for annotate, but does not elaborate on side effects, permissions, or behavior like diff comparison (only in schema). It adds some context but insufficient for a tool with 12 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short (two sentences) and front-loaded. It is concise but perhaps overly terse given the tool's complexity. It could be slightly more structured without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 12 parameters, no output schema, and no annotations, the description is incomplete. It does not explain return values, side effects, or usage context beyond the basic action descriptions. It relies heavily on the schema, which is insufficient for comprehensive understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 92%, so the schema already documents most parameters well. The description adds no extra meaning beyond the schema. Baseline is 3, and no additional value from the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is for screen capture and annotation, and differentiates the two actions: 'capture' for screenshot with compression/diff, and 'annotate' for screenshot with element bounding boxes. It effectively distinguishes itself from sibling tools like 'app' or 'device'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention when to choose 'capture' over 'annotate' nor does it compare to sibling tools. The usage context is only implied by the action names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• repl_listB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool lists sessions and their statuses, but does not disclose what 'statuses' means, whether it includes session IDs, or any side effects (e.g., whether it is read-only). This is a simple listing operation, but without annotations, the agent cannot infer safety or data details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that is front-loaded with the verb 'List' and the resource. There is no waste, and it is appropriately sized for a tool with no parameters and a straightforward purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (zero parameters, no output schema), and the description is brief. It does not describe the output format in detail, but without an output schema, some return value information would be beneficial. However, for a simple listing tool, this is minimally complete, lacking details on what constitutes a 'status' or if any login is required.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description does not need to explain parameters. With no parameters, schema coverage is 100% by default, and the description adds the semantics of what is returned: 'sessions and their statuses,' which is useful. This is a baseline 4 for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'List active REPL sessions and their statuses' clearly states the action (list) and the resource (active REPL sessions), adding 'and their statuses' to specify the scope. It distinguishes itself from sibling tools like repl_spawn or repl_kill, which perform actions, and from repl_list alone, though it does not explicitly differentiate from a potential sibling that lists all sessions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need to see active sessions, but it does not provide explicit when-to-use or when-not-to-use guidance. There are no alternatives mentioned, but given the sibling set, it is clear this is for listing, not managing sessions. It is minimally adequate but lacks explicit context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• repl_keyA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only states the action itself and gives no information about session prerequisites, potential side effects (e.g., ctrl-c terminating a process), or return behavior. This is a significant gap for a tool that sends control keys.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence that front-loads the action and lists the allowed keys without any filler. Every word is informational, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two required string parameters, the description covers the core action and allowed inputs. However, the lack of annotations, output schema, and any mention of session state, error conditions, or effects leaves noticeable gaps, making it only minimally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero descriptions (0% coverage), so the description must compensate. It adds meaning to 'key' by explaining it is a control key and listing examples, and it implies 'id' refers to a session. However, it does not explicitly define the id parameter or state that key must be limited to the enumerated values, leaving partial ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Send' and identifies both the resource ('a session') and the nature of the input ('control key (enter/ctrl-c/ctrl-d/tab/arrows)'). This clearly distinguishes it from sibling tools like repl_send, which likely sends arbitrary text, by specifying the exact type of key input.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by enumerating the supported control keys, indicating this is for sending special keys to a session. However, it does not explicitly name alternatives or state when not to use this tool, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• repl_killA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the burden of behavioral disclosure. It adds meaningful detail beyond the name by specifying the signal sequence 'SIGTERM, then SIGKILL', which communicates a graceful termination attempt followed by a forced kill. It does not mention irreversibility or side effects, but the mechanism is usefully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. It delivers the core action and key behavioral detail in a compact, immediately scannable format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description is reasonably complete: it names the action and the termination mechanism. It is slightly limited by lacking parameter clarification and usage guidance, but those gaps are less critical given the minimal interface.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and the description does not explain the 'id' parameter at all, even though it is required. The connection between 'id' and the REPL session being terminated is implicitly understandable from context, but the description adds no explicit semantic value beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Terminate') and clearly identifies the resource ('REPL session'), making the tool's action unambiguous. It also distinguishes itself from sibling REPL tools like repl_spawn, repl_send, and repl_list by focusing on termination.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when a REPL session should be ended. However, it provides no explicit guidance about when not to use it, no alternatives, and no mention of prerequisites such as the session being active or listed via repl_list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flowA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavioral traits. It explains the orchestration nature and mentions turbo's rich feedback, but it does not elaborate on side effects, error handling (though schema covers on_error/if_not_found), or how the tool interacts with subordinate tools. It is not misleading, but it is minimal in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, information-dense sentence that front-loads the core concept (orchestration) and immediately lists the three modes. Every clause adds value with no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (nested steps, multiple modes, no output schema), the description covers the high-level functionality and key usage distinctions. It omits details like error handling strategies and loop specifics, but those are present in the schema. The description is sufficient for an agent to understand the tool's scope and how to start using it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 89%, so the schema already documents parameters well. The description adds only the note that deviceId is injected per device (already in schema) and turbo is experimental, which is minor. It does not significantly enhance understanding beyond the schema, hence baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Flow orchestration' and enumerates the three action types (batch, run, parallel) with specific distinctions. This goes beyond a generic verb+resource and provides immediate differentiation between the modes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use each action type (batch for multi-command, run for automation with loops/conditionals, parallel for same action on multiple devices). It also mentions the experimental turbo option and the exclusion of system_shell and browser_evaluate in steps. However, it does not explicitly state when to avoid this tool or compare it to direct alternatives, though it's implicitly an orchestration meta-tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deviceA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions "on demand" loading for modules but fails to disclose side effects, permission requirements, or what happens to the system when enabling/disabling modules. Basic transparency is present but insufficient for a tool with 8 actions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with two clear clauses, front-loading the high-level purpose. Every part earns its place, and the structure immediately communicates the tool's scope without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, and the description does not explain return values or error conditions. Given the complexity (6 parameters, 8 actions), the description is incomplete for an agent to fully understand the tool's behavior and expectations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is high (83%), with parameter descriptions already present. The description adds meaning by grouping actions and implying which parameters apply to which group (e.g., deviceId for set, platform for list/set_target, module for enable_module/disable_module). This helps contextualize the parameter usage beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool handles "Device management + module loading" and then groups the actions into two categories: "list/set/set_target/get_target: devices" and "enable_module/disable_module/list_modules: load browser/desktop/store tools on demand." This provides a specific verb-resource mapping and distinguishes from sibling tools like app, flow, or input.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use each group of actions (device operations vs. module loading) but does not explicitly state when not to use this tool or provide alternatives among siblings. It gives clear context but lacks exclusionary guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• repl_spawnA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It clearly states that cmd is exec'd directly (no shell by default) and lists what is NOT interpreted (env prefixes, redirections, pipes, globs). This gives the agent critical behavioral context about how the process is spawned. It does not mention return format beyond session id or error handling, but for a spawn tool this covers essential transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact at three sentences with the core purpose front-loaded. It efficiently communicates the through-pty nature, return value, and critical shell-vs-no-shell distinction. Minor redundancy: 'env-var prefixes' is mentioned both in the list and in the env param guidance, but this is acceptable clarity rather than waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 parameters (2 required, nested objects) and no output schema, the description covers the most critical gaps: how cmd execution works, shell mode, and environment handling. It does not describe the cwd, cols, rows, promptRegex parameters or what happens on error, but for a spawn tool with rich input structure, the focused coverage on the most complex behavior (shell vs no shell) is appropriate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 38% (3 of 8 params have descriptions in schema: id, cmd, shell; the rest like cwd, env, cols, rows, promptRegex have no schema descriptions). The description adds meaningful context beyond the schema: it explains that cmd is exec'd directly, that environment variables cannot be set via shell prefixes but should use the env param, and that shell:true routes through /bin/sh -c. This compensates significantly for the schema gaps, though it does not describe cols, rows, or promptRegex.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Start an interactive REPL or CLI process under a PTY' and that it returns a session id. It distinctly distinguishes from siblings like repl_send, repl_expect, and repl_kill by specifying that this tool spawns a new process, which is a unique action among the sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes explicit guidance on when not to use shell syntax without setting shell:true, explaining that env-var prefixes, redirections, pipes, and globs require shell:true. However, it does not explicitly mention alternatives among siblings (e.g., when to use repl_send instead for sending input).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.