Server Quality Checklist

Tool Scores

• connect_sshA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses side effects (TCP socket with keepalive, leak if disconnect not called), errors (auth, unresolvable host, etc.), and behaviors like host resolution and banner capture. Adds context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with sections: overview, parameter guidance, side effects, errors, returns. Front-loaded purpose. No wasted words; each sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complete for a complex tool: covers all 6 parameters, side effects, error conditions, and return format. No gaps in information needed for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds valuable guidance: key_filepath tried first, password fallback, username defaults, banner_timeout behavior (0 skips). Adds meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it opens a persistent SSH connection and returns a session_id for use with 'execute'. It distinguishes from sibling tools like 'execute' (for local commands) and 'disconnect'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance: 'For local commands, call execute directly — no session needed.' Details parameter precedence, default behavior, banner timeout, and the lifecycle (connect, execute, disconnect).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• disconnectA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructive and idempotent behavior. The description adds detailed transparency by explaining that calling disconnect on an already-closed or unknown id is a safe no-op, that it terminates all running commands, invalidates command_ids, and closes channels and sockets. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections for lifecycle, side effects, when-not-to-use, and returns. It is front-loaded with the main purpose and every sentence adds value. Despite length, it is efficient and not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and annotations, the description covers all necessary aspects: lifecycle, side effects, alternatives, idempotency, return value, and parameter semantics. It is fully complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers the session_id parameter with a basic description. The description adds substantial meaning: it explains that the id is an opaque UUID from connect_ssh, is used with execute, and is retired by disconnect. It also clarifies that each connect_ssh produces a unique id, and reconnecting the same host yields a new one.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Close an SSH session and release all associated resources.' It distinguishes from sibling tools by explicitly noting it is not needed for local commands and that for stopping a single command, send_control should be used.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use and when-not-to-use guidance. It states that disconnect is for closing SSH sessions, not for local commands, and advises using send_control to stop a single command. It also explains the session lifecycle, making usage context clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• executeA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses statelessness, non-persistence of cd/env, and warnings about backgrounding. Annotations provide destructiveHint and readOnlyHint, and description adds context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections: purpose, parameter relationships, when not to use, side effects, returns. Every sentence adds value; no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all aspects: purpose, usage guidance, behavioral traits, parameter details, return types. No gaps given the tool's complexity and absence of output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and description adds significant extra semantics: parameter relationships (session_id vs shell, pause_timeout vs total_timeout), validation (ValueError on invalid ranges), and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource+context: 'Execute a command locally or over SSH in an isolated channel.' Distinguishes from sibling tools like respond, send_control, read_output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'WHEN NOT TO USE' section listing alternatives. Also explains parameter relationships and validation, guiding when to use local vs SSH.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_outputA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses key behaviors beyond readOnlyHint: incremental output (each call returns new output since last read), command lifecycle (retired on completion, future calls raise ValueError), and error conditions. No contradiction with readOnlyHint or openWorldHint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with sections like 'WHEN NOT TO USE', 'PARAMETER GUIDANCE', 'ERRORS', 'RETURNS'. Every sentence adds value; no filler. Concise yet comprehensive.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description fully documents return types with structure and meaning. Covers error scenarios, lifecycle, polling semantics. Complete for a polling tool of moderate complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (baseline 3), but the description adds significant value by explaining the roles of pause_timeout vs total_timeout: 'pause_timeout is the primary dial... total_timeout only caps actively streaming output.' This clarifies behavior beyond schema comments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear verb+resource: 'Poll new output from a running command without sending input.' It distinguishes itself from siblings like 'respond' and 'send_control' by specifying when to use each.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use: 'Use after execute returns status="partial" for non-interactive commands.' Also includes a 'WHEN NOT TO USE' section with direct alternatives (respond, send_control), guiding the agent clearly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• respondA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses side effects (writes to stdin, may trigger output/state change/exit), auto-appends newline, and raises ValueError for invalid/closed commands. Complements annotations (destructiveHint=true) with actionable behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with labeled sections (parameter guidance, when not to use, side effects, returns). Every sentence adds necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, description fully explains return values with status/output/exit_code structure. Covers error conditions and parameter interactions thoroughly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds value beyond 100% schema coverage by explaining auto-appending newline, recommending send_control for control codes, and clarifying pause_timeout/total_timeout usage. Each parameter gets contextual usage guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Write text to a running command's stdin' with specific verb and resource. Distinguishes from sibling send_control for control keys and execute for starting commands.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly provides 'WHEN NOT TO USE' clause (inside zellij) and guidance for parameter adjustments (pause_timeout vs total_timeout). Clearly contrasts with send_control for alternative input methods.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_controlA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (destructiveHint=true, idempotentHint=false), the description explains side effects (signal may terminate command, making command_id invalid), timeout interaction, and error handling. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections (signal handling, side effects, timeout interaction, errors, returns). Every sentence adds value, and the information is front-loaded with key usage guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters (2 required), no output schema, and the open-world/destructive annotations, the description fully covers behavior, return format, errors, and edge cases. It is complete for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema coverage, the description adds value by explaining signal case-insensitivity, common signals, and when to raise pause_timeout or total_timeout. It enriches understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool sends control keys or escape sequences to a running command, listing specific use cases (interrupts, TUI navigation, non-printable input) and explicitly distinguishes it from sibling 'respond' by noting that AI frameworks strip raw control bytes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Prefer this over respond for control keys.' It also details when to use different signals, how to adjust timeout for slow TUI repaints, and error conditions, giving clear context on when and when not to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.