Server Quality Checklist

Tool Scores

• close_connectionA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool terminates a session and removes it from global state, indicating a destructive operation. However, it does not mention potential side effects (e.g., if commands are interrupted), permissions required, or error handling, leaving gaps in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that is front-loaded with the core action ('Terminate an SSH session') and includes necessary detail ('remove it from global state'). Every word earns its place, with no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (destructive operation with no annotations and no output schema), the description is incomplete. It lacks details on parameters, return values, error conditions, and behavioral nuances, making it inadequate for safe and effective use by an AI agent without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, and the tool description does not explain the 'connection_id' parameter at all. It fails to add meaning beyond the schema, such as what a connection_id is, how to obtain it, or format examples, which is insufficient given the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Terminate an SSH session') and the resource affected ('remove it from global state'), distinguishing it from siblings like create_connection (creates) and get_connections (reads). It uses precise verbs that convey the tool's exact function without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when an SSH session needs to be ended and cleaned up, but it does not explicitly state when to use this tool versus alternatives (e.g., when to close vs. keep a connection open) or provide exclusions. The context is clear but lacks explicit guidance on tool selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_commandA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the tool executes commands and returns output/exit codes, but lacks critical behavioral details: whether commands run with specific permissions, timeouts, side effects (e.g., file modifications), error handling, or security implications. For a shell execution tool, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action and outcome. Every word earns its place, with no redundancy or unnecessary elaboration, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a tool that executes shell commands (a potentially complex operation), the description is minimally adequate. It covers the basic purpose and output format but lacks details on behavioral traits, error cases, or security considerations, leaving gaps for an AI agent to use it safely and effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (only 'command' has a description). The description adds no explicit parameter semantics beyond what the schema provides. However, with only 2 parameters and one documented in schema, the baseline is high. The description's context ('in an existing SSH session') implicitly clarifies 'connection_id' as an active session identifier, adding some value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Run a shell command'), resource ('in an existing SSH session'), and outcome ('return stdout/stderr/exitCode'). It distinguishes from siblings like create_connection (establishes session) and close_connection (terminates session) by focusing on command execution within an existing session.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('in an existing SSH session'), suggesting this tool should be used after establishing a connection. However, it doesn't explicitly state when to use alternatives like secure_execute_command or provide prerequisites (e.g., connection must be active). The guidance is present but incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_connectionA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses key behavioral traits: it opens an SSH session (implies network/authentication needs) and tracks it globally (stateful, reusable). However, it lacks details on error handling, timeouts, permissions, or what 'global state' entails (e.g., persistence across calls).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  It's a single, well-structured sentence that front-loads the core action and adds necessary context efficiently. Every word earns its place with zero waste, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is adequate for a setup tool but has gaps. It covers the purpose and state tracking well, but lacks details on return values (e.g., success/failure indicators), error cases, or prerequisites (e.g., valid credentials). It's minimal but not fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters fully. The description adds no additional meaning beyond implying 'machine_id' comes from 'get_available_connections' (which is in the schema) and 'title' is for display. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Open an SSH session') and resource ('to the given machine'), plus the additional effect of tracking it in global state for reuse. It distinguishes from siblings like 'get_available_connections' (list) and 'execute_command' (use).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It implies usage context by mentioning 'subsequent tool calls can reuse it,' suggesting this is a setup step before commands. However, it doesn't explicitly state when not to use it (e.g., for one-off commands) or name alternatives like 'secure_execute_command' for direct execution.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_connectionsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool returns data (not modifies) and specifies the scope ('global state'), but lacks details on permissions, rate limits, response format, or error handling. For a read operation with zero annotation coverage, this is a minimal but adequate disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the key information ('Return every STILL-OPEN SSH session') with no wasted words. It's appropriately sized for a simple tool with no parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is complete enough for basic understanding but lacks output details (e.g., format, structure) and behavioral context like error cases. It meets minimum viability but has clear gaps in a no-annotation scenario.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so no parameter documentation is needed. The description doesn't add parameter details beyond the schema, but since there are no parameters, a baseline of 4 is appropriate as it doesn't need to compensate for any gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Return') and resource ('every STILL-OPEN SSH session in global state'), with the 'STILL-OPEN' qualifier distinguishing it from sibling tools like 'get_available_connections' which might return different connection states. It precisely communicates what the tool does without being tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through the 'STILL-OPEN' qualifier, suggesting this tool is for monitoring active sessions rather than historical or all connections. However, it doesn't explicitly state when to use this versus alternatives like 'get_available_connections' or provide clear exclusions, leaving some ambiguity about sibling tool differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_available_connectionsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It clearly indicates this is a read-only operation ('List') that doesn't modify state, which is adequate for safety disclosure. However, it doesn't mention potential behavioral aspects like rate limits, authentication requirements, pagination, or what 'knows about' means operationally (e.g., cached data, discovered hosts).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('List every SSH-capable machine') and immediately qualifies the scope. There is zero wasted verbiage, and every word earns its place by adding specificity or exclusion criteria.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is nearly complete: it states the action, resource, and key constraint ('NOT yet connected'). It loses one point because it doesn't hint at the return format (e.g., list of hostnames/IPs) or potential empty results, which would help an agent interpret outputs despite no schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage (empty schema). The description doesn't need to explain any parameters, which is appropriate. It gets a 4 (not 5) because while it correctly omits parameter details, it doesn't explicitly state 'no parameters required' or similar, though the context is clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List') and resource ('every SSH-capable machine this server knows about'), with explicit scope qualification ('but is NOT yet connected'). It effectively distinguishes from sibling tools like 'get_connections' (which likely lists active connections) and 'create_connection' (which establishes connections).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool: to list SSH-capable machines that are known but not currently connected. It implicitly distinguishes from 'get_connections' (for active connections) and 'create_connection' (to establish connections). However, it doesn't explicitly state when NOT to use it or name alternatives, keeping it at a 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• secure_execute_commandA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively communicates key traits: the command is read-only (non-mutating), executes in an SSH session, and returns specific outputs (stdout, stderr, exitCode). It does not cover aspects like error handling, timeouts, or security constraints, but provides a solid foundation for safe usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads key information ('Run a read-only shell command') and efficiently covers execution context, behavior, and outputs. Every word earns its place, with no redundancy or fluff, making it highly concise and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is largely complete. It covers purpose, constraints, and outputs adequately. However, it lacks details on error cases (e.g., invalid connection_id) and does not reference the sibling tools for context, leaving minor gaps in full contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (only the 'command' parameter has a description). The description adds value by clarifying that 'command' must be read-only (e.g., 'ls, cat'), which is not in the schema. It also implies the purpose of 'connection_id' (for an existing SSH session), partially compensating for the lack of schema description for that parameter. However, it does not fully detail parameter formats or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the verb ('Run'), resource ('shell command'), and key constraint ('read-only' and 'does not mutate state'), with clear distinction from the sibling 'execute_command' (which presumably allows mutations). It specifies the execution context ('existing SSH session') and what is returned ('stdout/stderr/exitCode'), making the purpose highly specific and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for read-only commands in an SSH session. It implies an alternative (the sibling 'execute_command' for non-read-only commands) but does not explicitly name it or detail exclusions. No guidance is given on prerequisites like needing an active connection, but the context is sufficiently clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.