Server Quality Checklist

Tool Scores

• ssh_disconnectA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate idempotentHint=true and not destructive. Description adds that it terminates and frees resources, and returns success even if already closed, aligning with annotations. Slight gap: doesn't detail what 'frees associated resources' means, but overall good.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise with clear sections (overview, args, returns, examples, error handling). Every sentence adds value, and the structure is front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool, the description covers purpose, usage, error handling, idempotency, and return format. Output schema exists but description adequately summarizes the return value.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, session_id, has a description in the schema ('SSH session identifier to close') which is repeated verbatim in the description. No additional semantics or constraints are provided beyond the schema, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Close an SSH connection session' and elaborates that it terminates an SSH session and frees resources. It uses specific verb-resource pairing and distinguishes from sibling tools like ssh_connect, ssh_execute, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use and when-not-to-use examples, such as 'Use when: Close SSH connection to server' and 'Don't use when: Session doesn't exist' or 'Need to continue working with the server.' Also covers error handling and idempotency.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ssh_statusA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. Description adds error handling details like 'Returns No active sessions if no connections are established', which is useful context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise with clear sections (Args, Returns, Examples, Error Handling). Every sentence is informative. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given good annotations and sibling context, description provides necessary info. Could add more detail on the structure of the returned JSON, but error handling examples are helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% due to nested $ref, but description explains the params parameter containing session_id (optional), adding meaning that schema's definition alone might not capture.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Check the status of SSH connection sessions' with specific verb and resource. Distinguishes from sibling ssh_execute by saying not to use when commands needed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use examples ('Check all active SSH connections', 'Verify if connection to server is still active') and a when-not-to-use example with alternative (use ssh_execute instead).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ssh_download_fileA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds error handling details (session not found, file exists, permissions) and overwrite behavior, enriching transparency. However, the idempotentHint is not fully reinforced by the description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections (Args, Returns, Examples, Error Handling), front-loaded with the main purpose, and every sentence adds value without verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description covers the main use case, error conditions, return value, and even provides multiple examples. Output schema existence is noted, and the description complements it with return format details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is high (each parameter has a description). The description adds examples (e.g., '/var/log/syslog') and context for parameters, going beyond the schema. Baseline 3, plus extra value gives 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Download a file from the remote server to the local machine via SFTP', which is a specific verb+resource. It differentiates from siblings like ssh_upload_file (upload) and ssh_list_files (list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes explicit 'Use when' and 'Don't use when' examples, mentions multiple calls for multiple files, and lists error conditions. This provides strong guidance on when to use this tool vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ssh_upload_fileA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are minimal, so the description carries the full burden. It details error handling (session not found, file not exist, permissions, size) and return format, going beyond basic adverticement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with Args/Returns/Examples/Error sections, though slightly verbose. It remains focused and organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, annotations, and output schema, the description is complete with error handling and usage guidance. No gaps identified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes each parameter well. The description adds value with examples and default file_mode explanation, justifying a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Upload a local file to the remote server via SFTP', which is a specific verb and resource. It distinguishes itself from siblings like ssh_download_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use examples (e.g., uploading config.json) and explicit don't-use scenarios (e.g., when remote file exists, suggesting alternatives like ssh_execute with rm).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ssh_executeA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations include readOnlyHint=true and destructiveHint=true, and the description adds context about non-interactive shell execution and error handling (session not found, timeout). No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with sections (Args, Returns, Examples, Error Handling). Some redundancy could be trimmed, but overall efficient for the information provided.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the nested parameter structure and presence of an output schema, the description covers purpose, parameters, usage guidelines, error handling, and output format comprehensively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema description coverage is 0%, the tool description clearly explains each parameter (session_id, command, working_directory, timeout, response_format) with examples and constraints (e.g., timeout range 1-600).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool executes a command on a remote server via SSH, capturing stdout, stderr, and exit code. Distinguishes from sibling tools like ssh_connect and ssh_download_file by focusing on command execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides specific when-to-use examples (e.g., 'Check what Linux distribution is running') and explicit when-not-to-use cases (interactive commands, sequential commands). Also notes the prerequisite of a session_id from ssh_connect.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ssh_list_filesA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, but the description adds behavioral context by listing error cases (invalid session, missing directory, permissions) and mentioning return of file metadata. This goes beyond annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections for args, returns, examples, and error handling, but it is slightly verbose. Every sentence adds value, but a minor trim could improve conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (SSH session, multiple parameters, output schema exists), the description is complete. It covers all necessary aspects: purpose, parameters, examples, error handling, and output format. The presence of an output schema further reduces the need to explain return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the input schema by explaining session_id as 'from ssh_connect', providing examples for remote_path, and clarifying show_hidden default. Although schema coverage is technically low, the description compensates with detailed parameter explanations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists files and directories in a remote directory via SFTP, specifying the verb 'list' and the resource 'remote directory'. It distinguishes itself from siblings like ssh_download_file and ssh_execute by focusing on listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit examples of when to use the tool (e.g., 'List files in /var/log') and explicitly states when not to use it ('Don't use when: Need to list files on local machine (use local filesystem instead)'). This provides clear guidance on appropriate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ssh_connectA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses session creation, return value (session ID or error), and error conditions (auth failure, unreachable host). Complements annotations (readOnlyHint=false, destructiveHint=false) with additional context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with sections for purpose, args, returns, examples, and error handling. Every sentence is informative, and the description is appropriately concise for a complex tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all necessary aspects: purpose, parameters, return values, error handling, and usage boundaries. References sibling tools and provides complete guidance for an SSH connection tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The 'Args' section fully explains each parameter, including defaults, constraints, and relationships (e.g., password as alternative to private_key_path). Adds value beyond schema descriptions with examples and ranges.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Establish an SSH connection to a remote server' and explains it creates a session for subsequent operations, distinguishing it from sibling tools like ssh_disconnect or ssh_execute.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'Use when' and 'Don't use when' examples are provided, including alternatives like ssh_status to check existing sessions and guidance for multiple servers.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.