Server Quality Checklist

Tool Scores

• ssh_run_commandB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must cover behavioral traits. It only mentions the return values but fails to disclose important aspects like potential destructiveness, authentication requirements, or side effects of executing remote commands.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of two short sentences that are front-loaded with the core action and output. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 8 parameters and no output schema, the description is too minimal. It does not explain authentication methods, error handling, or connection behavior, leaving significant gaps for an SSH tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are already well-documented. The description adds no additional meaning beyond what the schema provides, meeting the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool SSHs into a remote host and runs an arbitrary shell command, and specifies the output (stdout, stderr, exit code). This distinguishes it from siblings like ssh_git_pull and ssh_run_script, which have more specific purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings (e.g., for one-off commands vs. running scripts). It lacks explicit context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ssh_git_pullB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions authentication methods but fails to disclose critical behaviors such as potential merge conflicts, the need for git to be installed on the remote, or any output/error handling. This lack of transparency could lead to misuse.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences, the first defining the core action and the second adding authentication support. It is front-loaded and avoids unnecessary detail, though it could be slightly more structured (e.g., bullet points).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 9 parameters, no output schema, and no annotations, the description covers the main action and authentication but omits important context like error scenarios, prerequisites (git on remote), and behavior of optional parameters. Adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds useful context on authentication options (private key vs password) but does not elaborate on other parameters like branch, port, or passphrase beyond what the schema already provides. No additional semantic value for those.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action: SSH into a remote host and run git pull in a specified repo directory. The name and description together precisely define the tool's purpose, distinguishing it from more general SSH command execution tools like ssh_run_command.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for updating a git repository via SSH but provides no explicit guidance on when to use this tool versus siblings (e.g., for other commands use ssh_run_command). No exclusions or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_ssh_configA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It covers replacement of existing entries but misses important traits: what happens if the file doesn't exist, permissions required, backup behavior, or error handling. For a mutation tool, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff. The first sentence defines the core action, and the second provides context. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 parameters including a nested object and no output schema, the description covers basic behavior but lacks details on file creation, error states, and default port behavior. It is adequate for simple use cases but incomplete for complex ones.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description adds no extra meaning beyond the schema, making it adequate but not additive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'Write (or update)' and resource 'Host entry in local ~/.ssh/config file', and distinguishes itself from sibling tools (ssh_git_pull, ssh_run_command, ssh_run_script) which focus on running commands rather than configuring SSH.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Useful for setting up SSH config before running deployment commands' implies when to use, but does not explicitly state when not to use or provide alternatives. No prerequisites or error conditions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ssh_run_scriptA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the method (piping to bash) and lack of local temp files, but omits details on output handling, error behavior, or authentication requirements beyond the parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action. No superfluous information; every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 9 parameters and no output schema, the description is adequate but not complete. It covers the core behavior but lacks details on return values, error handling, or authentication context, which could aid an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear descriptions for all 9 parameters. The description adds no extra meaning beyond the schema, meeting the baseline without improvement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (SSH and execute) and the resource (multi-line shell script). The mention of piping to bash and no temporary files distinguishes it from sibling tools like ssh_run_command, which likely runs single commands.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for multi-line scripts but does not explicitly state when to use this tool over siblings. No 'when not to use' or alternative tool names are given, leaving the agent to infer from the context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.