Server Quality Checklist

Tool Scores

• add_credentialC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states it adds a credential but doesn't cover critical aspects: whether this requires admin permissions, if it overwrites existing credentials, error handling (e.g., invalid file paths), or what happens on success (e.g., confirmation message). It mentions 'private key file path' but not how it's stored or used. This leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action and resource, making it easy to scan. Every part of the sentence contributes directly to the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a mutation tool for SSH credentials), lack of annotations, no output schema, and low schema coverage, the description is incomplete. It doesn't explain return values, error cases, or behavioral nuances. For a tool that likely involves security-sensitive operations, more context is needed to ensure safe and correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only mentions 'private key file path', which maps to one of the four parameters (privateKeyPath). It doesn't explain the other parameters (host, name, username), their purposes, formats, or constraints. The description adds minimal value beyond the schema, failing to address the coverage gap adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Add') and resource ('SSH credential'), specifying it's for a new credential with a private key file path. It distinguishes from siblings like list_credentials (read) and remove_credential (delete), though it doesn't explicitly mention rsync_copy or ssh_exec. The purpose is specific but could be more detailed about what 'add' entails.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing private key file), exclusions, or comparisons to siblings like ssh_exec (which might use credentials). The description implies usage for adding SSH credentials but lacks contextual advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remove_credentialC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool performs a removal operation, implying mutation, but doesn't address critical aspects like whether this is destructive, irreversible, requires specific permissions, or what happens on success/failure. This is a significant gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is appropriately sized and front-loaded, with zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation operation with no annotations, 0% schema coverage, and no output schema), the description is incomplete. It lacks details on behavior, parameters, return values, and error handling, making it inadequate for safe and effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the single parameter 'name' is undocumented in the schema. The description adds no semantic information about this parameter, such as what 'name' refers to (e.g., credential identifier), its format, or examples. It fails to compensate for the schema's lack of documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Remove') and resource ('stored SSH credential'), making the purpose immediately understandable. It doesn't explicitly distinguish from siblings like 'add_credential' or 'list_credentials', but the verb 'Remove' inherently differentiates it from those operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'add_credential' or 'list_credentials'. The description lacks context about prerequisites (e.g., needing an existing credential to remove) or exclusions, leaving usage entirely implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ssh_execC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the authentication mechanism but doesn't describe execution behavior (e.g., synchronous/asynchronous, timeout handling, error propagation, output capture, or security implications). For a potentially destructive remote execution tool, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core functionality without unnecessary words. It's appropriately sized for a tool with this complexity level and gets straight to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a remote execution tool with 4 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what happens during execution, how output is returned, error handling, security considerations, or typical use patterns. The agent would need to guess about important behavioral aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for all 4 parameters, the description doesn't add any semantic information beyond what's implied by parameter names. It mentions 'private key file path' which aligns with the privateKeyPath parameter name but doesn't explain format expectations, path resolution, or relationships between parameters like host and username.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Execute command') and resource ('over SSH using private key file path'), making the purpose immediately understandable. It doesn't distinguish from sibling tools like rsync_copy or credential management tools, but it's specific enough to understand what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like rsync_copy or when SSH execution is appropriate. It mentions the authentication method (private key file path) but doesn't explain prerequisites, error conditions, or typical use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rsync_copyC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions copying via rsync but doesn't cover critical aspects like whether it overwrites files, handles symlinks, requires specific permissions, has rate limits, or what happens on errors. This leaves significant gaps for a file operation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any fluff. It's front-loaded and every word contributes to understanding the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of file copying with rsync, no annotations, and no output schema, the description is inadequate. It lacks details on behavior, error handling, and output format, which are essential for safe and effective use of this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional meaning beyond implying the need for paths and direction, which is already clear from the schema. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Copy files/directories') and the mechanism ('via rsync'), specifying both the resource (files/directories) and the scope (between local and remote server). It doesn't explicitly differentiate from sibling tools like ssh_exec, but the focus on copying distinguishes it from credential management tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like ssh_exec for other operations, nor does it mention prerequisites such as needing stored credentials. It only states what the tool does, not when or why to choose it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_credentialsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does but doesn't describe how it behaves: no information about return format (e.g., list structure, credential details), pagination, error conditions, or authentication requirements. This is a significant gap for a tool that accesses sensitive data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse. Every word earns its place without redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of listing credentials (sensitive data access) with no annotations and no output schema, the description is incomplete. It doesn't explain what information is returned (e.g., credential IDs, types, metadata), security implications, or how to handle the results, leaving critical gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema already fully documents the input. The description appropriately doesn't add parameter details, and the baseline for 0 parameters is 4. It could be 5 if it explicitly stated 'no parameters required' but this is minor.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all stored SSH credentials'), making the purpose immediately understandable. However, it doesn't differentiate from potential sibling tools like 'get_credentials' or 'search_credentials' if they existed, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'add_credential' or 'remove_credential'. It doesn't mention prerequisites, context, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.