Server Quality Checklist

Tool Scores

• control_serviceB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits itself. It does mention the requirement for sufficient privileges, which is valuable context. However, it does not describe side effects (e.g., service downtime), return values, synchronous execution, or error behavior, leaving significant gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the primary action and the resource. It is concise and free of filler, earning full marks for structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and the schema is reasonably detailed, but no output schema exists and the description does not explain return values or outcome reporting. Additionally, the 'status' action present in the enum is omitted from the description, which is a completeness gap. The description is adequate for basic use but leaves room for clarification.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%—both parameters have clear descriptions ('Exact service name', 'Action to perform'). The description adds only minimal context (Windows-specific) beyond the schema, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Start, stop, or restart') and a specific resource ('a Windows service'). It distinguishes itself from sibling tools like list_services and get_service by focusing on control operations. Though it omits the 'status' action from the enum, the core purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives. It mentions the privilege requirement but does not frame it as a selection criterion. Sibling tools like get_service or list_services could be referenced for related tasks, but no such direction is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_serviceA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. It only states that it gets detailed status, but does not disclose whether this is a safe, non-mutating operation, what the output structure looks like, or how it handles non-existent services. This is a notable gap for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no unnecessary words. It front-loads the action and resource, making it immediately clear what the tool does.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool, the description is adequate but leaves gaps. With no output schema, it fails to specify what 'detailed status' includes or what error behavior to expect. This is a moderate gap in an otherwise simple context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter, and the parameter description clarifies 'exact name (not display name)'. The main description does not add extra semantics beyond 'by name', so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get'), the resource ('detailed status of one Windows service'), and the lookup method ('by name'). It distinguishes the tool from siblings like list_services and control_service by focusing on a single service's status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need the status of a single Windows service by its exact name, which provides clear context. It does not explicitly mention exclusions or alternatives, but the clarity of the purpose makes the intended use obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_servicesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden. It discloses the return fields (Name, DisplayName, Status) and the optional filter, but does not explicitly state that the operation is read-only, nor does it mention permission requirements or error conditions. The behavior is mostly clear from the verb 'List'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, each informative and necessary: the first states the purpose and the second states return fields. No redundant or irrelevant text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with no output schema and no annotations, the description adequately covers purpose and return fields. It implies that omitting the filter lists all services, but it does not specify whether the filter applies to Name, DisplayName, or both. Overall, it is mostly complete but has minor ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description for the filter parameter already provides complete details about the wildcard format (e.g., 'Sql*'). The tool description only says 'name pattern' and repeats what the schema says, adding no new meaning. Given 100% schema coverage, the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' with a clear resource 'Windows services' and mentions an optional filter. This distinguishes it from siblings like get_service (likely single-service) and control_service (state changes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for enumerating services but does not explicitly mention alternatives or exclusions. It lacks comparisons to get_service or control_service, so guidance is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sftp_downloadA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It adds useful information about the execution environment ('in-process', 'no scp.exe, no WSL', 'headless') but does not mention side effects like file overwriting, error handling, or authentication precedence. The coverage is partial but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the action and destination, then adds a compact caveat about the in-process approach. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 9 parameters, no output schema, and no annotations, the description could offer more context on authentication modes or path handling, but the schema covers parameters. The in-process detail is valuable, yet the overall picture is only moderately complete for an agent deciding whether and how to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description's mention of 'remote host' and 'this Windows host' loosely aligns with host and localPath, but it adds no additional parameter-level meaning beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Download' with a clear resource 'a file', and specifies source and destination ('remote host' to 'this Windows host'). It also distinguishes from siblings by mentioning 'in-process (ssh2 — no scp.exe, no WSL, headless)', which sets it apart from alternative transfer methods and tools like sftp_upload.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context that this is an in-process, headless SFTP download without relying on external executables, implying it should be used when such dependencies are unavailable or undesirable. However, it does not explicitly name alternatives or state when not to use it, so it falls short of full exclusionary guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sftp_uploadA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the operation is in-process using ssh2, does not rely on scp.exe or WSL, and is headless, which is useful context beyond the raw function. However, it does not mention whether existing remote files are overwritten, how authentication is selected (despite multiple auth params), or what the tool returns upon success/failure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The first sentence states the mechanism and constraints; the second gives a concrete use case. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 9 parameters and no output schema, the description gives the core purpose and a use case, but it lacks information about auth selection (password vs key), overwrite behavior, and expected return values. While the schema covers parameter meanings, the description doesn't fully contextualize the operation for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides a description for every parameter (100% coverage), so the baseline is 3. The description does not add any parameter-specific guidance beyond the generic upload action, so it stays at baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Upload), the resource (local file to a remote host), and the protocol (SFTP). It distinguishes itself from siblings by noting in-process execution (ssh2, no scp.exe/WSL) and headless operation, and mentions a primary use case for deploying scripts/configs to Linux hosts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit use case: 'Use to deploy scripts/configs to Linux hosts.' It also hints at when this tool is appropriate by contrasting with scp.exe/WSL (in-process, headless). However, it does not name alternative sibling tools or state when not to use it, so it's not as strong as a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_powershellA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses key behaviors: hidden process (no console window), returns stdout/stderr/exit code. However, it does not mention potential side effects, permissions, or security implications, which are relevant for an execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, and every word adds value. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 params and no output schema, the description covers return values (stdout, stderr, exit code), the hidden-process behavior, and general use cases. It lacks details on error handling or execution policy, but is mostly complete for practical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all parameters (script, cwd, timeoutMs). The description adds no extra parameter-specific meaning beyond what the schema descriptions provide, thus baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Run' and resource 'PowerShell script/command on this Windows host', with a specific detail about hidden process. It distinguishes from siblings like ssh_exec/winrm_exec by specifying 'this Windows host' and from run_program by focusing on PowerShell.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context: 'Use for any Windows command, file, or system task.' This implies broad applicability on Windows, but does not explicitly state when NOT to use it or name alternative tools for other scenarios. Lacks explicit exclusion criteria but is helpful enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ssh_execA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the in-process/headless nature, auth methods, and return values (stdout, stderr, exit code). However, it does not mention potential destructive effects or output truncation behavior, though some of that is in the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no fluff: first sentence gives the core action and key differentiators, second gives usage context, third covers auth and returns. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 10 parameters and no output schema, the description covers the core purpose, usage context, auth options, and return values. It omits details like output caps and tail behavior, but those are well-covered in the schema. Overall sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema fully documents all 10 parameters. The description adds minimal extra meaning beyond mentioning auth methods (privateKeyPath or password) and return values, which is redundant with schema descriptions. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb ('Run a command'), resource ('remote host over SSH'), and distinct characteristics ('fully in-process', 'no ssh.exe, no WSL'), distinguishing it from sibling tools like winrm_exec and run_powershell.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use for Linux hosts or any OpenSSH target', giving clear context for when to use this tool. It implies Windows alternatives exist through sibling tool names but does not explicitly name them, so it's not a perfect 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• winrm_execA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the transparency burden. It discloses one important behavior ('Output captured in-process (no console window)') and notes the WinRM requirement, but it does not elaborate on authentication needs, potential side effects, or exit code handling. This is a moderate level of disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences that are tightly packed with essential info: action, method, target, key differentiator, and output behavior. Every clause earns its place, with no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 parameters, no output schema), the description covers the core essentials: what it does, how it connects, and a key behavioral detail. It lacks some operational nuance like return value format or explicit rest of the lifecycle, but it is largely complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with well-described parameters, so the baseline is 3. The description does not add parameter-specific meaning beyond what the schema already provides, such as elaborating on the command field or authentication options. It remains at baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Run a command on a remote Windows host via PowerShell Remoting' and distinguishes it from SSH-based alternatives by explicitly noting 'no SSH server or agent needed.' This makes the purpose immediately apparent and differentiates it from siblings like ssh_exec.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context on when to use the tool: when targeting Windows hosts with WinRM enabled. The explicit mention that SSH is not needed serves as an implicit alternative, but it does not explicitly state 'use this instead of ssh_exec' or address run_powershell, so it falls short of full exclusionary guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• system_infoA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. 'Return' implies a read operation, which is useful, but there is no mention of prerequisites, permissions, or edge cases. It is adequate for a simple informational tool but lacks deeper behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that directly states the tool's purpose and scope with no filler or redundant information. Perfectly concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (no parameters, no output schema, no annotations), this description fully captures what the agent needs to know to select and invoke it appropriately. Nothing important is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, making schema coverage trivially 100%. With no parameters, a baseline of 4 applies; the description does not need to add parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and clearly names the resource ('OS, CPU, memory and disk summary') and scope ('for this Windows host'). It unequivocally distinguishes this from sibling tools like list_services or run_powershell by indicating a system-level overview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context—this is for a read-only summary of the local Windows host. However, it does not explicitly name alternatives or state when not to use it, so it falls slightly short of the highest standard.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_programA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses execution mode (direct, no shell, hidden process), output capture behavior, and lack of shell quoting. This is solid transparency, though it doesn't detail timeout/default behavior or error handling for non-zero exit codes, which would add further clarity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with purpose, and every sentence contributes: purpose, usage examples, differentiation from alternative, and a key parameter hint. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description effectively covers the tool's core behavior, usage context, and output capture. Since there is no output schema, it doesn't explicitly describe the exact return format, but it does state the captured fields (stdout/stderr/exit code). Combined with the parameter-rich schema, it's sufficiently complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by emphasizing the args array is passed verbatim with no shell quoting, which reinforces the schema's 'passed verbatim' note and clarifies a common point of confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Run'), specifies the resource ('native executable'), and clearly states the output captured ('clean stdout/stderr + exit code'). It also distinguishes itself from run_powershell by noting the reliability difference, which provides clear differentiation from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool ('console programs like gh, git, docker, node, python') and contrasts with run_powershell, warning against using it for native stdout capture. This provides clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.