Server Quality Checklist

Tool Scores

• remote_closeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that the tool closes a cached session and that the next call reopens a fresh one. It mentions idempotency implicitly but lacks detail on side effects like process termination. Still, it provides useful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences: one for action, one for context, one for usage guidance. It is front-loaded with the primary purpose and avoids unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema needed? context says output schema exists but not shown), the description adequately covers purpose, usage, and behavior. The existence of an output schema means return values need not be described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The sole parameter 'host' has 0% schema description coverage, and the tool description adds no additional meaning beyond its name. The description does not specify expected format, validation, or examples for the host value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'close' and the resource 'cached persistent bash session for <host>'. It distinguishes itself from sibling tools (local_exec, policy_check, remote_job) by focusing on session lifecycle management.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Rarely needed' and specifies the condition for use: 'to reset a session whose state has gotten dirty'. It also explains that the session is auto-managed by remote_shell, guiding when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• local_execA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It covers credentials, sudo, timeout behavior, keepalive pings, and risk flagging. However, it does not explicitly describe the return format or output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but well-structured with bullet points and sections. It front-loads the purpose. While verbose, the complexity of the tool justifies the detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters, no annotations, and existing output schema, the description covers all necessary aspects: usage, parameters, behavior, and safety. It is complete enough for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds rich semantics for all 4 parameters: command, secrets (redacted env vars), timeout (required, no default, ceiling), and use_sudo (out-of-band password). Provides clear usage details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a command on the MCP server's own machine, distinguishing it from remote_exec. The verb 'run a command' and resource 'local machine' are specific and accurate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (local commands) and when not (remote, use remote_exec). Also explains it is off by default and requires an environment variable. Provides guidance on secrets and sudo usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• policy_checkA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: dry-run, no execution, returns 'ALLOWED' or 'BLOCKED: <reason>', does not consume rate-limit tokens. It also warns about default permissive policy, which is critical for correct interpretation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections, examples, and a warning. It is moderately concise; the warning block is valuable but slightly lengthy. Still, it earns its space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers return values, default policy, and how to inspect policy state. With output schema present, it is complete for a dry-run tool, leaving no ambiguity about behavior or expected results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description compensates thoroughly: explains host as accessibility check, command as permission check, and provides examples with expected behavior, adding meaning beyond schema field names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it's a dry-run to check security policy, provides examples, and distinguishes from siblings by emphasizing non-execution. The examples for host and command make the purpose concrete.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises using before risky multi-host operations and warns about default permissive policy. It explains the two usage forms but does not explicitly exclude alternatives like local_exec for actual execution.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remote_jobA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It thoroughly discloses behavioral traits: background execution via nohup, login shell behavior, persistence across restarts, concurrent job limits, TTL cleanup, and why sudo/secrets are rejected. Also notes best-effort persistence and recovery via ps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is quite long but well-structured with sections for actions and limits. However, it includes a 'Manual fallback' section that may be extraneous for an AI agent. It is front-loaded with the main purpose but could be more concise by trimming less essential details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (11 parameters, 4 actions, persistence, limits), the description is highly complete. It covers all actions with parameter requirements, explains output schema fields implicitly, mentions persistence behavior and limits, and provides a fallback. The presence of an output schema (context indicates exists) reduces the need to describe return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It explains most parameters in context of actions: for poll, it explains since, max_bytes, tail; for cancel, signal; for submit, command, host, login. It explains the login parameter's default and ability to opt out. However, not all parameters are explicitly described (e.g., host's default is not clarified, tail default is not mentioned), but overall adds significant meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a command in the background and returns a job_id immediately. It distinguishes itself from sibling tools by noting when to use remote_exec (quick commands) versus this tool (long tasks). The four actions (submit, poll, cancel, list) are clearly enumerated with specific purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance: use for long tasks, use remote_exec for quick tasks. Also specifies what not to do: sudo/secrets are rejected and explains why, directing to remote_exec or remote_shell instead. Provides a manual fallback alternative using remote_exec.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.