Server Quality Checklist

Tool Scores

• sh_logsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses key behaviors: condensed view, cursor usage, and difference between live and durable records. However, it lacks details on potential side effects, rate limits, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (about 3 sentences) and front-loaded with the primary purpose. Every sentence adds value, but could be slightly more structured with bullet points.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains return values (condensed stdout/stderr, status, exit code, running_ms). It also covers the cursor mechanism and lifecycle behavior. Enough for an agent to understand what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context beyond the schema by explaining the purpose of cursors (to get only new output) and the distinction between live and exited processes. This enhances understanding of parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it reads output of a background run by id, specifying the resource (background run) and action (read output). It also distinguishes from sibling tools like sh_run (starts runs) and sh_kill (kills runs), though not explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on using cursors for incremental polling ('ideal for tailing a dev server'). It also explains the behavior for live vs. finished processes, but does not explicitly state when not to use the tool or suggest alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sh_checkpointA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that .git and node_modules are excluded, which is helpful. However, with no annotations, the description could mention idempotency, storage location, or overwrite behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Sufficient for a simple tool with few parameters and no output schema, though could mention overwrite behavior or error handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters with descriptions; the tool description adds minimal extra semantic info beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it snapshots a directory under a label for later restoration, using specific verb 'Snapshot' and distinguishing from sibling sh_restore.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to take a checkpoint before a risky or irreversible change, and mentions restoration via sh_restore, providing clear context for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sh_detailA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does disclose that the tool does not re-run the command. This is a key behavioral trait. However, it lacks details on side effects, authentication requirements, rate limits, or error behaviors, which would be helpful for an agent to fully understand the tool's operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences that front-load the primary purpose and then provide usage context. Every word is necessary, and there is no redundancy or filler. It is well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has three well-documented parameters and no output schema, the description covers the essential functionality and usage scenario. It explains when to use the tool and what it returns at a high level. However, it does not elaborate on the return format or potential edge cases, which would raise it to a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Since schema description coverage is 100%, the baseline is 3. The description adds no additional information about the parameters beyond what the schema already provides, such as the meaning of the 'id' or how to use 'match' and 'selector'. The schema alone is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves full stored output for a previous sh_run by ID without re-running. It distinguishes itself from siblings like sh_run by emphasizing the 'without re-running' aspect and the 'addressable output store', making the purpose specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to use this tool after a condensed result has hidden needed lines, providing clear context for when it is appropriate. However, it does not mention alternatives or explicitly state when not to use it, leaving some room for ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sh_killA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description fully bears the load. Discloses key behavioral traits: whole process group signaling, SIGTERM→SIGKILL escalation after 2s, idempotency for already-exited runs, and specific return status.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise yet information-dense. Every sentence adds value: purpose, signaling behavior, return status, and idempotency. Front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers major aspects: purpose, parameters, behavior, return status, and edge case (already-exited). Lacks detail on error handling for invalid ids, but overall sufficient for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline 3. Description adds value by explaining id provenance (from sh_run) and signal behavior (escalation), which goes beyond schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (stop) and resource (background run), and differentiates it from siblings like sh_logs and sh_detail by specifying it kills a run started with sh_run background:true.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context on when to use (to stop a background run), behavior like signaling process group and signal escalation, and mentions polling sh_logs for confirmation. Lacks explicit alternatives or when-not-to-use instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.