Server Quality Checklist

Tool Scores

• list_commandsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description adds 'Read-only: never re-runs anything' which is consistent with annotations (readOnlyHint, idempotentHint). Provides useful behavioral context beyond what annotations declare.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. Main purpose stated upfront, followed by filter support and usage guidance. Excellent structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given parameter count (6) and absence of output schema, description covers return format, what is not included (no output text), and filter usage. Fully sufficient for agent to use tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters with descriptions. Description provides usage examples for some parameters (e.g., exit='fail'), but does not add significant new info beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'List the user's recent terminal commands' and specifies format (plain text with id, time, cwd, exit code, duration, output size). Distinguishes from siblings by indicating when to use each tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly gives guidance: 'Use this to browse history; use search_output when looking for specific text, and get_output to read what a command actually printed.' Also mentions same filters as search_output.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_outputA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behaviors beyond annotations: ANSI stripping, secret masking, truncation with gap marker, error behavior ('not found'), and read-only nature. Annotations already indicate read-only and idempotent, but the description adds specific, actionable details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the tool's purpose, followed by parameter explanations. Every sentence adds essential information, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description fully explains what is returned (output text, metadata) and covers edge cases (large output, error). Combined with complete annotations and parameter schema, it leaves no significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining the meaning of negative ids for relative addressing and the default and truncation behavior for max_bytes, exceeding what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves the full recorded output of a terminal command, including command line, exit code, cwd, and timing. It distinguishes itself from sibling tools (search_output, list_commands, diff_output) by focusing on output retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains how to obtain a command id (from search_output/list_commands or relative addressing) and how to handle large outputs (using max_bytes). It does not explicitly exclude any scenarios, but the read-only note implies safe usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_outputA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description reinforces annotations by stating read-only and never re-runs commands. Adds that secrets are masked by default, and details the return format including fields and snippet behavior. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Packs essential information into a few sentences with clear front-loading of purpose. No unnecessary words, structured logically from what to when to how.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 parameters, no output schema, and moderate complexity, the description covers purpose, usage, alternatives, return format, error behavior, and safety. Sufficient for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description does not add significant detail beyond what the schema already provides for individual parameters, though it explains the overall search result behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs full-text search over terminal commands, outputs, and notes, with a concrete example. It distinguishes itself from sibling tools by specifying when to use list_commands and get_output instead.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using this tool to find which command produced specific output, and contrasts with list_commands for browsing and get_output for reading full output. Also notes empty returns are not errors.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• diff_outputA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, idempotentHint, destructiveHint. The description adds valuable context: secrets are masked by default, the tool never re-runs anything (read-only), and identical outputs return a note. This disclosure exceeds what annotations alone provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise yet complete, structured with a clear opening statement followed by behavioral details and parameter explanations. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains the return format (unified diff, plain text) and includes a note for identical outputs. It covers all key aspects: use cases, default behavior, error conditions, and security masking, making it fully informative.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for each parameter. The description adds context for the 'id' parameter (explains -N syntax) and clarifies 'other' as the older side. While baseline is 3 due to high coverage, the extra semantics justify a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool produces a unified diff between two runs' outputs. It distinguishes from siblings like get_output (which returns full output) and search_output/list_commands (different purposes), making the tool's purpose specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance on when to use with only 'id' (diffs against previous same command) versus with 'other' (explicit pair). It also explains error conditions (no earlier identical command line) and behavior for identical outputs, fully guiding the agent's decision.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.