Server Quality Checklist

Tool Scores

• read_outputC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It states 'Read the recent output' but fails to explain if output is truncated, if 'recent' implies limits, or how max_bytes affects behavior. No side effects or safety info.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no extraneous words, but it is too brief, omitting necessary details. It is suitable in length for clarity but lacks substance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema and annotations, the description should compensate. It does not explain return values, possible output formats, or how session output is managed. Incomplete for a 2-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema coverage is 0% and the description provides no explanation of 'session_id' (what it identifies) or 'max_bytes' (its purpose and default). The description adds no value over raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Read' and identifies the resource as 'recent output of a managed CLI session'. It clearly distinguishes the tool's function from siblings like 'send_text' or 'start_session'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, such as after sending commands with 'send_text' or 'send_key'. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_keyC
  2.8/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, and the description lacks any behavioral details such as side effects, safety, or what happens when a key is sent (e.g., simulation, state changes).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The single sentence is very concise and front-loaded, but it omits necessary structure like parameter descriptions or usage context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, no annotations, and minimal parameter info, the description fails to convey essential context like return values, error conditions, or prerequisites (e.g., active session).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description only reiterates examples from the key enum and ignores session_id entirely, adding minimal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool sends a special key (enter, escape, arrows, ctrl-c) and distinguishes from siblings like send_text which sends text input.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like send_text or read_output. No context or exclusions provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• close_sessionC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must disclose behavior. It only says 'terminate' without detailing side effects, what happens to the session process, or any prerequisites for closing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (one sentence) but lacks any structure or additional context. It is minimally acceptable but not well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and low complexity, the description is overly minimal. It does not mention whether the action is irreversible or what happens to active processes inside the session.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the session_id parameter beyond its name, failing to compensate for the lack of schema explanations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Terminate' and the resource 'managed CLI session', distinguishing it from siblings like list_sessions and start_session.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use vs alternatives, but the context of closing a session is implied. No exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_textB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description only reveals the submit behavior (appending Enter) but omits other behavioral traits like effects on session state, handling of edge cases (e.g., empty text, inactive session), or required prerequisites, despite no annotations existing to cover these.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences provide the core functionality and an important option without extraneous information. The structure is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 3 parameters and no output schema, the description is minimally adequate. It covers the main action and the submit toggle but lacks details about session_id sourcing, return value, or error conditions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds meaning only for the 'submit' parameter (explaining its effect). It does not elaborate on 'text' or 'session_id' beyond their names, leaving some ambiguity for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Type text into a session') and its key option ('Set submit=true to append Enter'), differentiating it from sibling tools like send_key which handles key presses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description lacks guidance on when to use this tool versus alternatives such as send_key or start_session. No context for appropriate or inappropriate scenarios is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_sessionB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, but it only shares basic behavior: starting a PTY command and default cwd. It omits critical details like session persistence, resource consumption, blocking nature, or that sessions need to be closed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose. Every sentence adds value: purpose, default behavior, return value. No redundant or irrelevant details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite good purpose clarity, the description lacks context for a complex interaction tool. No mention of how to interact via sibling tools (send_text, read_output), lifecycle management, error handling, or environment. The return value is specified, but behavioral guidelines are missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% (no parameter descriptions), so the description must compensate. It clarifies cwd's default but says nothing about command structure, cols, or rows, leaving their meaning and constraints (e.g., terminal dimensions) unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool starts any CLI program in a managed PTY within the project, with a specific verb and resource. It distinguishes from sibling tools like close_session or list_sessions, as it initiates a session.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for starting a new CLI session but does not explicitly provide when to use or when not to use it, nor does it mention alternatives among the sibling tools. Usage context is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sessionsA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries full burden. It fails to disclose behavioral traits such as whether the operation is read-only, safe, or has any side effects. For a list operation, stating it is a read-only query would be helpful but is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded and contains no extraneous information. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description is adequate. However, it could optionally mention the return format (e.g., list of session IDs or names) to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the description does not need to add parameter meaning. Baseline 4 applies because no parameter documentation is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'List CLI sessions in this Cicall project' clearly states the verb (list), resource (CLI sessions), and scope (this Cicall project). It distinguishes the tool from siblings like start_session and close_session by focusing on listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need to view existing sessions, but it provides no explicit guidance on when to use this tool versus alternatives (e.g., if you need to interact with a session, you'd use other tools). No when-not or alternative recommendations are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.