Server Quality Checklist

Tool Scores

• bob_resume_sessionB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions 'auto_edit approval' as a key behavior but does not explain the consequences (e.g., potential file modifications, destructive actions, or permission requirements). This is insufficient for a tool that likely mutates state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded and free of filler. It efficiently conveys the core action and a key modifier (auto_edit approval), making it highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 7 parameters, no annotations, and no output schema, a one-sentence description is inadequate. It omits behavioral implications, return value expectations, and any guidance on the optional parameters, leaving too much uncertainty for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 29%, so the description must compensate, but it does not. It only implies the roles of session and prompt without adding detail on mode, model, stdin, workspace, or include_directories. The mention of 'previous' and 'new' adds minimal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('Resume a previous Bob Shell session') and adds the unique action of running a new prompt with auto_edit approval. This clearly distinguishes it from siblings like bob_prompt (new sessions) and bob_list_sessions (listing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when resuming a session) but does not explicitly contrast it with alternatives or state when not to use it. The use case is clear from context, but no exclusions or alternative names are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bob_list_sessionsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavioral details. It only states the basic listing action and scope, but omits any information about output format, sorting, default behavior when the optional workspace parameter is omitted, or whether it is a read-only operation. This is a significant gap for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no redundant words. It is perfectly sized for a straightforward list operation and front-loads the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one optional parameter, the description is mostly adequate. However, it does not clarify the default for 'current' when the workspace parameter is omitted, which is a gap given the schema marks it optional. Since there is no output schema, a brief mention of what is returned would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides full coverage for the single 'workspace' parameter with a clear description. The tool description adds no extra meaning beyond the schema, simply restating the workspace scope. With 100% schema coverage, baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' with a clear resource 'Bob Shell sessions' and scope 'current project workspace.' It clearly distinguishes this from the sibling tools (prompt, version, resume) by explicitly naming the action of listing sessions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context of listing sessions is clear and logically implies use before resuming or prompting. There are no explicit exclusions, but the tool's purpose is self-evident given the sibling names. No alternatives are named, but none are needed for a listing operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bob_promptA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses important behavioral traits: non-interactive execution and auto_edit approval (edits auto-approved, shell still confirmed). With no annotations provided, this fills a critical transparency gap about side effects. However, it does not cover other potential behaviors like output handling or error conditions, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the operative verb, mode (non-interactive), and key approval behavior. Every word contributes value; there is no fluff or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that runs a shell with potential file modifications, the description is somewhat terse. It conveys the approval mode but omits what the output looks like, whether it streams, or how errors surface. While the schema covers parameter details, the description does not fully contextualize the execution behavior beyond the approval policy.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides full documentation for all 6 parameters (100% coverage), including descriptions for mode, model, stdin, prompt, workspace, and include_directories. The description does not add parameter-specific meaning beyond the schema, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the primary action: running IBM Bob Shell non-interactively with a specific approval setting. The verb 'Run' and resource 'IBM Bob Shell' are explicit, and this distinguishes it from sibling tools like bob_version, bob_list_sessions, and bob_resume_session.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives. It does not mention sibling tools or suggest scenarios (e.g., 'use this to execute a task, bob_list_sessions to view existing sessions'). The usage context is wholly implied by the tool name and purpose, not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bob_versionA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. 'Return' explicitly indicates a read-only operation, and 'health check' implies non-destructive intent. Although it doesn't detail edge cases (e.g., if the shell isn't installed), for a simple version query this level of transparency is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that includes the key action and purpose ('health check') without any filler. It is optimally concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description is fully complete. It states what the tool returns and its intended use, leaving no critical gaps for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty (zero parameters), so the baseline is 4. There are no parameter semantics to explain, and the description adds no unnecessary detail about parameters, which is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') with a clear resource ('installed IBM Bob Shell version'), making the tool's function unambiguous. It also distinguishes this from sibling tools (bob_prompt, bob_list_sessions, bob_resume_session) which focus on prompts and sessions, not version info.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'health check' immediately signals when to use this tool—i.e., for verifying the installation/status of IBM Bob Shell. It provides clear context for use, though it doesn't explicitly state when not to use it or mention alternatives. Since no alternatives are relevant for a version check, this is adequate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.