Server Quality Checklist

Tool Scores

• create_workspaceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool creates a workspace, implying a mutation operation, but doesn't describe what happens after creation (e.g., whether it's immediately active, if it requires additional setup, or if there are limits on workspace count). It mentions the workspace is 'isolated', which adds some context about separation, but lacks details on permissions, error conditions, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently conveys the core purpose without unnecessary words. It is front-loaded with the key action and resource, making it easy to scan and understand. Every part of the sentence earns its place by specifying what is created and its intended use.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity as a creation operation with no annotations and no output schema, the description is insufficient. It doesn't explain what the tool returns (e.g., success confirmation, workspace details), potential errors, or how the created workspace integrates with sibling tools like 'execute_command'. For a mutation tool in this context, more completeness is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'workspace_id' documented as a 'Unique workspace identifier'. The description doesn't add any additional meaning beyond this, such as format constraints or examples. Since schema coverage is high, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't need to given the schema's completeness.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create') and resource ('new isolated workspace for commands'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'list_workspaces', but the verb 'Create' versus 'List' provides inherent distinction. The description avoids tautology by specifying what is being created and its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_workspaces' or 'destroy_workspace'. It mentions the workspace is 'for commands', which hints at a context involving command execution, but doesn't specify prerequisites, dependencies, or typical workflows. No explicit when/when-not instructions or alternative tool references are included.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_commandC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions execution 'in the background' and that it 'returns the output', but doesn't address critical aspects like security implications, permission requirements, timeout behavior, error handling, or whether commands persist across sessions. For a potentially dangerous shell execution tool, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core functionality without waste. It's appropriately front-loaded with the main action and includes the key behavioral aspect (background execution) and outcome (return output). Every word serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a shell execution tool with no annotations and no output schema, the description is inadequate. It doesn't explain what 'background exec window' means, doesn't address security concerns, doesn't describe output format or error conditions, and provides no guidance on appropriate usage. Given the potential danger of shell commands and the lack of structured safety information, this leaves significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description doesn't add any meaningful parameter semantics beyond what's in the schema - it mentions 'shell command' which aligns with the schema's 'command' description, but provides no additional context about command syntax, restrictions, or workspace implications.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('execute a shell command') and resource ('background exec window'), and specifies the outcome ('return the output'). It doesn't explicitly differentiate from siblings like 'start_process' or 'send_input', but the focus on shell commands and background execution provides reasonable distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'start_process' or 'send_input'. The description mentions the 'background exec window' but doesn't explain what that means or when it's appropriate versus other execution methods. No prerequisites or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_inputC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool sends input but doesn't cover critical aspects like whether this is a read-only or destructive operation, potential side effects (e.g., if input triggers process termination), authentication needs, rate limits, or what happens if the process isn't running. This is a significant gap for a tool that interacts with processes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It is front-loaded with the core action and target, making it easy to parse quickly. Every part of the sentence contributes essential information, earning its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of interacting with processes (implied by sibling tools like 'start_process' and 'stop_process'), no annotations, no output schema, and incomplete parameter coverage, the description is inadequate. It doesn't explain what 'send input' entails (e.g., simulating keystrokes, piping data), potential errors, or return values, leaving the agent with insufficient context for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67% (2 out of 3 parameters have descriptions), with 'text' and 'target_window' documented but 'workspace_id' lacking a description. The description adds no additional parameter semantics beyond what the schema provides, such as explaining the purpose of 'workspace_id' or clarifying 'target_window' options ('ui' vs 'exec'). Baseline 3 is appropriate as the schema does most of the work but leaves one parameter unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Send input') and target ('to a running process in a specified window'), which is specific and actionable. However, it doesn't explicitly differentiate from sibling tools like 'execute_command' or 'get_output', which might involve similar process interactions, leaving some ambiguity about when this specific tool is the right choice.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'execute_command' (which might start a process) or 'get_output' (which retrieves output). It mentions a 'running process' but doesn't specify prerequisites (e.g., must have a process started via 'start_process') or exclusions, leaving the agent to infer context from sibling names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stop_processB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses the action ('sends Ctrl+C') which suggests an interrupt signal, but doesn't cover critical behavioral aspects like whether this requires specific permissions, what happens if no process is running, if the action is reversible, or any error conditions. The disclosure is minimal for a potentially destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Stop the currently running process') and adds clarifying detail ('in the exec window (sends Ctrl+C)'). Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a parameter with 0% coverage, the description is incomplete. It lacks information about the tool's behavior in edge cases, what it returns, and the parameter's purpose. For a tool that interacts with running processes, more context is needed for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, but the description doesn't mention any parameters at all. Since there's only one optional parameter ('workspace_id'), the tool likely functions without it, so the description's omission isn't critical. The baseline for 0 parameters would be 4, but here we have 1 undocumented parameter, so it's scored slightly lower.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Stop') and target ('currently running process in the exec window'), with the specific mechanism 'sends Ctrl+C' providing additional clarity. It doesn't explicitly differentiate from sibling tools like 'destroy_workspace' or 'execute_command', but the scope is well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a process is running, but provides no explicit guidance on when to use this tool versus alternatives like 'destroy_workspace' (which might terminate a workspace) or 'send_input' (which could send other signals). No prerequisites or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• destroy_workspaceB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the destructive nature ('Destroy') and scope ('all its processes'), which is critical, but lacks details on permissions needed, confirmation steps, error handling, or what happens to associated data. For a high-impact mutation tool, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with zero wasted words. It immediately conveys the core action and scope without redundancy or unnecessary elaboration, making it highly efficient for an AI agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is incomplete. It doesn't cover critical aspects like return values, error conditions, side effects on related resources, or safety warnings. Given the high-stakes nature of workspace destruction, more context is needed for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'workspace_id' documented as 'Workspace to destroy'. The description adds no additional parameter semantics beyond what the schema provides, such as format examples or validation rules, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Destroy') and resource ('a workspace and all its processes'), distinguishing it from siblings like create_workspace, list_workspaces, start_process, and stop_process. It precisely communicates the tool's destructive scope beyond just the workspace itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like stop_process or delete operations. The description implies it's for permanent removal but doesn't specify prerequisites, irreversible consequences, or contextual triggers for choosing destruction over other actions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_workspacesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states it lists 'all active workspaces,' which implies a read-only operation, but doesn't disclose behavioral traits like whether it requires authentication, has rate limits, returns paginated results, or what 'active' means. For a tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action and resource, making it easy to parse quickly. Every word earns its place by conveying essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 params, no output schema) and lack of annotations, the description is minimally adequate but has clear gaps. It states what the tool does but doesn't cover behavioral aspects like return format or usage context. For a simple list tool, it meets basic needs but could be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add param info, but that's appropriate here. Baseline is 4 for 0 params, as the schema fully handles the lack of parameters without requiring description compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all active workspaces'), making the purpose immediately understandable. However, it doesn't differentiate from potential sibling tools like 'get_output' or 'execute_command' that might also retrieve workspace information, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'create_workspace' and 'destroy_workspace' available, there's no indication of whether this should be used for discovery before creation/deletion or as a standalone query. No explicit when/when-not instructions are present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_processB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that processes can be long-running or interactive and run in different windows, but lacks critical details: whether this requires specific permissions, if processes persist across sessions, error handling, resource limits, or what happens on failure. For a process-starting tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, and the second adds crucial context about window targeting. Every sentence earns its place with no wasted words, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of starting processes (which can involve permissions, resource management, and error handling), no annotations, no output schema, and only moderate schema coverage, the description is incomplete. It covers basic usage but misses behavioral aspects like what the tool returns, how to handle errors, or prerequisites. For a tool with siblings like 'stop_process' and 'send_input', more context on integration would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67% (2 out of 3 parameters have descriptions). The description adds some value by explaining the 'target_window' parameter's purpose ('ui' for interactive apps, 'exec' for background processes), which complements the schema's example. However, it doesn't provide additional meaning for 'command' or 'workspace_id' beyond what the schema offers, and with moderate coverage, the description only partially compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Start a long-running or interactive process.' It specifies the verb ('Start') and resource ('process'), and distinguishes it from siblings like 'execute_command' by mentioning long-running/interactive nature and window targeting. However, it doesn't explicitly differentiate from 'stop_process' or 'send_input' in terms of process lifecycle management.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for long-running or interactive processes, with guidance on window selection ('ui' for interactive apps, 'exec' for background). It implies alternatives by mentioning defaults and use cases, but doesn't explicitly name when to use 'execute_command' instead or other siblings like 'stop_process' for termination.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_outputA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what gets captured (terminal output) and distinguishes between window types, but doesn't mention important behavioral aspects like whether this is a read-only operation, what format the output returns, if there are rate limits, or authentication requirements. It provides some context but leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with just two sentences that are front-loaded with the main purpose. Every word earns its place - the first sentence states the core function, and the second provides essential qualification about window options.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 parameters with 50% schema coverage and no output schema or annotations, the description provides basic context about window selection but doesn't fully compensate for the missing information. It doesn't explain what format the output returns, how to interpret it, or provide complete parameter semantics. For a tool that retrieves terminal output, more guidance on response format would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning for the 'window_name' parameter by explaining the difference between 'ui' and 'exec' windows, which complements the 50% schema description coverage. However, it doesn't mention the 'workspace_id' parameter at all, leaving half of the parameters without semantic context beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('current terminal output'), and distinguishes between UI and background windows. However, it doesn't explicitly differentiate from sibling tools like 'send_input' or 'execute_command' that might also interact with terminal output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance by explaining when to use 'exec' window versus default 'ui' window, but doesn't explicitly state when to use this tool versus alternatives like 'execute_command' for running commands or 'send_input' for sending input. No explicit exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.