Server Quality Checklist

Tool Scores

• bashA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It explicitly states that state persists across calls (including directory and env vars), that there is no internet access, and that state persists across discussions. It also discloses constraints and recommendations (avoid large output, run background commands) and suggests a specific technique (sed). This is rich, transparent behavior description beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, starting with the core purpose and then presenting bullet-pointed usage tips. It has a slight redundancy: 'State is persistent across command calls and discussions with the user' repeats the initial statement about persistence. However, the overall structure is clear and front-loads the most important information without wasting words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's flexibility and only one parameter, the description covers key operational aspects: persistence, network capabilities, output handling, and background execution. It does not mention exit codes or the format of command output, which could be useful, but for a general shell tool the guidance is sufficient for an agent to use it effectively. The absence of an output schema is partially mitigated by the description's behavioral notes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers the single 'command' parameter fully, providing its type and a note about relative paths. The description adds value by offering practical guidance on how to construct commands: using sed for line inspection, avoiding excessive output, and handling long-running processes. This enriches the parameter's usage semantics beyond the schema, earning a score above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Run commands in a persistent shell,' specifying the verb (run), resource (commands in a shell), and a key behavioral trait (persistence). It distinguishes itself from the sibling tool str_replace_editor, which is for file editing, making it a precise and unambiguous purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage: it mentions the shell is persistent, warns about no internet access, advises avoiding large output, and recommends running long-lived commands in the background. Though it doesn't explicitly compare with alternatives, the context and tips implicitly guide appropriate use. It doesn't state when not to use this tool, but the guidance is practical and specific.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• str_replace_editorA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Transparency is high: no annotations are present, so the description takes the full burden. Detailed behavioral information is abundant, exposing persistence, the behavior of view when file vs. directory, the inability to create an existing file, truncation of long outputs with the marker, and the exact conditions of str_replace.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The text is long, but structured with sections and command-scoped notes. It is organized by command, then by operational notes (view, insert, str_replace) with clear field labels. The amount of detail useful is given by the complexity of the tool, and is not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output scheme, but the description provides a complete operational picture even without a return description: command list, state persistence, file vs directory behavior in view, truncation of long output, create and insert restrictions, and the str_replace disambiguation rules are all present. Well covered for a tool that accepts many parameters and custom states.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the input schema already covers 100% of the parameters, the descriptions go further with practical semantics: for str_replace, old_str refers to exact/consecutive line matching and uniqueness; new_str is optional semantics; file_text is used in create; view_range explains mixed ranges with 1-based indexing and -1 sentinel; insert_line includes his post-insertion rule.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with specific verb and resource ('custom editing tool for viewing, creating and editing files'), then lists the four commands (view, create, str_replace, insert) with their exact semantics. Its 'custom editing tool' framing distinguishes it from a generic bash/shell tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly defines what the tool does and explains each command's role. The directives include operational caveats (persistent state, directory vs. file output of view, no overwrite on create, output truncation marker) and dedicated notes on how to use str_replace (exact line match, uniqueness requirement, whitespace sensitivity). These rules are stated directly in the tool description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.