Server Quality Checklist

Tool Scores

• ContextSaveC
  2.2/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotation contradiction: description claims to save (write) but readOnlyHint=true indicates read-only. No other behavioral traits disclosed (e.g., file overwrite behavior, output). Flagged as annotation contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise with two clear sentences, though the bullet-point style is less formal. Front-loaded with main action, but includes minor implementation details (e.g., random 3 word id).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema and no description of return value or behavior. Does not explain what the tool outputs or side effects. Missing key context for a complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% coverage, but description provides guidance for 'id' (random unique id) and 'project_root_path' (empty for none). However, 'description' and 'relevant_file_globs' lack explanation. Adds marginal value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states it saves description and file contents to a text file, which implies a write operation. However, the readOnlyHint annotation contradicts this, making the actual purpose unclear to the agent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs siblings like FileWriteOrEdit or Initialize. Only includes brief instructions for parameters but no usage context or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ReadImageC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the description adds no extra behavioral context such as what the tool returns (e.g., image format, encoding). It misses the opportunity to complement the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. However, its extreme brevity sacrifices informational value for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema and parameter descriptions, the definition is incomplete. It does not specify the return value or any constraints, making it insufficient for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description does not compensate. It does not clarify what 'file_path' should be (e.g., absolute vs relative path, supported image formats). The parameter remains vague.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Read an image from the shell' clearly states the tool's action (read) and resource (image), and distinguishes it from siblings like ReadFiles which likely handles text files. However, it does not elaborate on the scope or format.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives such as ReadFiles or FileWriteOrEdit. The description lacks context about scenarios or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• InitializeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds behavioral context beyond annotations by detailing the initialization steps (setting workspace, reading files, configuring mode). It does not contradict the readOnlyHint=true annotation, as no destructive actions are mentioned. The description could mention if any session state is persisted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and uses a bulleted list format that is easy to scan. Each sentence carries clear instructions. It could be slightly more organized by grouping related parameters, but overall it is efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 8 parameters, no output schema, and annotations, the description covers main usage scenarios but lacks information about what the agent receives or any side effects. It assumes prior knowledge of the wcgw shell tools. This is adequate but could be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the schema for key parameters like any_workspace_path, initial_files_to_read, mode_name, and allowed_globs. It explains default values and usage context. However, task_id_to_resume and thread_id are not elaborated, but schema coverage is high enough.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is an initialization tool to be called at the start of the conversation before using shell tools. It specifies the resource (shell) and distinguishes from sibling tools by being a prerequisite.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidelines for different call types (first_call, user_asked_mode_change, etc.) and explains when to set parameters like mode_name and allowed_globs. However, it does not explicitly say when not to use this tool or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• FileWriteOrEditA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations specify destructiveHint: true, and the description details the editing process (full content replacement or search/replace). It explains the behavior regarding percentage threshold and formatting rules. No contradiction with annotations. The description adds context beyond annotations, such as the SEARCH/REPLACE format requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is quite long but well-structured with bullet points, headings, and an example. It is front-loaded with the main action and then details. While every part serves a purpose, it could be slightly more concise without losing necessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, destructive behavior, no output schema), the description covers the editing method, percentage calculation, and formatting rules thoroughly. It lacks information on error handling, file creation behavior (if file doesn't exist), and return values, but these are minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema descriptions cover all parameters (100% coverage), the description significantly enhances meaning by explaining how percentage_to_change is calculated, the two modes for text_or_search_replace_blocks, and providing a full example with search/replace block rules. This goes well beyond the schema's brief descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool writes or edits a file, with specific instructions on using percentage-based changes. It distinguishes itself from siblings like BashCommand (shell commands) and ReadFiles/ReadImage (reading), leaving no ambiguity about its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidelines on when to use full file content vs. search/replace blocks based on percentage_to_change (>50 vs <=50). It also includes a detailed example and rules for SEARCH/REPLACE blocks. However, it does not explicitly state when to avoid using this tool in favor of siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ReadFilesA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already set readOnlyHint=true, and the description adds valuable behavioral context: absolute paths required, tilde allowed, and optional line range extraction. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured as a clear bullet list, front-loading the main purpose. The third bullet about line numbers is necessary but adds some length; overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and a simple file reading operation, the description covers essential information: file paths input, line range option, and path conventions. No gaps for typical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by explaining the 'file_paths' parameter: absolute paths only, tilde allowed, and line range syntax. It could be more explicit about the array nature of the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Read full file content of one or more files.' This specifies the verb (read), resource (files), and scope (full content), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidelines for when to use line number extraction ('Only if the task requires line numbers understanding') and shows the exact format. However, it does not contrast with sibling tools like BashCommand or FileWriteOrEdit, which could help an agent choose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• BashCommandA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses statefulness, output truncation, single-command execution, need to wait for completion, and background command behavior. Adds context beyond annotations (destructiveHint, openWorldHint).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Bullet points make it scannable. Could be slightly trimmed, but every sentence adds value. Front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with no output schema, it explains return values (status, working directory), truncation behavior, and how to handle long-running commands. Covers all critical aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 80% of parameters with descriptions. Description adds no direct parameter details but supplements with usage guidance (e.g., not to send Ctrl-c prematurely) that aids parameter selection.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Execute a bash command' and distinguishes from sibling tools like ReadFiles and FileWriteOrEdit by explicitly forbidding their use for file operations. Also highlights statefulness.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly specifies when to use this tool (commands, background jobs) and when not to (file read/write, background with &). Provides alternatives and instructions like running pwd on errors.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.