Server Quality Checklist

Tool Scores

• EditA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description covers key behavioral traits: edit will fail if old_string not unique, errors without prior read, and replace_all behavior. It could enhance by explicitly stating it's a mutation (destructive) action, but overall transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose and provides detailed bullet points. While slightly lengthy, every sentence adds value. Could be tightened without losing clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description sufficiently explains behavior and constraints (reading before editing, uniqueness failure). It addresses common pitfalls, making it complete for a string replacement tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage, but the description adds valuable context: uniqueness constraint for old_string, difference from replace_all, and emphasis on exact matching (indentation). This goes beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Performs exact string replacements in files.' with a specific verb ('replace') and resource ('files'). It distinguishes from siblings: Grep (search), Read (read), Write (write).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: must read file before editing, preserve exact indentation, prefer editing existing files, avoid emojis, uniqueness requirement for old_string, and when to use replace_all. It also tells when not to use (e.g., avoiding unnecessary new files).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• WriteA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It discloses overwrite behavior, the read-first requirement (and that failure occurs otherwise), and preference for Edit. Could mention directory creation or error handling, but covers key behavioral traits well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One clear introductory sentence followed by bullet points with zero redundancy. Every sentence contributes meaningful guidance. Front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, but no return value explanation needed. Description covers prerequisites (read first), alternatives (Edit), and prohibitions (markdown/emojis). Complete for a file-write tool of moderate complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters with descriptions. The tool description adds no additional meaning beyond the schema (e.g., no format constraints or examples). Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'Writes a file to the local filesystem,' a clear verb+resource statement. It distinguishes from siblings by specifying when to use Write vs Edit (modifications) and Read (must read first for existing files).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (create new files or complete rewrites), when to avoid (prefer Edit for modifications), and prerequisites (must read file first if existing). Also includes prohibitions on markdown and emojis unless requested.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ReadA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It comprehensively discloses behaviors: default 2000 lines, 256KB limit, cat -n output, multimodal for images, PDF pagination rules, notebook handling, directory exclusion, and empty file warning.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with a clear opening and bullet points. Every sentence serves a purpose, but the description is somewhat lengthy. Front-loaded with the main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and no output schema, the description is remarkably complete. It covers all parameters, edge cases (empty file, directory, screenshots), and special file types (images, PDF, notebooks). No gaps identified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds significant value beyond schema: file_path must be absolute, limit only for large files, pages only for PDF, offset as line number. It also provides context like defaults and error conditions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it reads files from the local filesystem, with specific verb and resource. It distinguishes from sibling tools (Edit, Grep, Write) by focusing solely on reading. The title and description are aligned.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Extensive guidelines are provided: absolute path requirement, default line limit, file size handling, PDF and notebook specifics, and directory exclusion. However, it does not explicitly state when to use this tool over alternatives, though the sibling names imply differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• GrepA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but the description fully discloses behavior: result limits (250), pagination via head_limit and offset, encoding handling, default modes, and line numbers. It implicitly indicates read-only nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections and bullet points. It is concise yet covers all necessary usage details without repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description covers essential aspects: usage, parameters, defaults, encoding, and pagination. It is sufficient for an agent to invoke the tool correctly, though it could mention that output is textual.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage, so baseline is 3. The description adds value by explaining the purpose of output modes, the default for line numbers, and the effect of head_limit=0. This extra guidance justifies a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a search tool for file contents using regular expressions, and explicitly differentiates from sibling tools (Edit, Read, Write) by stating it should be used for search tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage instructions: 'ALWAYS use Grep for search tasks. NEVER invoke grep or rg as a Bash command.' Also explains when to use different output modes (content, files_with_matches, count) and pagination.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.