Server Quality Checklist

Tool Scores

• writeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses key behaviors: overwriting content, creating parent directories if missing, and recording a content hash that affects edit. These are important for the agent to understand side effects. Missing details like idempotency or failure handling, but overall sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: three sentences. The first sentence states the core action, the second adds directory creation behavior, and the third explains the hash recording. Every sentence adds value without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple write tool, the description covers primary behavior and important side effects. It does not mention return values, but an output schema exists (context confirms) which likely covers that. Overall, it provides enough information for the agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds basic meaning by using parameter names in context: 'Write `content` to `file_path`'. It also explains that parent directories are created for filePath. However, with 0% schema coverage, more detail on constraints (e.g., path format, content encoding) would be beneficial. The description provides minimal yet adequate semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Write `content` to `file_path`, overwriting any existing content.' It specifies the verb (write), resource (file), and that it overwrites. It also distinguishes from sibling tools by noting that the content hash is recorded, requiring a fresh read for a follow-up edit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context about when to use this tool by mentioning the interaction with edit: 'a follow-up `edit` will require a fresh `read`.' This implies write is for creating or fully overwriting files. However, it does not explicitly state when to prefer write over edit or when not to use it, which would be clearer.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• editA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses all behavioral traits: requirements to read first, uniqueness constraint, no changes since read, and return type (unified diff). This is thorough and transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a clear structure: a single sentence explaining the action, followed by bullet-pointed requirements and a return statement. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a file editing tool with 3 parameters, no annotations, but an output schema, the description covers prerequisites, behavior, and return value. It is fairly complete, though it could elaborate on edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description does not add much meaning beyond the parameter names used in context. It explains usage but not parameter semantics like types or formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Replace' and the resource 'string in file_path', and mentions it returns a unified diff. It distinguishes from sibling tools 'read' and 'write' which have different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies prerequisites: must have read the file, old_string must appear exactly once, and file must not have changed. It provides clear conditions for use, though it does not explicitly contrast with sibling tools like 'write'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• readA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description fully discloses behavior: returns line-numbered content, records file hash for subsequent edit enforcement, and supports optional offset/limit. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each adding value. First sentence states the main action. No redundant information. Well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all necessary aspects: action, optional parameters, return format (line-numbered), and side effect (hash recording). Suitable for a low-complexity read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must add meaning. It explains offset as 1-based and limit as number of lines. Path parameter is not described, but the context is sufficient for a read tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Read a file'), the resource, and optional features (offset, limit). It distinguishes itself from sibling tools (edit, write) by its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (reading files) and provides context about hash recording for edit enforcement. However, it does not explicitly state when not to use or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.