Server Quality Checklist

Tool Scores

• createA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that a separate writing model handles structure/wording and that a fact-check loop with the edit tool follows. However, it does not clarify whether the tool writes to a file or returns content, nor does it mention side effects or permissions, leaving ambiguity in the output behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is composed of four purposeful sentences, each conveying essential workflow instructions. It is slightly verbose but every sentence earns its place, covering purpose, prerequisite, constraint, and follow-up step without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 12 parameters and no output schema, the description explains the high-level workflow and constraints well. However, it fails to describe the exact return value or delivery mechanism of the document, which is a significant gap for a 'create' tool. The missing output detail leaves the agent uncertain about what to do with the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so a baseline of 3 applies. The description adds meaningful semantics by instructing the agent to supply 'facts' and 'structured material' and explicitly prohibiting pasting a draft, which directly clarifies how parameters like context and notes should be populated. This goes beyond the terse schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool writes a software document (plan, RFC, design doc, README, guide, ADR) from supplied facts, giving a specific verb and resource. It also distinguishes from the sibling edit tool by specifying that corrections are sent through edit, making its unique role clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit when-to-use guidance: investigate the repository first and pass structured material, while also giving when-not-to-use instructions: do not draft prose yourself or paste a draft. It also points to the edit tool as the alternative for corrections, naming the sibling explicitly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reviewA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the non-mutating behavior ('never a rewrite'), the output form ('findings with locations'), and a possible null result ('no significant editorial problems'). It doesn't cover potential input limits but is transparent about core behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly written sentences. The first sentence front-loads the purpose and details what findings address; the second covers output format, constraints, and outcome expectations. No filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description partially explains the return value ('findings with locations' and the 'no significant editorial problems' possibility). It lacks a detailed findings structure, but for a review tool this is adequate to guide an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and each parameter already has a clear description in the schema. The tool description adds no additional parameter-specific meaning, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource ('Get editorial findings on a document') and enumerates concrete finding categories (inventories, structure, missing motivation, serialized prose). It also clearly distinguishes itself from the 'edit' sibling by stating 'never a rewrite'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly sets the context: use this tool for editorial feedback on a document. It provides a when-not ('never a rewrite') and a possible outcome, but it does not explicitly name when to prefer this over 'create' or 'edit'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• editA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does well: it discloses that edits will be the smallest coherent change, the document's voice is preserved, and untouched text is kept intact. This goes beyond the schema by describing the tool's conservative editing philosophy and the expected form of user input.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences: the first states the core action and constraint, the second instructs on the form of changes, and the third clarifies preservation behavior. Every sentence earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter tool with no annotations and no output schema, the description covers inputs and behavioral constraints well. The only gap is that it does not explicitly state the return format (e.g., the revised full document), but the action 'revise' implies the output is the updated document.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description adds meaningful semantics to the 'changes' parameter by requiring facts rather than replacement prose, and to 'document' by implying it must be verbatim current text. This helps the agent phrase inputs correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Revise an existing document' with the qualifier 'smallest coherent change.' This clearly distinguishes it from the sibling tools 'create' and 'review' by targeting modification of existing content rather than generation or assessment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for use: editing existing documents and requesting changes as facts rather than prose. It does not explicitly name alternatives or state when not to use the tool, but the 'existing document' phrasing and focus on minimal change imply when this tool is appropriate versus siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.