Server Quality Checklist

Tool Scores

• batch_renderA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility and clearly discloses that the tool is not implemented, will always return an error with a fixed format, and includes the exact error structure. It also explains planned architecture, which is extra context beyond the minimal stub behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The first sentence and return format are concise, but the 'Day 2/3 work' paragraph contains speculative implementation details that are not essential for an agent using the tool. This adds mild bloat to an otherwise well-structured stub description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a not-implemented stub, the description fully informs the agent that the tool will fail and what response to expect. It even includes a future roadmap. This is complete given the tool's current state, though if the tool were implemented, more behavioral details would be needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers both parameters completely with descriptive text for each (output_dir and repo_list_path), so baseline is 3. The description adds no additional parameter semantics beyond the schema's already clear definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Batch-render'), resource ('README diagrams'), and scope ('across a list of fleet repos'). This distinguishes it from sibling tools like render_svg, which presumably handles single renders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly warns 'NOT IMPLEMENTED YET,' which is crucial guidance against using the tool. However, it does not explicitly mention alternatives or provide when-to-use vs. when-not-to-use guidance beyond this warning.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• render_svgA
  3.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It fully discloses that the tool is not implemented, that it will always return a failure object with 'success': False, and specifies the error_type and suggestions fields. This is complete transparency for a stub tool, leaving no ambiguity about its runtime behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description contains unnecessary implementation details about 'Day 2 work', 'bassimeledath's warm-Chromium approach', and 'locally vendored excalidraw JS bundle' that do not help an agent select or invoke the tool. The return format is clearly structured, but the clutter makes it less concise than it should be.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with an output schema (return format), the description adequately covers the essential context: the not-implemented status, the underlying technology, and the exact failure return format. The main gap is the absence of any note about what the tool would do if it were implemented, but since it is a stub, the current information is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides descriptions for both required parameters (elements_json and output_path) with 100% coverage. The tool description adds no additional meaning or nuance beyond what the schema states, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Render Excalidraw element JSON to SVG via headless Chromium (Playwright)', providing a specific verb, resource, and method. This distinguishes it from sibling render tools like batch_render by indicating it handles a single conversion, and the 'NOT IMPLEMENTED YET' status clarifies its current non-functional state without obscuring the intended purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'NOT IMPLEMENTED YET' warning implicitly signals the agent not to use this tool until it is ready, which is a form of usage guidance. However, it does not explicitly state when it should be used, nor does it point to alternatives such as batch_render for current rendering needs, leaving the when-to-use guidance incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mermaid_to_excalidrawA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description bears full responsibility. It discloses the stub behavior and exactly what return format to expect (success: False, error_type: 'not_implemented', suggestions), plus explains the planned implementation. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the purpose and the critical 'NOT IMPLEMENTED' notice. The implementation-plan paragraph adds context but is arguably unnecessary; still, the overall structure is organized and not overly verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an unimplemented tool, the description fully covers what an agent needs to know: the tool will fail gracefully with a specific error format. The optional rendering mention is vague since there is no render parameter, but that is minor given the not-implemented status.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides clear descriptions for both parameters (output_path and mermaid_source) with 100% coverage. The description does not add additional parameter-level meaning, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the intended function: converting Mermaid diagrams to Excalidraw elements and optionally rendering. It is specific and distinguishes from sibling tools (status, render_svg, batch_render).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'NOT IMPLEMENTED YET', which is a clear instruction that the tool should not be used for actual conversions. While it doesn't name alternatives, it provides strong contextual guidance on when to avoid the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of explaining behavior. It discloses the return format, the fact that it's the sole functional tool, and that other tools are stubs. It does not mention side effects, but for a status read tool that is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two lead sentences, a return format block, and an example. It is front-loaded with purpose and includes only essential context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter status tool, the description fully covers what it does, what it returns, and how it fits into the larger toolset. The provided return format and example make it complete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is 100% and the description correctly adds no parameter details. The baseline score of 4 applies because no parameter ambiguity exists.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb 'Report' and a clear resource: 'server identity and which capability groups are wired up yet.' This unambiguously states the tool's function and distinguishes it from the sibling rendering tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides the context that this is the only functional tool in the scaffold, implying that all others return not_implemented. This tells the agent when to use it (when wanting real functionality) but does not explicitly name alternatives or give a fuller when-not-to-use scenario.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.