Server Quality Checklist

Tool Scores

• validate_diagramA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the transparency burden. It adds specific details about the types of errors checked and accepted input formats, which is useful. However, it does not explicitly state that the operation is non-destructive or describe behavior on invalid input or file-not-found conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the purpose and specific checks, with no redundant or extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a single well-documented parameter and an existing output schema, the description covers the tool's purpose and input format effectively. Minor omissions about failure modes or side effects prevent a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully documents the 'scene' parameter with the same content as the description. The description adds no extra semantic value beyond the schema, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Check') and resource ('Excalidraw scene'), enumerating specific validation checks. This distinguishes it from siblings like create_diagram and diagram_backend_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (for validating a scene) but does not explicitly contrast it with alternatives or mention exclusions. Sibling tools clearly serve different purposes, so the lack of explicit alternatives is acceptable but not ideal.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_diagramA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behaviors: shapes laid out automatically, arrows bound to shapes, labels attached to containers, and optional file writing. However, it omits potential side effects such as file overwrite behavior if save_path exists, and does not mention failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core function. Each sentence earns its place: main action, output quality details, and optional file write. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters and an output schema, the description gives a solid high-level understanding of the output (editable scene with bound arrows) and optional file write. It does not explain error conditions or model-specific usage, but the schema covers parameter details and the output schema likely covers return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds minimal parameter context by mentioning optional file writing (save_path), but does not elaborate on direction, title, or use_model beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Describe a system in plain English and get back a complete, editable Excalidraw scene' – a specific verb (get back) and resource (Excalidraw scene). It clearly distinguishes the creation function from sibling tools like validate_diagram and diagram_backend_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use the tool: when you need a diagram from a plain-English description. It sets context with details about layout and optional file writing, but does not explicitly mention when not to use it or reference alternatives like validate_diagram, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• diagram_backend_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It transparently explains what the tool reports and adds interpretive value ('telling a downgrade apart from a deliberate offline run'), which goes beyond the title. It does not describe side effects or authorization needs, but for a read-only status check with zero parameters, this is adequate and not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no superfluous content. The first sentence states the core function; the second adds practical value. It is front-loaded and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter tool with an output schema, the description is complete. It explains what the tool does and why it's useful, and the existence of an output schema means the return values need not be described in the text. No gaps are apparent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description does not need to explain parameters since there are none. The schema coverage is 100% (empty schema), and no parameter information is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: it reports whether a language model is configured or if the server will use the offline rule parser. This specific verb+resource (says/reports backend status) distinguishes it from siblings like create_diagram and validate_diagram, which perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage context: 'Useful for telling a downgrade apart from a deliberate offline run.' This implies when to use the tool (when you need to distinguish between a downgrade and an intentional offline mode), but it does not explicitly mention when not to use it or compare it to alternatives. Still, the guidance is sufficient given the simple nature of the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.