Server Quality Checklist

Tool Scores

• create_diagramA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses automated layout/styling (e.g., 'no need to specify coordinates'), but does not mention return value, error handling, or limitations. The output schema exists but isn't referenced.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short, front-loaded sentences with zero wasted words. Every sentence adds value: first states the action, second clarifies automation and reduces cognitive load for the LLM.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters and existence of output schema, the description covers the core task adequately. Missing explicit info on return value (but output schema exists, so not required per rubric). Minor gap: no mention of file saving behavior even though output_path parameter implies it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The main description adds minimal extra meaning beyond schema: it reiterates that coordinates are not needed, but parameter details are in schema. No significant enhancement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Create a new Excalidraw diagram from structured node and connection data', using a specific verb and resource. It distinguishes from sibling tools (get_diagram_info, mermaid_to_excalidraw, modify_diagram) by focusing on creation, handling layout/styling/rendering automatically.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for creating diagrams from structured data but provides no explicit guidance on when to use it versus siblings (e.g., not for modifying existing diagrams or converting from Mermaid). No 'when not to use' or alternative recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• modify_diagramA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses it modifies (not creates), supports iterative edits, and warns to check current state. However, with no annotations, it misses details on atomicity, partial failures, or side effects of operations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short paragraphs: purpose, capabilities, and a critical warning. No redundant wording, front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers the tool's purpose, iterative nature, and prerequisite call. With an output schema present, return value explanation is not needed, but error handling or operation ordering constraints are missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed descriptions of each operation's parameters. Description adds context about iterative editing but does not supplement parameter meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool modifies existing Excalidraw diagrams and distinguishes it from siblings like create_diagram by specifying 'modify an existing' and 'without recreating the entire diagram'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to call get_diagram_info first to understand current state before modifications. Lacks explicit when-not-to-use guidance but the prerequisite is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_diagram_infoA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that the tool provides node ids, labels, component types, and connection topology, implying a read-only operation. No annotations exist, but the description sufficiently covers expected behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose and usage. No superfluous information; every word contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description appropriately lists the summary contents (node ids, labels, etc.) and provides sufficient context for a single-parameter read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description for file_path. The tool description adds no additional parameter details beyond confirming the file is an existing Excalidraw diagram, meeting the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a structured summary of an existing Excalidraw diagram, distinguishing it from siblings like 'create_diagram', 'modify_diagram', and 'mermaid_to_excalidraw'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises calling this tool BEFORE modify_diagram to understand current state. While it doesn't specifically exclude other scenarios, the context is clear for appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mermaid_to_excalidrawA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description details the supported Mermaid subset and auto-detection of component types. With no annotations, it carries the full burden and mostly covers behavior, though missing details on file overwrite or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct and well-structured with bullet points for supported features. Every sentence adds value, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity and presence of an output schema, the description covers essential aspects like supported syntax and auto-detection. Minor gaps exist (e.g., error handling), but overall it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline is 3. The description adds value beyond the schema by elaborating on the supported syntax and theme options (e.g., 'professional' uses Helvetica). This aids parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool converts Mermaid flowchart syntax into an Excalidraw diagram. It specifies the supported subset, which distinguishes it from sibling tools like create_diagram, get_diagram_info, and modify_diagram that do not involve conversion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool—when you have Mermaid syntax and need an Excalidraw diagram. However, it does not explicitly state when not to use it or mention alternatives, though the context is adequate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.