Server Quality Checklist

Tool Scores

• get_diagram_examplesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description bears full responsibility. It indicates a read operation ('Get'), but lacks details on safety, idempotency, or error behavior. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no extraneous words. Every word serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description should clarify return format. It only says 'example diagram code' without specifying structure, completeness, or example nature.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes the parameter with 100% coverage. The description adds no extra semantic value beyond referencing the type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves example diagram code for a specified type. It distinguishes from siblings like generate_diagram (which creates) and list_icons (which lists icons), but does not explicitly differentiate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives, nor any exclusions or contextual hints. The agent is left without context for decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_diagramC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are empty, so description must bear full burden. It mentions 'regenerate' implying mutation, but does not disclose side effects (e.g., overwriting files), required permissions, failure modes, or dependencies on existing diagrams. Very limited behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with verb and object, includes scope constraint. No wasted words; information density is optimal for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, yet description omits what the tool returns (e.g., file path, success status). Also lacks explanation of the 'app-only' constraint. Given the tool's simplicity, more context on return value and environment assumptions would be expected.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so each parameter is already documented. Description adds no extra meaning beyond the schema fields. Baseline score of 3 is appropriate as it does not enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action ('Regenerate a diagram') and resource ('diagram'), and adds context ('from updated code', 'app-only tool'). It hints at a specific use case compared to sibling 'generate_diagram' but does not explicitly differentiate regeneration from initial generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'generate_diagram'. The term 'app-only' is ambiguous and does not clarify prerequisites or scenarios where this tool is appropriate. No exclusion criteria or usage context beyond the name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_diagramB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It mentions that the code must contain a Diagram() call, but does not disclose potential errors, execution constraints, or side effects beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is concise and front-loaded, containing no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (code execution, file output, timeout), the description is minimal. It does not address return values, error handling, or execution environment, leaving gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no additional semantic value beyond the schema's parameter descriptions, but it reinforces the overall purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'Generate' and the resource 'diagram from Python code using the diagrams package DSL'. It distinguishes from sibling tools like get_diagram_examples and list_icons, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives, nor does it mention prerequisites or exclusions. The agent must infer usage from context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_iconsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It only states the tool lists icons, lacking details on read-only nature, sort order, or any constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no superfluous words, effectively conveying the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description omits return format details (e.g., flat vs. hierarchical list) and pagination. This leaves gaps for an agent to understand the full result structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters have schema descriptions covering filter options. The description reiterates 'organized by provider and service' but adds no meaningful extra information beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists diagram icons organized by provider and service, which distinguishes it from sibling tools that generate or refresh diagrams.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The context of sibling tools suggests it is for browsing available icons, but no when-not or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.