Server Quality Checklist

Tool Scores

• get_diagram_examplesA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what the tool returns ('ready-to-use example code,' 'Dictionary with example code... organized by example name') and its educational purpose. However, it doesn't mention potential limitations like rate limits, authentication needs, or error conditions. The description adds value but lacks comprehensive behavioral context for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, usage instructions, example categories, features, parameters, returns) and front-loaded key information. However, it includes some redundancy (e.g., repeating parameter details) and could be more concise by eliminating the bulleted list of features, which adds length without critical information. Most sentences earn their place, but there's minor room for trimming.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is mostly complete. It explains the purpose, usage, parameter options, and return format. The main gap is the lack of output schema, but the description compensates by specifying the return type ('Dictionary with example code...'). It could improve by detailing error cases or example structure, but it's sufficient for this simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter (diagram_type) with its options and default. The description repeats this information in the 'Parameters' section and adds context about what each category demonstrates (e.g., 'aws: AWS cloud architecture diagrams'), but doesn't provide significant additional semantics beyond what's in the schema. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get example code for different types of diagrams.' It specifies the verb ('Get') and resource ('example code for different types of diagrams'), and distinguishes it from sibling tools like generate_diagram (which creates diagrams) and list_icons (which lists icons). The description explicitly mentions using these examples before creating custom diagrams with generate_diagram.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage instructions, including when to use this tool ('to understand the syntax and capabilities... before creating your own custom diagrams') and when to use an alternative ('When ready... use generate_diagram'). It also lists example categories and explains how to use the examples as templates, offering clear guidance on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_iconsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool 'dynamically inspects the diagrams package,' describes the hierarchical return structure, and explains efficiency benefits. However, it doesn't mention potential limitations like rate limits, error conditions, or authentication needs, which could be relevant for a dynamic inspection tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, usage instructions, example workflow, returns) and uses bullet points effectively. It's appropriately sized for a tool with two parameters and complex usage patterns. Minor verbosity in the example workflow could be tightened, but overall it's efficient and front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is largely complete. It explains the purpose, usage, parameters, and return format. The main gap is lack of output schema, but the description compensates by describing the return structure. It could benefit from more behavioral context (e.g., error handling), but covers core needs adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds value by explaining how parameters work together in a workflow (e.g., 'Call with provider_filter to get all services and icons for that provider'), but doesn't provide additional semantic details beyond what's in the schema. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List available icons from the diagrams package, with optional filtering.' It specifies the verb ('list'), resource ('icons from the diagrams package'), and scope ('available providers, services, and icons'). It distinguishes from sibling tools like 'generate_diagram' and 'get_diagram_examples' by focusing on listing rather than generating or retrieving examples.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage instructions with three scenarios (no filters, provider_filter only, both filters) and an example workflow. It explains when to use this tool ('more efficient than loading all icons at once') and implies alternatives by distinguishing from sibling tools. The guidance is clear and actionable for an AI agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_diagramA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and delivers comprehensive behavioral details: it explains the tool executes code with show=False, returns a file path, has timeout control (90s default), saves to specific directories, supports various diagram types, and includes important constraints like case-sensitive icon names and no imports needed. This covers execution behavior, output format, and operational limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (USAGE INSTRUCTIONS, CODE REQUIREMENTS, etc.), but it's verbose with repetitive information (e.g., multiple mentions of workspace_dir and icon usage). Some content could be condensed without losing clarity, as not every sentence adds unique value beyond what's already stated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (code execution, file generation) and lack of annotations/output schema, the description provides complete context: it covers purpose, usage workflow, behavioral details, parameter guidance, supported diagram types, return format, and critical implementation notes. No significant gaps remain for an agent to understand and invoke the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds significant value by explaining parameter semantics beyond the schema: it emphasizes workspace_dir as 'CRITICAL' for saving diagrams accessibly, clarifies code must start with 'with Diagram(' and use specific icons, and mentions filename generation behavior. However, it doesn't fully detail timeout implications or all parameter interactions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a diagram from Python code using the diagrams package' and specifies it generates PNG output without display. It distinguishes from siblings by mentioning they should be used first for preparation (get_diagram_examples, list_icons), establishing a clear workflow hierarchy.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage instructions with a numbered list (1-8), including when to use sibling tools first, specific coding requirements, and critical context like 'Always send the current workspace directory.' It clearly defines the workflow and prerequisites for successful tool invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.