Server Quality Checklist

Tool Scores

• umlforge_frontend_componentsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true. Description adds behavioral details beyond annotations: lists six specific outputs (hierarchy graph, interaction sequence, responsibility table, etc.) and notes report_mode limitations. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections, front-loaded purpose. Some redundancy (e.g., 'Arg' list duplicates schema names). Overall efficient but not minimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, 0% schema coverage, and existence of output schema, description adequately covers tool behavior: when to use, outputs, and parameter list. Could add more detail on parameter semantics but outputs are well-described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description must compensate. However, it only lists parameters in an 'Args' section without explaining their purpose beyond what schema titles show. No additional semantic details about format, constraints, or relationships.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource: 'Design a frontend component tree and interaction flow'. Distinguishes from siblings with explicit 'USE THIS WHEN' and 'NOT FOR' sections listing specific alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit when-to-use (React/Vue/Angular/Svelte feature design, component hierarchy, prop drilling) and when-not-to (backend uses, full codebase, stakeholder arch) with sibling tool names. Provides clear context and exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• umlforge_ai_agentA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate read-only and open-world hints. The description adds context about produced artifacts (diagrams, reports), licensing restrictions for report_mode, and warning flags for tool overload. It does not contradict annotations and provides valuable behavioral insights beyond the structured fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (USE THIS WHEN, NOT FOR, Produces, Args) but is somewhat verbose, especially the 'Produces' list. It could be slightly more concise without losing clarity. Still, the organization aids readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multi-agent orchestration) and the presence of an output schema, the description covers all essential aspects: purpose, usage, produced artifacts, parameters, and constraints (licensing). It is thorough enough for an agent to effectively invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description compensates with a detailed 'Args' section explaining each parameter's purpose with examples (e.g., 'pipeline_purpose: What the agent system does (e.g. "Research pipeline...")'). This adds significant meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Design an AI agent pipeline or multi-agent orchestration system.' It provides specific use cases and distinguishes from siblings with explicit 'NOT FOR' sections, making the tool's role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes dedicated 'USE THIS WHEN' and 'NOT FOR' sections that list specific scenarios and alternative tools, such as 'use umlforge_stakeholder_arch' for general architecture. This provides excellent guidance for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• umlforge_living_docsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations (readOnlyHint=true, openWorldHint=true) are complemented by description stating it requires existing diagrams and produces updated output. Adds context on report_mode limitations (Pro/Team/Enterprise only) and format of output (%% changelog headers). No contradictions observed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with labeled sections and front-loaded purpose. Every sentence adds value, though slightly verbose with repeated emphasis on NOT FOR cases. Still efficient for the amount of information conveyed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all aspects: purpose, when to use/not use, prerequisites, parameter details, output description, and licensing note. Given the tool's complexity (4 params, 2 required) and presence of output schema, the description leaves no critical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description explains each parameter in detail: current_diagrams required with format and fencing, sprint_changes as changes summary, affected_files as optional, report_mode effect and licensing. This fully compensates for lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb+resource: 'Update diagrams you already have' with specific scope (sprint/PR changes). Distinguishes from siblings by explicitly listing what it is NOT for and suggesting alternative tools (umlforge_reverse_engineer, umlforge_stakeholder_arch).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit 'USE THIS WHEN' and 'NOT FOR' sections with concrete conditions (e.g., already have Mermaid diagrams, code changed in sprint/PR). Also gives specific alternatives for non-use cases, making guidance actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• umlforge_reverse_engineerA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and destructiveHint, but the description adds significant context about outputs (diagrams, report, architectural smells) and the mutual exclusion of github_url and codebase. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (USE THIS WHEN, NOT FOR, Produces, Args). It is front-loaded with the main purpose. Slightly verbose but each sentence adds value; condensing the NOT FOR list slightly could improve conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 parameters, multiple output types), the description covers all usage constraints, output details, and parameter documentation. It compensates for the missing output schema description by explaining what each output contains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description provides detailed semantics for each parameter (e.g., accepted GitHub URL formats, default value for max_nodes, and the effect of report_mode). This fully compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Analyse' and resource 'existing codebase' producing UML diagrams. It distinguishes from siblings via explicit 'NOT FOR' section naming alternatives like umlforge_stakeholder_arch and umlforge_living_docs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a 'USE THIS WHEN' list for appropriate contexts and a 'NOT FOR' list with explicit sibling tool names, offering clear guidance on when to use this tool vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• umlforge_suggestA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=true and destructiveHint=false, so the description's implication of non-destructive recommendation aligns. However, openWorldHint=true is not addressed, leaving ambiguity about potential side effects. The description adds detail about the output (pre-filled parameters, explanation) beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, use cases, examples) and is front-loaded with the key message. Although it is relatively long, each sentence adds value for understanding the tool's purpose and usage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (one parameter, clear annotations, no nested objects), the description covers all necessary aspects: what it does, when to use it, what to provide, and what the response will contain. There are no apparent gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has only one parameter (task_description) with no description (0% coverage). The tool description provides extensive guidance on what to include in this parameter, with examples and details on how context improves results, fully compensating for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool helps identify the right tool and returns a pre-filled tool call. It distinguishes itself from sibling tools by being the suggestion entry point, with the opening line 'START HERE if you are unsure which tool to use.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly lists when to use this tool (e.g., unsure which of the 12 tools, describe a goal) and provides concrete examples. It also mentions the response will include an explanation and an alternative, guiding the agent effectively.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• umlforge_stakeholder_archA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations (readOnlyHint=true) already indicate no state modification. The description adds context on outputs (diagrams, optional notes) without contradicting annotations. No side effects or permissions mentioned, but sufficient given annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, conditions, output, args). Every sentence adds value, and the length is appropriate for the detail provided.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and the detailed description covering purpose, usage, and parameters, the description is complete for an agent to decide when and how to invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description provides detailed explanations for each parameter (system_description, audience_description, report_mode) with examples and constraints, adding significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates C4 architecture diagrams for mixed audiences. It distinguishes from siblings by listing specific alternatives in the NOT FOR section (e.g., umlforge_reverse_engineer, umlforge_api_sequence).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit USE THIS WHEN and NOT FOR sections provide clear guidance on when to use this tool versus alternatives, with specific sibling tool names for exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• umlforge_api_sequenceA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description adds depth beyond annotations: explains produced artifacts (sequence diagram with failure paths, dependency table, optional report), and notes report_mode license restriction. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with headings and bullet-like sections, front-loaded purpose. Slightly verbose but all content earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters, no nested objects, output schema exists), description covers inputs, outputs, constraints (license), and use cases thoroughly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, description fully explains each parameter: services as comma-separated list, user_journey as plain English action, report_mode with license caveat.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it diagrams how services call each other for a specific user action or API flow. It also distinguishes from siblings with 'NOT FOR' examples.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly lists when to use (trace request, design API contracts, failure paths) and when not to use, with named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• umlforge_erd_schemaA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true and destructiveHint=false, indicating no state changes. The description reinforces this by stating it 'designs' and 'produces' a schema without modifying data. It also discloses the Pro/Team/Enterprise restriction on report_mode, which is a behavioral constraint not present in annotations. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized into sections with headings and bullet points. It front-loads the primary purpose, then provides usage context, output details, and parameter descriptions. Every sentence adds value without redundancy. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, output schema present), the description covers all essential aspects: purpose, usage boundaries, behavioral constraints, parameter semantics, and output expectations. It addresses the 'contextual completeness' dimension fully for this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description includes a dedicated 'Args' section that explains each parameter's purpose, examples, and defaults. For example, 'domain_description: What the database stores' and 'report_mode: True → also produce a Design Score report. Pro/Team/Enterprise only.' This fully compensates for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb and resource: 'Design a database schema.' It distinguishes from siblings through the 'NOT FOR' section, explicitly naming alternative tools (umlforge_reverse_engineer, umlforge_api_sequence, umlforge_event_driven). The purpose is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes explicit 'USE THIS WHEN' and 'NOT FOR' sections, providing clear context for when to invoke the tool and when to choose a sibling. It gives concrete conditions and directs to alternatives, leaving no ambiguity for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.