Server Quality Checklist

Tool Scores

• audit_planC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description is read-only and idempotent per annotations, and it does not contradict them. It implies a non-destructive analysis action ('score a proposed library plan') without adding extra behavioral context such as side effects, error conditions, or dependencies. Since annotations already cover the safety profile, the description is minimally sufficient but does not disclose anything beyond them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, information-dense sentence with no fluff. It front-loads the core action and object, and every word contributes to the meaning. This is a model of conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with six parameters, including a nested structure (verifications) and multiple enums, the description is far too sparse. It does not clarify the expected format of the 'proposed library plan,' what constitutes a 'score,' or how the nine categories relate to the inputs. While annotations confirm safety, the description lacks crucial operational detail for an agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description was expected to compensate by explaining parameters, but it does not mention any of the six parameters (verifications, existing_stack, etc.). The sentence offers no clues about what 'verifications' or 'selected_libraries' mean, leaving the agent without useful semantic grounding for the inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly uses an active verb ('score') targeting a specific resource ('proposed library plan') and specifies the criteria ('nine OrchestrUI quality and licensing categories'). This distinguishes it from sibling tools like list_libraries or get_install_instructions, though it does not explicitly name a sibling for contrast. The purpose is clear and non-tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There is no mention of use cases, prerequisites, exclusions, or references to sibling tools like recommend_stack or search_components. Users are left to infer the tool's role in their workflow without explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_library_guidanceB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful context about the returned guidance categories and 'official-source' framing, but does not disclose behavioral details such as lookup failure handling, authentication, or rate limits. With annotations present, this is adequate but not exceptional.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that front-loads the core action ('Return'), names the target ('one ecosystem'), and lists the content categories without filler. Every element contributes meaning and the length is appropriate for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one parameter, openWorldHint=false, and read-only annotations, the description is nearly sufficient. However, there is no output schema, so the exact return structure is not documented, and library_id semantics are left implicit. The listed guidance categories give a reasonable overview, making this minimally complete but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for explaining library_id. The description only says 'for one ecosystem' and does not clarify that library_id identifies the library/ecosystem, how to obtain it, or what format is expected. This is a meaningful gap given there is no schema-level parameter description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the verb ('Return') and the resource ('guidance for one ecosystem'), and enumerates the content categories (use, avoid, compatibility, legal, official-source). It is reasonably distinct from siblings like get_install_instructions and list_libraries, though it does not explicitly differentiate itself and 'ecosystem' versus 'library' is slightly imprecise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells what the tool returns but does not give any guidance on when to use it versus alternatives such as get_install_instructions, recommend_stack, or search_components. There is no when-to-use, when-not-to-use, or alternative routing, leaving the agent to infer context from the tool name and sibling list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_install_instructionsB
  3.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, but the description adds valuable context: the tool returns inert text and never executes shell or package-manager commands. This reinforces the safety profile and goes beyond the annotations. No contradiction exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences and both earn their place. The primary purpose is front-loaded, and the important behavioral safety note is included without filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, read-only tool, the description conveys the essential purpose and safety behavior. However, the lack of parameter explanation, especially for component, and no specification of what 'official install command' looks like in the response make it only minimally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the lack of parameter documentation. It does not: library_id is somewhat self-explanatory, but component's role is ambiguous, and no parameter details are provided beyond schema property names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns an official install command as inert text, identifying the resource and the tool's non-executing nature. It does not explicitly differentiate itself from sibling tools like get_library_guidance, which would earn a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus alternatives. The description explains what the tool does but gives no context about when an agent should select it over get_library_guidance or recommend_stack.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_componentsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds that it uses a 'verified metadata-only fallback,' which is useful behavioral context beyond annotations. However, it does not disclose quirks like rate limits, parameter defaults, or what happens with empty results. With annotations covering safety, the description adds moderate value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence, concise and front-loaded with the primary action. It includes a key fallback detail efficiently. However, it could be more structured with a second sentence about when to use it, earning a 4.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 parameters, no output schema, and one required parameter. The description adds minimal context about the fallback but does not explain parameter usage, return format, or how it relates to sibling tools like get_install_instructions. For a search tool, an agent needs to know what 'metadata' means, how query filtering works, and the impact of 'live' parameter. The description leaves too much unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it mentions no parameters at all. It does reference 'selected library', which implies library_id, but does not explain the purpose of live, limit, or query. Given 4 parameters and 0% coverage, the description should provide more detail on parameters. Baseline for zero coverage is low, so a score of 3 is generous given the lack of parameter explanation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Search') and resource ('a selected library's public official registry'), distinguishing it from sibling tools like get_install_instructions and list_libraries. It also notes a metadata-only fallback, adding specificity. However, it does not explicitly mention the returned component metadata or differentiate from all sibling tools explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (searching for components in a library's official registry) and hints at a fallback for libraries without one. But it does not provide explicit 'when-not-to-use' guidance or mention alternatives like get_library_guidance. It gives some context but lacks explicit routing among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recommend_stackB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered and the bar is lower. The description adds one genuinely useful behavioral trait: task text is a 'legacy inference path,' signaling it is a deprecated/fallback input rather than the primary one. No contradiction with annotations exists, but the description does not disclose output shape, ranking behavior, or handling of conflicting inputs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single 21-word sentence with zero filler. The primary function is front-loaded ('Compute a minimal UI composition from structured host/task profiles') and the legacy qualifier trails naturally. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a high-complexity tool: 6 parameters, two nested objects containing roughly 18 fields total, no output schema, and 0% schema coverage. A single sentence cannot carry that weight. The agent is left without any indication of what the tool returns, how existing_stack or constraints influence the recommendation, or how the duplicated fields relate. Significant gaps remain for correct invocation and interpretation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description carries the burden and partially compensates: it maps 'structured host/task profiles' to host_profile and task_profile, and 'task text' to the task parameter, clarifying the primary vs legacy input paths. However, constraints, existing_stack, and rive_asset_rights are entirely unmentioned, and the duplication of constraints/rive_asset_rights at both top level and inside task_profile is not clarified. Meaningful but incomplete compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Compute a minimal UI composition') with a clear input source ('structured host/task profiles'), and the title anchors the resource as a 'minimal UI stack'. It is distinguishable from siblings — list_libraries, search_components, and get_install_instructions are about discovery/guidance, while this composes a recommendation. Minor ambiguity: 'composition' vs 'stack' terminology isn't unified, and the exact nature of the output is left abstract.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'with task text supported as a legacy inference path' is an implicit usage signal: structured profiles are the preferred path and free-text task is a fallback. However, there is no explicit when-to-use vs alternatives (e.g., when to pick this over search_components or list_libraries), and no exclusion criteria or prerequisites are stated. The guidance is implied rather than spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_librariesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=false, so the agent knows this is a safe read-only operation. The description adds 'Returns metadata only'—a useful disclaimer that goes beyond the schema. However, it does not detail what 'metadata' includes (e.g., supported versions, installation status), but the annotations carry most of the safety burden. This is slightly better than a baseline 3 because it explicitly limits scope, yet not rich enough for a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that names the purpose, the fixed set (seven ecosystems), the optional filter, and the return type. Every word earns its place; it is front-loaded with the core action and avoids fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only list tool with one optional parameter and no output schema, the description is mostly complete. It tells the agent what the tool does, the scope (seven ecosystems), and that it returns metadata only. The only minor gap is that it does not explain what happens if the role filter does not match—but given the trivial complexity and rich annotations, this does not significantly impair usability.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the single optional parameter's description ('Optional role fragment such as charts or marketing-ui') is already explicit in the schema. The tool description merely mentions 'optionally filter by role' without adding new semantic detail (e.g., whether filtering is case-sensitive, partial match, or returns all if empty). Since the schema fully covers the param, a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the verb ('List'), the resource ('the seven supported UI ecosystems'), and adds a key distinguishing detail ('optionally filter by role. Returns metadata only'). This clearly differentiates it from siblings like get_library_guidance or recommend_stack by emphasizing a fixed set (seven) and metadata-only scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use it (to list UI ecosystems) and hints at a filtering option (by role), but it does not explicitly state when not to use it or name alternatives. Sibling names suggest related tools, but the description does not explicitly route to them, so it misses the 'when-not' guidance that would earn a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.