Server Quality Checklist

Tool Scores

• build_review_checklistA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so safety is covered. The description adds valuable context by disclosing that selection is 'deterministic local keyword relevance' (not AI) and that the tool returns 'every entry from each selected checklist,' which clarifies the behavior beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main action, and contains no redundant or filler information. Every sentence contributes meaning, making it appropriately concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core purpose, return behavior, and deterministic selection, while annotations cover safety. However, the 'limit' parameter's semantics are not explained in either the schema or the description, leaving ambiguity about whether it limits the number of checklists or entries. This is a minor gap, so the description is not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides descriptions for 'description' and 'categories', and constraints for 'limit'. The tool description adds context about matching based on a described UI but does not clarify the role of 'limit'. With 67% schema coverage, the description only marginally supplements the parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Select... and return every entry'), identifies the resource ('Checklist.design checklists'), and defines the scope ('screen, component, product area, or flow'). It also distinguishes itself from an AI design review and implicitly from siblings like search_checklists by emphasizing the return of every entry from each selected checklist.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when describing a screen, component, product area, or flow, but it does not explicitly state when to use this tool versus the sibling tools (e.g., search_checklists, get_checklist) or provide exclusions. Usage is implied rather than articulated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_checklistA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds that the response includes all sub-elements and that a qualified identifier is preferred, hinting at resolution behavior. However, it doesn't disclose what happens with ambiguous identifiers, not-found cases, or whether responses are paginated — minor gaps given the good annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first states the core function and return contents; the second gives a practical input tip. Every word earns its place, with no redundancy or filler. The most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only, idempotent single-get tool with no output schema, the description sufficiently communicates what the caller will receive (all checklist components) and how to identify the target (qualified identifier). It could go further by stating error behavior or what happens if the checklist doesn't exist, but those are not critical given the simple context and strong annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds an explicit example ('mobile/checkout') and advises preferring a qualified identifier, which clarifies the format and resolution preference beyond the schema's generic 'UUID, slug, or qualified category/slug identifier.' This elevates the score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Return one complete checklist' — a specific verb and resource — and enumerates the full scope of what's included (every item, suggestion, step, image URL, related checklist, collection, attribution). This clearly distinguishes it from siblings like list_checklists (which likely returns summaries) and search_checklists (which finds checklists).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The only guidance is 'Prefer a qualified identifier such as mobile/checkout,' which is about input format, not when to choose this tool over alternatives. There's no explicit mention of when to use get_checklist vs list_checklists or search_checklists, though it's implied that you use this when you need the full content of a single checklist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_checklistsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the bar is lower. The description adds that it returns summaries (not full checklists) and supports filtering, but does not disclose ordering, pagination behavior, or response format. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short, front-loaded sentences with no redundant wording. The core action ('Browse checklist summaries') is stated first, and the second sentence efficiently covers filters and the sibling tool alternative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with strong annotations and a schema defining pagination, the description covers return type (summaries), optional filters, and when to use search_checklists. It omits explicit pagination wording, but defaults are in the schema and it is a standard pattern.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50% (query and category have descriptions). The description repeats these and adds 'exact text phrase' to clarify query semantics, but does not explain limit/offset, which remain self-explanatory pagination fields. Moderate compensation for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool browses checklist summaries and supports optional filtering by category and phrase. It explicitly differentiates this tool from search_checklists, which is for relevance-ranked discovery, making the purpose specific and distinct from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly directs users to use search_checklists for relevance-ranked discovery, providing a clear alternative for a different use case. It also implies this tool is for structured browsing/filtering, giving clear context for when to choose it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_checklistsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnly, idempotent, and non-destructive hints, covering safety. The description adds non-obvious behavioral context: results are ranked locally and include matching entry excerpts, which is valuable beyond what annotations convey. No contradictions detected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the action, with no redundant or filler content. Every phrase adds value, specifying search scope and result behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with no output schema, the description adequately covers searchable fields, ranking, and excerpts, giving the agent a reasonable expectation of behavior. A minor gap is the lack of mention of when to use this tool versus alternatives, but that is more usage guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers query and category with descriptions (67% coverage). The description adds meaning by listing the searchable fields, which clarifies what the query targets and compensates for the moderate schema coverage. It does not address limit or category, but these are already described or self-evident.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches checklists across multiple specific fields (names, descriptions, categories, items, suggestions, and flow steps), with the verb 'Search' and a well-defined resource. It distinguishes itself from sibling tools like list_checklists and get_checklist by describing the search scope and local ranking behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage via the verb 'Search' but does not explicitly state when to choose this tool over alternatives such as list_checklists or get_checklist. No exclusions or alternative recommendations are provided, so the guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_catalog_summaryA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds valuable detail about what the tool returns (source, attribution, refresh time, counts), which is beyond the annotations. No behavioral contradictions are present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that immediately states the purpose and enumerates the return fields. There is no redundant filler or repetition of the title/name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description fully specifies what the tool returns. It covers the essential components expected from a summary view, making it complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty. The description correctly omits parameter details. As the schema coverage is effectively 100% (no params to document), a baseline score of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Return' with a clear resource ('catalogue summary') and enumerates the exact fields provided. This differentiates it from sibling tools like list_checklists (which likely lists items) and get_checklist (which retrieves a single checklist).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its usage context by focusing on aggregate summary data (counts by category), which is clearly distinct from the sibling tools. However, it does not explicitly state when to prefer it over alternatives or provide exclusions, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.