Server Quality Checklist

Tool Scores

• get_utility_infoA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must cover behavior. It indicates a read operation without side effects, but does not address error cases (e.g., missing slug), authentication needs, or rate limits. This is adequate for a simple retrieval but lacks full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently conveys the core function and included fields, with no extraneous information. It is well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple single-parameter schema and no output schema, the description adequately explains what the tool returns and how to use it. It could mention the outcome for invalid slugs, but overall it is sufficiently complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% with a description for 'slug'. The description's mention of 'by slug' mirrors the schema, adding no new semantic value beyond what the schema already provides. Therefore, the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a full catalog entry for a single utility/agent by slug, listing specific fields (name, tagline, description, price, etc.). It differentiates from sibling tools like list_utilities and search_utilities, which handle multiple results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies the required identifier (slug) but does not explicitly provide when or when not to use this tool versus siblings. However, the context of siblings implies that get_utility_info is for a single detailed entry, while list_utilities gives an overview and search_utilities finds by criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_utilitiesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses caching (5 minutes) and return structure (catalog JSON with utilities and agents arrays), which is transparent for a simple read-only tool. However, it does not mention authentication or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two efficient sentences: first states purpose and source, second adds caching and return format. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description sufficiently covers the action, source, caching, and return format. It could mention the JSON structure details but is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the baseline is 4. The description adds no parameter info since none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('every utility and agent') and distinguishes from siblings by indicating it returns all items from the registry, whereas siblings like get_utility_info or search_utilities likely target specific items.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies broad listing use but does not explicitly state when not to use it or mention alternatives like search_utilities or get_utility_info for filtered queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_utilitiesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses fuzzy matching behavior, fields searched, result limit of 25, and relevance-based sorting. While it does not mention pagination or error handling, the provided traits are sufficient for a read-only search operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose, and every sentence adds value. No extraneous information, efficiently conveying scope and behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool with one parameter and no output schema, the description adequately covers input (keyword) and output (top 25 sorted results). It lacks mention of empty results or pagination but is complete for typical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a single query parameter already described with examples. The description adds meaningful context beyond schema by specifying fuzzy matching and the fields searched, enhancing understanding of how the parameter is used.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Fuzzy-search the CustomClaw registry by keyword' with specific verb and resource. It details matching fields (slug, name, tagline, description, category) and output (top 25 sorted by relevance), clearly distinguishing it from siblings like get_utility_info (specific utility) or list_utilities (all utilities).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for keyword search but does not explicitly state when to use this tool versus alternatives (e.g., for exact match use list_utilities, for specific utility use get_utility_info). No exclusion criteria or when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• install_utilityA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that it writes files, returns a list of files and npm dependencies, and refrains from running npm install. However, it does not mention potential side effects like overwriting existing files or required permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, efficiently structured. The first sentence states the main action, the second provides critical usage details (session_id and npm install behavior). No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema but the description fully explains return value (list of files and npm dependencies) and what the tool does not do (npm install). For a tool with 3 parameters and no nested objects, this is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description adds context beyond the schema: slug is clarified as 'utility slug', target_dir includes guidance on absolute vs relative paths, and session_id explains where to find it (Stripe checkout session_id from receipt email).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch a CustomClaw utility payload and write its files into target_dir', which is a specific verb and resource. It distinguishes from sibling tools (get_utility_info, list_utilities, search_utilities) which are all informational while this is an installation tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to pass session_id (for paid utilities) and clarifies that the tool does NOT run npm install itself, leaving that decision to the host agent. This helps the agent understand when and how to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.