Server Quality Checklist

Tool Scores

• list_modulesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool returns an overview of modules, capabilities, and delivery forms, but does not disclose whether the data is static, cached, or reflects real-time state. With no annotations, the description should clarify if this is a read-only, safe operation. A score of 3 reflects the description gives some useful behavioral context but lacks clarity on side effects or data freshness.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 18 words, very concise. It is front-loaded with the purpose ('Overview of the catalog') and then lists what it covers. It could be slightly more structured (e.g., bullet points implied), but for zero-param tool it's efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description must explain what the tool returns. It mentions 'the four modules, how many capabilities each holds, and what the two delivery forms mean,' which gives a clear idea of the output structure. However, without any annotations or output schema, the description should also clarify whether the output is a list, a summary, or a single object. The description is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero parameters and schema description coverage is 100%. Since there are no parameters to document, the description is not required to add anything. A baseline of 4 is appropriate because there is nothing to clarify.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the verb (list/overview) and resource (catalog of modules, capabilities, delivery forms). It distinguishes from siblings: siblings 'get_capability' and 'search_capabilities' imply focused queries, while this tool provides a high-level catalog summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is an overview tool, but does not explicitly state when to use it vs siblings. For example, it doesn't say 'Use this for broad orientation; use get_capability for details on one module, or search_capabilities for filtered queries.' Guidance is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_capabilityA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries full burden. It discloses the output contents (version, repository, license facts, etc.) and implies a read operation via the tool name 'get.' Does not explicitly declare read-only or mention side effects, but is fairly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, information-dense sentence: it lists the output contents and gives a clear usage directive. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool (1 param, no output schema, no annotations), the description covers purpose, usage, and output details. It does not explain potential errors or explicitly confirm read-only nature, but overall is sufficient for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single param with a description already stating 'Get it from search_capabilities.' The tool description does not add new parameter semantics beyond what the schema provides, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides 'full detail for one capability by id' and lists specific contents (version, package size, etc.), distinguishing it from sibling tools like search_capabilities which likely return a list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to 'Call this before installing anything,' providing a clear when-to-use instruction. The schema also notes the id should come from search_capabilities, implying a prerequisite. No explicit alternatives or when-not cases, but the context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_capabilitiesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavioral disclosure. It explains that the tool returns specific fields (name, module, etc.) and mentions a direct download URL for free items, implying paid items may behave differently. It doesn't cover query behavior or side effects, but for a non-destructive search tool, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact at three sentences, each earning its place: purpose, usage guidance with examples, return fields. Front-loaded with the core action. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 4 parameters and no output schema, the description is fairly complete. It covers purpose, when to use, what is returned. Could mention how parameters affect results but the schema handles that. Sibling tools fill the remaining gaps. Not missing anything critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with each parameter already well-described (type, enum, constraints). The description adds overall context about free text matching and return fields but does not provide additional semantics beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Search' and the resource 'AgentSkill catalog of installable AI-agent capabilities'. It specifies the tool's purpose: to find capabilities the agent doesn't have. This distinguishes it from siblings 'get_capability' (likely detail) and 'list_modules' (module listing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use the tool: 'Use when the user wants a capability the agent does not have'. It provides concrete examples (e.g., market data, image upscaling). It doesn't explicitly state when not to use or name the sibling tools as alternatives, but the use case is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.