Server Quality Checklist

Tool Scores

• list_categoriesA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden but only states the action; it does not mention read-only nature, pagination, or response format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, direct sentence with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool, the description conveys the essential purpose; however, the absence of an output schema means the return structure is unspecified, but this is acceptable for a simple list.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters; the schema fully documents this, so baseline 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and the resource ('all available skill categories'), distinguishing it from siblings like `spawner_skills` or `get_skill`.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool vs alternatives such as `spawner_skills`; no exclusions or alternative tools are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_skillA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the YAML files included, which is useful, but it does not disclose potential side effects, authentication requirements, or error behavior. The getter nature is clear from 'Get', but additional transparency is lacking.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is concise, front-loaded with the main purpose, and lists the exact files without waste. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple two-parameter schema and no output schema or annotations, the description is sufficiently complete. It clarifies what the tool does and includes the relevant YAML files, though it could mention return format or error handling to be fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with both parameters clearly described in the input schema. The tool description adds no additional meaning about parameters, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get complete details for a specific skill' with a specific verb and resource, and lists the exact YAML files included. This distinguishes it from sibling tools like list_categories or spawner_validate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving a skill's details, but it does not explicitly state when to use this tool over alternatives or provide exclusions. No comparison with sibling tools is given, so usage guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• spawner_watch_outA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden of behavioral disclosure. It states that the tool returns common pitfalls and avoidance advice, which is useful, but it does not mention side effects, auth requirements, error handling, or the exact return format. For a read-only 'get' operation, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (two sentences, ~20 words) and front-loaded with the main action ('Get sharp edges'). Every word adds value, and the parenthetical explanation clarifies the jargon without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema or annotations), the description adequately covers what the tool does and what the result contains (pitfalls and how to avoid them). It does not explain return format or edge cases, but for a simple lookup tool, this is sufficient. A higher score would require more detail on output structure, but it's not necessary here.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of the single parameter with a clear description. The tool description adds no extra meaning beyond the schema—it just rephrases 'technology' as 'for a technology.' Therefore, the baseline of 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('sharp edges for a technology'), and explains what sharp edges are (common pitfalls and how to avoid them). It is distinct from sibling tools like get_skill or spawner_skills, which likely focus on other aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when you need pitfalls and warnings for a technology) but does not explicitly provide alternatives or exclusions. It lacks clear 'when to use vs when not to use' guidance compared to examples like 'search_calls_extensive'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• spawner_validateA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It accurately describes the operation (fetching validation rules) but does not disclose behavioral details such as what happens if no skills match, the return format, or whether the rules are returned as a list or object. The description is honest but minimally informative beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences that directly state the purpose and a usage hint. There is no filler, and every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and no output schema, but the description does not specify what the returned validation rules look like or how they are formatted. It subtly implies a use case but leaves some ambiguity about the result structure. Barely adequate for an AI agent to know what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the 'technology' parameter as 'Technology or pattern to get validations for (e.g., react, api, stripe),' and the description adds no additional meaning beyond the schema. With 100% schema coverage, the baseline is 3, and the description does not exceed that baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Get validation rules') and a specific resource ('from skills that match a technology or pattern'). This distinguishes it from sibling tools like spawner_skills or get_skill, which likely deal with listing or retrieving skills themselves.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on when to use the tool: 'Use these to validate code quality.' It does not explicitly mention alternatives or when not to use it, but the recommended use case is clear and appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• spawner_skillsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the search operation and return of matching skills with descriptions, but lacks details on matching semantics (e.g., partial vs exact), sorting, limits, or any side effects. This is a basic but acceptable disclosure for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant words. It effectively states the action, criteria, and return value without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple tool with one parameter and no output schema. The description covers the search criteria and return content, which is adequate for basic usage. It could mention that results are a list or include more about result structure, but given the simplicity, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single 'query' parameter, providing baseline 3. The description adds value by explicitly stating the query can be a name, category, or keyword, clarifying the parameter's accepted semantic range beyond the schema's generic description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches and retrieves skills by name, category, or keyword with a specific verb ('search and retrieve') and resource ('skills'). This distinguishes it from siblings like get_skill (likely specific retrieval) and list_categories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when you have a search query for skills, but does not explicitly contrast with alternative tools like get_skill or list_categories. It provides clear context for searching but no explicit exclusions or when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.