Server Quality Checklist

Tool Scores

• check_dependency_licenseA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explains the meaning of results ('A permissive result means no source-disclosure duty; a blocked result means...') but does not state whether the operation is read-only, has side effects, or any error conditions. This is adequate but not rich; a note about being non-destructive would improve it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and immediately provides usage timing. Every sentence adds value: the first defines what it does and when to use it; the second explains result interpretation. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters (3 enums), an output schema, and no annotations, the description covers the key behavioral context: when to call, what question it answers, and how to interpret results. It ties the distribution_model to the shipping model. It does not mention error handling or edge cases, but the output schema likely handles that. Overall, it's complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already explains all parameters in detail (including enums and examples). The description adds no parameter-specific guidance beyond the schema, which meets the baseline for high coverage. It does hint at the importance of distribution_model by referencing 'how this project ships', but this is not about parameter syntax.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Determine whether adding or keeping a single open source dependency creates a legal obligation.' It also contextualizes it with 'given how this project ships' and explicitly says to call it BEFORE adding a dependency, which distinguishes it from broader checks like check_manifest_licenses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance: 'Call this BEFORE adding a new dependency to a project, and when auditing an existing one.' It also explains what a permissive vs. blocked result means, giving actionable next steps. However, it does not mention when NOT to use this tool (e.g., for multiple dependencies) or name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_licenseA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses that the tool covers all shipping models at once, which is a useful behavioral trait, but it does not describe return format, side effects, or assumptions about the linkage parameter. The description offers some insight but lacks a broader behavioral picture.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose and followed by usage context. No redundant words or filler. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description gives enough for an agent to know when and how to invoke the tool. It clearly differentiates from siblings and describes the input. It could briefly mention what the output entails (e.g., per-shipping-model breakdown), but the missing details are not critical for invoking correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The tool description does not add extra meaning beyond the schema – it merely mentions the input in passing. Baseline of 3 applies because the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('explain') and the resource (SPDX license identifier/expression), and specifies the scope ('what it requires across every shipping model at once'). It further distinguishes the tool from sibling tools by noting it is for the license itself 'rather than a specific package', with concrete examples.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly provides a condition for use: 'Use when the question is about the license itself rather than a specific package'. Examples (comparing AGPL vs GPL, deciding what a project may depend on) clarify the intended scenario, and the phrasing implicitly contrasts with dependency-checking tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_manifest_licensesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the tool's behavior (scans manifest, reports dependencies with obligations), notes automatic format detection, and explains that package-lock.json embeds licenses, avoiding registry lookups. It implies a read-only operation but does not explicitly state absence of side effects. Minor gap, but context is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (three sentences) and front-loaded: the first sentence defines purpose, the second covers usage, and the third adds actionable input guidance. Every sentence earns its place with no fluff, and the structure flows logically.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given only two parameters, no output schema, and no annotations, the description fully covers the tool's purpose, when to use it, and how to provide input effectively. It also addresses the distinction from siblings, making it complete for the context. No gaps are evident.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds significant value by advising to pass a lockfile for transitive dependencies and explaining why package-lock.json is best (embeds licenses, no lookups). It also lists accepted formats implicitly through schema, but the description reinforces the preferred format and rationale, going beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb-resource pair: 'Scan an entire dependency manifest and report every dependency whose license creates an obligation for this shipping model.' It clearly distinguishes from sibling tools check_dependency_license (single dependency) and explain_license (license explanation) by focusing on a whole manifest and filtering based on the shipping model.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage context is given: 'Use when reviewing a project as a whole, preparing for due diligence, or after a large dependency change.' It also provides actionable advice to pass a lockfile (package-lock.json) and explains why it's preferred, covering when to use this tool over alternatives. This exceeds the requirement for clear context and exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.