Server Quality Checklist

Tool Scores

• audit_dependenciesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool returns a verified finding, the lead, split, and verdict basis, and states it is deterministic. It does not cover potential side effects or auth requirements, but the behavior is clearly outlined.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, densely packed with information, and front-loaded with the core action ('Audit a single manifest/lockfile'). Every sentence contributes without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple two-parameter input and no output schema, the description adequately covers input format, supported file types, and output components. It lacks error handling or edge cases, but is reasonably complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description explains the two parameters: 'raw lockfile text' corresponds to lockfile_content and 'its filename' to filename, with examples. This adds meaning beyond the schema, though parameter-level details are minimal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it audits a single manifest/lockfile and returns a verified finding. It distinguishes from siblings (audit_project, audit_repo) by focusing on a single file, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description instructs users to pass raw lockfile text and filename, and lists supported file types. It implies this tool is for individual files but does not explicitly contrast with sibling tools for projects or repos, leaving some room for interpretation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_projectA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Given no annotations, the description fully discloses behavior: deterministic, no LLM, versions from installed lockfile, advisories via OSV. It also explains the pairing effect on dependency classification.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear summary, followed by necessary details. It is slightly lengthy but each sentence adds value. Minor redundancy in the pairing explanation could be tightened.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the usage and parameters are well covered, the description lacks details about the output format (e.g., the structure of the verified result, how advisories are presented). Without an output schema, this gap reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description compensates fully by explaining the files parameter as a {filename: content} map, giving concrete examples (package.json + package-lock.json), and clarifying allowed prefixes and value interpretations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool audits manifest/lockfiles together, with a specific verb and resource. It distinguishes itself from siblings by emphasizing the pairing logic to recover direct-vs-transitive split.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear when-to-use guidance, explaining the benefit of sending both manifest and lockfile, and that single-file works too. It does not explicitly mention alternatives like audit_dependencies or audit_repo, but the context is sufficient for correct usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_repoA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that it shallow-clones, returns an error (no fabrication), and shares output shape with audit_dependencies. No annotations exist, so description carries burden; it could mention more about side effects or prerequisites but is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each with distinct purpose: action, usage context, and behavior. No wasted words, front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required param, no output schema, no annotations), the description covers all essential aspects: purpose, when to use, error behavior, and output shape similarity. Sibling tools are referenced, providing adequate context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description adds value by explaining that the single parameter is a GitHub repo URL and contextualizing its use. It does not provide format constraints but the parameter is straightforward.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool shallow-clones a public GitHub repo and returns a verified result. It distinguishes from sibling tools by specifying the input type (repo URL vs raw lockfile).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool ('Use when you have a repo URL rather than a raw lockfile') and references the sibling tool audit_dependencies for comparison. Also provides an error guarantee.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.