Server Quality Checklist

Tool Scores

• get_package_docsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so description must cover behavioral traits. It only states the return type (README or description text) but omits error handling, authentication needs, or rate limits. With no annotations, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers basic purpose and return type, but lacks details on error cases, output format, or behavior when package not found. No output schema, so description could add more completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with both parameters well-described ('Package name' and 'Language/ecosystem' with enum and auto-detection note). Description adds no extra parameter semantics beyond workflow advice.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool fetches documentation content and returns README or description text. However, it does not differentiate from the sibling 'search_package_docs', which might also return documentation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using get_package_info first to check version, establishing a clear workflow. No exclusions or alternatives mentioned for when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cache_statsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, and the description does not disclose behavioral traits such as whether the operation is read-only, if it resets statistics, or requires authentication. The simple verb 'Get' implies non-modifying behavior, but this is not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the tool's purpose efficiently with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no parameters and no output schema. The description lists the three statistics returned, which is sufficient for an agent to understand the return value. It could be improved by noting the format (e.g., numbers or object), but is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters in the input schema, and the description adds no parameter information beyond what the schema already shows. Per calibration, 0 params baseline is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'cache statistics', specifying the included fields (total entries, valid, expired). This distinguishes it from sibling tools like get_package_info, which deal with packages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description only states what it does, without context for selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_package_docsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes discovery sources, result selection (highest-scoring pages), and bounding constraints. No annotations provided, so description carries the full burden; it adequately discloses non-destructive read behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, front-loaded with purpose, zero wasted words. Efficiently covers purpose, sources, return format, and constraints.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and complete parameter descriptions, description covers what the tool does, sources it uses, and result bounding. Lacks explicit return structure but sufficient for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear descriptions for all 4 parameters. Description does not add additional detail beyond schema, so baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states action (search), resource (package documentation), and scope (discovers pages from specific sources). Differentiates from sibling tools like get_package_docs and get_package_info by focusing on search and bounded results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for searching documentation but lacks explicit when-to-use or when-not-to-use compared to siblings like get_package_docs. No guidance on alternatives or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detect_project_dependenciesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses the local-only constraint and lists supported file types, which is helpful. However, it does not describe additional behaviors such as recursion depth, handling of missing files, performance guarantees, or whether it scans subdirectories.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. It front-loads the key action and resource, then adds supported ecosystems and the local-only constraint. Every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 optional parameters, no output schema, and no annotations, the description covers purpose, supported formats, and local scope well. Missing details include the output structure (does it return a list of dependency names? versions? tree?) and error handling for missing manifests.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Parameter schema coverage is 100%, and the description adds value by clarifying 'root' as the project root directory, 'include_dev' as dev/test/peer dependencies, and listing supported ecosystems not detailed in the schema. This goes beyond the schema's basic descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Inspect manifests/lockfiles') and resource ('local project directory') and clearly identifies supported ecosystems (Python, npm, Rust). It distinguishes from sibling tools like get_package_info by focusing on local dependency detection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly notes 'Local-only — runs on the user's machine,' setting clear context for when to use this tool (local project analysis) vs. alternatives. However, it does not explicitly state when not to use it or mention alternative tools for remote analysis.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_package_infoA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the behavior clearly: what data is returned (version, description, docs URL, repository, license) and mentions auto-detection of ecosystem. It does not cover rate limits or errors, but the behavior is transparent for a simple lookup tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, two sentences covering what the tool does and the supported ecosystems. No redundant information, each sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains the return values. It covers the ecosystem parameter and its auto-detection. However, it does not mention any constraints (e.g., network dependency) or error cases, but for a simple metadata tool it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both parameters. The description adds value by listing the supported ecosystems explicitly (Python via PyPI, JS/TS via npm, Rust via crates.io) and mentioning auto-detection, which goes beyond the enum values in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'gets metadata for a package' and lists specific metadata fields (version, description, docs URL, repository, license). It also specifies supported ecosystems (Python, JS/TS, Rust), distinguishing it from sibling tools like get_package_docs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving package metadata but does not explicitly state when to use versus alternatives or when not to use. No exclusions or conditional guidance are provided, thus adequate but lacking differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.