Server Quality Checklist

Tool Scores

• docs_get_readmeA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare read-only, idempotent, open-world, and non-destructive, and the description adds substantial behavioral detail beyond that: verbatim markdown for npm/PyPI, HTML-converted-to-text for Cargo, truncation at 12000 chars with a 'truncated' flag, fallback via jsDelivr, and explicit 'never fabricates content' error handling. This is rich, non-obvious context that fully informs the agent of side effects and output nuances.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear paragraphs: purpose, ecosystem nuances, args, return format, error handling. It is longer than average, but the complexity of three registries and important caveats (truncation, HTML conversion, fallback) justifies the length. The 'Args' section is somewhat redundant with the schema, preventing a perfect score, but every other sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, so the description must document return values, and it does: 'readme', 'truncated', 'source_url', 'fetched_at'. It also covers truncation length, error behavior, fallback logic, and per-ecosystem differences. Combined with rich annotations and a small param count, this description is exceptionally complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage of all three parameters, including defaults ('version' description notes it defaults to latest) and enums. The description's 'Args' section largely repeats this schema information without adding new parameter-level semantics. The ecosystem-specific differences (e.g., Cargo returns HTML-converted-to-text) are more behavioral than parameter semantics, so the description adds only marginal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a highly specific verb and resource: 'Fetch the real README/description for a package, straight from the registry.' It immediately distinguishes itself from sibling tools by emphasizing 'not a summary, not a paraphrase, not AI-generated,' which contrasts with docs_search_docs or docs_get_package_info. The per-ecosystem details further clarify scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates when to use this tool: when you need the exact, unmodified README rather than a synthesized or summarized description. It explains the behavior across npm, PyPI, and Cargo, giving context on what 'real' means per ecosystem. However, it does not explicitly name sibling tools or state when not to use this tool, so it lacks explicit exclusions/alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• docs_search_docsA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnly, idempotent, openWorld), the description adds meaningful behavioral details: it matches keywords over actual fetched docs, does not summarize/paraphrase, returns empty array on no matches, and returns an error if the package/docs cannot be fetched. This gives the agent a clear model of what to expect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with bolded section headers and front-loaded purpose, but the Args list largely duplicates the input schema, making it slightly longer than necessary. It remains clear and every non-Args sentence carries important behavioral context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description fully specifies the return JSON structure (snippets array with match/context/line_hint, source_url, fetched_at) and error handling (error string vs empty array). It covers all five parameters' semantics via schema plus the tool's overall behavior, making it complete for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description's Args section repeats schema information (exact name, max cap, default version) without adding new parameter-level semantics; the main added value is return-format and error-handling context rather than richer parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Search for keywords inside a package's real README/docs' and immediately distinguishes itself by promising 'verbatim matching snippets with surrounding context' and explicitly ruling out summarization or general knowledge. This clearly differentiates it from sibling tools like docs_get_readme or docs_resolve_library.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description establishes clear context for when the tool is appropriate: keyword matching over real fetched documents, with a note that it does not summarize or answer from general knowledge. However, it does not explicitly name alternative tools or state when not to use it, so it stops short of the 'explicit when/when-not' standard.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• docs_get_package_infoA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial context beyond the annotations: it makes a live HTTP request, never returns cached/guessed/simulated data, returns explicit errors for missing packages or network failures, and provides verification fields. This complements the readOnly/idempotent hints without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose sentence, an Args block, a Returns line, and an Error Handling section. It is information-dense without redundancy, earning its length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the rich schema/annotations, the description fully covers return fields and error behavior despite lacking an output schema. It tells the agent exactly what to expect, making it complete for safe invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters are fully described in the schema with 100% coverage, so the description's Args section mostly reiterates the schema. It adds example package names but no additional semantic constraints beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it fetches real, current package metadata from npm, PyPI, or crates.io, with a specific verb and resource. It distinguishes itself from sibling tools by focusing on metadata retrieval rather than vulnerabilities, readmes, or search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (when live, accurate metadata is needed) and emphasizes its live HTTP behavior. However, it does not explicitly compare against sibling tools or state when not to use it, so alternatives are not named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• docs_check_vulnerabilitiesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, and destructiveHint=false. The description adds substantial behavioral context beyond this: it performs a live query, never estimates risk, reports zero when OSV has no advisories, and gracefully handles README fetch failures by returning null rather than failing entirely. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (overview, args, return format, error handling). Every sentence contributes meaningful context—no fluff or repetition. Despite its length, it remains scannable and each element earns its place, making it appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, so the description fully explains the return JSON structure, including fields and types. It also covers error handling, edge cases (zero vulnerabilities, failed README fetch), and the live nature of the query. Given the tool's complexity (4 params, external API, conditional behavior), the description is comprehensive and leaves no major gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all four parameters are fully described in the schema. The description's Args section largely repeats the schema (e.g., defaults, package name exactness) without adding new meaning or constraints. It adds little value beyond what the schema already provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks a specific package version against OSV.dev for vulnerabilities and goes beyond a CVE list by fetching the fixed version's README. It distinguishes itself from siblings like docs_get_package_info and docs_search_docs by focusing specifically on vulnerability checking and remediation guidance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit context on when to use the tool: for a live vulnerability check on a real package version. It also clearly notes limitations compared to dedicated vulnerability-intelligence tools (VulnCheck, Snyk) and explains that zero vulnerabilities is a valid result, not an error. This gives clear usage boundaries and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• docs_resolve_libraryA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds detail beyond the readOnly/idempotent hints by specifying the exact live search endpoints (npmjs.com and crates.io) and explaining that PyPI calls return an explicit explanatory message rather than fabricated results. It also specifies the exact error condition ('Error: ...' only on real request failures) and the return JSON shape, which is not available from an output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear one-sentence purpose, then uses labeled sections (Args, Returns, Error Handling) to keep details scannable. Although it goes into depth on the pypi limitation, each sentence earns its place, and the structure prevents it from feeling bloated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, but the description fully specifies the return JSON structure (candidates, source_url, fetched_at) and covers error handling and ecosystem-specific behavior. It also provides a fallback suggestion for PyPI, making the tool's behavior predictable and complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage with descriptions for all three parameters, so the baseline is 3. The description adds concrete query examples ('react' or 'async http client'), the per-ecosystem search API mapping, and the note that max_results defaults to 5. This adds practical color, but the schema already captures the essential constraints, so a 4 rather than 5 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific, action-oriented statement ('Turn a fuzzy or partial name into real, ranked candidate package names, using the registry's own live search index'), which clearly defines the tool's output and approach. It also differentiates this from the sibling docs_get_package_info by naming it as the alternative for exact lookups, distinguishing this tool's search/resolution purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says when to use this tool (fuzzy or partial name → real candidates) and when not to: for PyPI it states there is no official search API and recommends using docs_get_package_info with the exact package name instead. It also clarifies per-ecosystem behavior, giving agents actionable context for choosing this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.