Server Quality Checklist

Tool Scores

• get_prerequisitesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses matching behavior (case-insensitive, partial name resolution), output format with examples, and edge cases (root concept, not found). Lacks mention of side effects or rate limits, but these are not critical for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with Args and Returns sections, and front-loaded with purpose. Somewhat verbose but every sentence adds value, so it maintains clarity without being overly long.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters) and lack of output schema, the description is remarkably complete. It explains purpose, usage context, parameter behavior, return format with examples, and edge cases. No significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description adds significant meaning. For 'domain' it specifies to use exact name from list_domains; for 'concept' it details matching behavior and partial resolution. This compensates well for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Return the full ordered chain of concepts to understand before a target concept.' It specifies the verb, resource, and scope, and distinguishes from siblings by directing to alternative tools for different tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'for onboarding, gap-filling, or sequencing study.' Also provides clear exclusions: for two-directional neighborhood use query_ckg, for exact name resolution use search_concepts. This provides excellent guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_domainsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses no arguments, describes return format with example, and implies read-only behavior, fully adequate for a list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences front-loading purpose, usage, and output. Every sentence adds value with zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-argument list tool with output schema, description provides complete context: what it returns with example. No gaps given simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist (0 params, 100% coverage). Description states 'Takes no arguments' which adds value beyond schema. Baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists every CKG domain, using specific verb 'list' and resource 'domain'. It distinguishes from siblings (query_ckg, get_prerequisites, search_concepts) by noting they require a domain argument.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Call this FIRST, before the other tools' and explains that the domain required by siblings must be an exact name returned here, providing clear when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_ckgA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Though no annotations are provided, the description details key behaviors: case-insensitive matching, partial name resolution, depth limits (upstream 1-5, downstream fixed 2 hops), return format (Markdown report with two trees and taxonomy tag), and error handling (similar names listed if not found).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a succinct summary, usage guidance, parameter details, and return format. Each sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (dependency subgraph), zero schema coverage, and no annotations, the description covers all essential aspects: purpose, parameters, behavior, return format, error handling, and differentiation from siblings. The output schema exists but the description adds necessary context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing clear meanings for each parameter: domain (exact name from list_domains), concept (case-insensitive partial match), and depth (default 3, range 1-5). It also explains how concept matching works.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the dependency subgraph around a concept, specifying both prerequisites and downstream concepts. It distinguishes itself from the sibling tool 'get_prerequisites' by noting that the latter is for only upstream chains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises when to use this tool ('local neighborhood of a concept') and when not to ('For ONLY the upstream prerequisite chain, use get_prerequisites instead'). It also suggests calling 'search_concepts' if the concept name is uncertain.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_conceptsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so description carries full burden. It fully explains behavior: case-insensitive substring match, return up to 20 results with taxonomy tags, or a 'no concepts' message. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with summary first, then parameter details and return description. Approximately 100 words with no fluff, earning each sentence's presence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complete given context: explains behavior, parameters, return format, and usage context relative to siblings. Lack of error handling details is acceptable for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has no descriptions (0% coverage), but description provides full semantics: domain is exact name from list_domains, query is substring with examples. Return format is also detailed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds concepts in a domain by partial name, with a specific verb and resource. It distinguishes itself from siblings by advising use before query_ckg or get_prerequisites when exact labels are unknown.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use (before query_ckg/get_prerequisites when label is unknown) and describes the behavior (case-insensitive substring match). Implicitly suggests alternatives for when label is known.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.