Server Quality Checklist

Tool Scores

• cag_contextC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits, but it only says 'Build CAG' without explaining side effects, output format, or any processing details. It gives a minimal hint of construction but fails to convey what the agent should expect regarding safety or results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly conveys the action and resource. It is appropriately concise with no wasted words. It could be slightly expanded to include usage context, but as written it is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of annotations and output schema, the description is incomplete. It tells what the tool does but does not explain what the generated CAG looks like, how depth affects the result, or any return format. An agent would lack critical invocation guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (both 'entity' and 'depth' have descriptions). The tool description adds no extra parameter meaning beyond what the schema provides, which aligns with the baseline 3 for high coverage. It does not compensate with additional context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Build') and resource ('BambiSleep knowledge graph'), and introduces the acronym CAG (Context Augmented Generation). It clearly differs from sibling tools like rag_query or semantic_search by emphasizing construction of context. However, it does not explain what CAG output entails, so it is clear but not fully precise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given for when to use this tool versus alternatives. There is no mention of prerequisites, scenarios, or exclusions. The description only states the action, leaving the agent to infer usage without support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rag_queryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'RAG' which implies generation, but does not disclose whether the tool is read-only, what the output format is, or any side effects (e.g., costs, latency). This is a significant gap for a query tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence without wasted words. However, it is bare-bones and does not preface key details like scope or result format, so slightly more structure could improve usefulness without sacrificing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so the description should explain return values and usage context. It does neither. Given the complexity of RAG (with generation) and the existence of a similar sibling tool, the description is incomplete for an agent to fully select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides descriptions for all three parameters, giving 100% coverage. The description adds no extra meaning beyond the schema, so it relies on the baseline for high coverage. No additional semantic insight is offered.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Perform RAG query') and the resource ('BambiSleep data'), making it distinct from generic queries. However, it does not explicitly distinguish itself from the sibling tool 'semantic_search', which could cause confusion about which to use.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like semantic_search, or when not to use it. The description simply states what it does without any contextual direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits, but it only states the high-level action. It does not explain whether the operation is read-only, what the return format is, how the threshold affects results, or handling of edge cases like low similarity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no redundant information. It is efficiently structured and front-loaded with the key action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations, no output schema, and the existence of closely related sibling tools, this description is insufficient. It does not explain how semantic search differs from RAG querying, what practical effect the threshold has, or any operational nuances, leaving the tool contextually incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters (query and threshold) have descriptions in the schema, covering 100% of parameters. The description text adds no extra semantic detail beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'perform semantic search' and identifies the resource 'BambiSleep content', making the core function clear. However, it does not differentiate from the sibling tool 'rag_query', which may serve a similar retrieval purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives like rag_query or cag_context. The description simply states the action without context, exclusions, or preferred scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_entitiesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the action but does not disclose any behavioral traits such as output format, side effects, or required permissions, leaving the agent with only the bare function. This is insufficient for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence with no wasted words, front-loading the action and purpose. Every word earns its place, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has one documented parameter, but the description omits how the 'section' parameter affects extraction and does not specify the output format beyond 'entities and relationships'. With no output schema, this gap in return value detail reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully documents the single 'section' parameter with enum values and a description, so schema description coverage is 100%. The description adds no additional parameter semantics beyond what the schema already provides, meeting the baseline but not exceeding it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'extract' and identifies the resource 'entities and relationships' from 'BambiSleep content', clearly distinguishing it from the sibling search/query tools. It clearly states the tool's function and scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or alternatives are provided. The implied use case is when structured entity/relationship extraction is needed, but the description does not contrast with sibling tools like rag_query or semantic_search, leaving usage guidance implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_metadataA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description is the only source of behavioral disclosure. It only says 'Get metadata' but does not mention read-only nature, return format, potential cost, or any other behavioral traits, which is a gap even for a simple read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that wastes no words. It clearly conveys the tool's purpose without any filler, fitting the conciseness ideal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (no params, no output schema) and the description adequately defines its scope relative to siblings. While it could list example metadata items, the minimal complexity keeps this from being a major gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds no parameter semantics (as there are none), but it clarifies the target resource ('BambiSleep knowledge base'), which is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves metadata about the BambiSleep knowledge base, using a specific verb ('get') and resource ('metadata'). This distinguishes it from sibling tools that handle queries, search, or entity extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There are no mentions of use cases, exclusions, or comparisons with sibling tools, leaving the agent without decision-making context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.