Server Quality Checklist

Tool Scores

• semantic_searchD
  1/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no behavioral traits, side effects, or operational details disclosed because the description contains no information about the tool at all. With zero annotations and zero meaningful description, the burden was entirely on the description, and it provides nothing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness1/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While one could argue a single directive is 'concise,' this is not conciseness—it is tragic under-specification. Every word is spent restating the evaluation prompt rather than defining a tool. There is no structure, no front-loading, and no useful information conveyed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides zero of the context needed to use the tool. Even for a hypothetical 0-arg operation this would be inadequate, but for an undefined operation it is entirely hopeless. The agent has no way to proceed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters are defined anywhere in the input. The framework itself notes that 0 parameters means a baseline of 4, but in this case the absence of parameters is not the same as a well-understood 0-arg tool—the input is entirely empty of semantic content. There is no schema for the description to supplement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose1/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description is a verbatim copy of the evaluation framework's own instructions—there is no tool name, title, or purpose stated anywhere. It is entirely impossible to determine what tool this describes, let alone distinguish it from siblings. This is a complete failure of communication.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance whatsoever on when to use this tool vs. alternatives. The text is meta-instruction about how to reply to the original prompt, not about the tool itself. No context, no exclusions, nothing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_recent_memoriesA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states the primary action and parameters, without revealing whether the operation is read-only, any authorization requirements, result ordering specifics, or error behavior. This is minimal transparency for a retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a single-sentence purpose statement followed by a formatted Args list. It is front-loading the main functionality and avoids any fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose and parameters, but given the absence of annotations, output schema, and usage guidance, it is not fully complete. It lacks details about return format, ordering, edge cases, or interactions with sibling tools, leaving some room for ambiguity in invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no descriptions for parameters (coverage 0%), but the description thoroughly explains each parameter: limit with range and default, user_id as a restriction, and agent_id as a restriction. This adds meaningful context beyond the schema's type/default definitions, fully compensating for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Return the most recently created memories.' This specifies the verb (return), resource (memories), and scope (most recently created), which distinguishes it from sibling tools like search_memories or recall that likely perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving recent memories but provides no explicit guidance on when to choose this tool over alternatives like search_memories or hybrid_search. There are no references to other tools or exclusions, leaving the agent to infer the appropriate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recallA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description is the sole source of behavioral info. It discloses the three-layer approach, context merging, and ranking, but does not explicitly state that it is read-only, describe the output format, or mention any side effects or limitations. This is a moderate level of transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a brief intro followed by a clear argument list. Every sentence adds value, and it avoids unnecessary verbosity. It front-loads the core purpose and then details parameters efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks details about the return value or output structure, which is important for a retrieval tool. It also omits potential edge cases or evaluation criteria. While the tool is not highly complex, the absence of output information leaves a notable gap in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description gives meaningful explanations for all four parameters (query, user_id, agent_id, limit), including purpose and default for limit. This fully compensates for the 0% schema description coverage, providing essential context beyond the schema's bare titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Three-layer memory retrieval' and explains its value over plain keyword search. However, it does not explicitly distinguish itself from sibling search tools like search_memories or semantic_search, so it lacks direct differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides guidance on when to use it ('Better than plain keyword search for "what do I remember about X"') and highlights its unique features (merging session context, ranked relevance). Yet it does not mention when not to use it or name alternative tools as options.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_memoryA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states the permanent destructive nature of the action ('permanent' and 'vanish'), which is critical transparency. However, it does not mention any side effects, permissions, or confirmation requirements beyond the description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise — two sentences with zero waste. The first sentence states the action and permanence, the second provides the alternative, and the args block is clearly structured. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 parameter, no output schema, no nested objects), the description is largely complete. It covers the action, permanence, and alternative. The only minor gap is that it doesn't explicitly describe return values or error behavior, but given the minimal complexity and lack of output schema, this is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the lack of parameter details. It does so by providing a concise explanation of the `memory_id` parameter: 'The id of the memory to delete (required).' This adds a clear, actionable meaning beyond the schema's bare 'Memory Id' title, though it could be slightly more descriptive about how to obtain this id.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Delete a memory by its id' — a specific verb and resource. It distinguishes itself from other memory-related tools by mentioning an alternative for non-permanent removal, which helps differentiate from recall/search/store tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides guidance on when to use this tool versus an alternative: 'permanent — use forget/apply via the REST API for reversible archiving instead if you just want it to fade, not vanish.' This clearly communicates the intended use case and offers a specific alternative for a different scenario.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_memoriesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the burden. It discloses key behaviors: case-insensitive substring matching, optional filters, and default limit. It does not mention output structure or edge cases, but covers the core behavioral contract.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a one-line summary followed by a clear Args block. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters, no output schema, and no annotations, the description handles parameters and behavior well. It omits return value format, but for a search tool the result is implied to be memories; minor gap but acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description provides a full Args section explaining each parameter (query, user_id, agent_id, type, limit) with defaults and constraints, fully compensating for the absence of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs keyword search (case-insensitive substring match) on memories, with a specific verb and resource. It distinguishes itself from semantic/hybrid search siblings by emphasizing substring matching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It implies usage for substring-based retrieval and lists filter parameters, but does not explicitly state when to choose this over sibling tools like semantic_search or hybrid_search, nor when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• store_memoryA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does an excellent job: it explains that the embedding parameter is deprecated and ignored, that the server computes its own embedding, and that memories are scoped by agent_id. This goes beyond the schema and provides critical behavioral context for the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear opening line and a bulleted parameter list. It is slightly verbose due to the detailed parameter explanations, but every sentence adds value, especially the deprecation note and scoping explanation. It is appropriately sized for an 8-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 parameters, no annotations, no output schema), the description is remarkably complete. It covers all parameters, explains scoping, deprecation, and defaults, and provides examples. The agent has everything it needs to invoke the tool correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, and it does thoroughly. Every parameter is explained with examples and constraints (e.g., importance range, agent_id examples, type categories, embedding deprecation). This adds significant meaning beyond the raw schema, which only provides types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Store a new memory in HCC.' It uses a specific verb ('store') and resource ('memory in HCC'), and the detailed parameter explanations distinguish it from sibling tools like search_memories, recall, and delete_memory, which are all about retrieval or deletion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool (to store a new memory) and explains the scoping behavior of agent_id, which is crucial for understanding how memories are isolated. However, it does not explicitly mention when not to use it or name alternative tools for similar operations, though the sibling list makes that implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evaluateA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does disclose that the tool returns a value indicating whether to store (implied by 'should_store comes back true') and notes that agent_id is for future tuning. However, it does not explicitly state the exact return format or any side effects, though the implied boolean and purpose are reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the purpose, followed by a clear usage guideline and parameter list. Every sentence contributes meaning, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three parameters and no output schema, the description is fairly comprehensive. It explains the tool's role, when to use it, and parameter semantics. The return type is implied but not explicitly stated, and it lacks details on error handling or edge cases, so it's not fully complete for a production tool without annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only types and defaults, with zero descriptions. The description's Args section explains each parameter's meaning (content as text to evaluate, agent_id as the asking agent for tuning, user_id as whose content this is), significantly adding value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: to ask HCC's orchestrator whether content is worth remembering before storing it. It uses a specific verb (ask) and resource (orchestrator) and distinguishes itself from siblings by explicitly positioning it as a pre-store gate for store_memory.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to use this tool ('only call store_memory for content where should_store comes back true') and when to override, including a concrete example (user says 'remember this'). This provides clear guidance on using it versus the alternative store_memory.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hybrid_searchA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden—and it delivers: it discloses server-side embedding via ollama, jieba segmentation for BM25, RRF fusion, rerank latency and silent fallback to RRF order when the reranker is unavailable, and the requirement to provide at least one of query/embedding. These behavioral details go well beyond the schema and give the agent a solid model of the tool's execution.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized: a two-sentence overview of the algorithm and use case, then a structured Args block. Every sentence conveys essential information—no filler or repetition. The length is justified by the complexity of a hybrid retrieval tool with optional reranking.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (dual retrieval branches, fusion, optional reranking) and the absence of annotations and output schema, the description provides robust coverage: algorithm, default behavior, server-side processing, latency implications, fallback semantics, and parameter constraints. The only omission is the exact return shape, but for a search tool this is acceptable and the description is otherwise complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by explaining every parameter: the dual role of query, the advanced/optional nature of embedding, the restriction semantics of user_id/agent_id/type, the limit's range and default, and the rerank flag's latency and fallback. This is complete, actionable parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise definition: 'Hybrid search: BM25 full-text + vector similarity, fused with Reciprocal Rank Fusion.' It then identifies the tool as the 'Best default choice for "find memories about X"', clearly distinguishing it from siblings like semantic_search or search_memories by combining exact keyword and semantic matching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The tool is explicitly positioned as the 'best default choice' for memory retrieval, with an example of when hybrid search shines (different wording like '显卡' vs 'GPU'). It explains when to provide query vs embedding and mentions the optional rerank behavior, but does not explicitly name sibling alternatives or state when not to use this tool, making the guidance clear but not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.