Server Quality Checklist

Tool Scores

• vault_dropC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only states 'delete a record' but does not mention permanence, side effects, required permissions, or consequences. The destructive nature is implied but no additional transparency is offered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It is highly concise and front-loaded. However, it may be under-specified for the tool's needs, but conciseness itself is not the issue.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple one-parameter operation, the description is minimal. It does not explain what happens after deletion (e.g., return value, success/failure behavior, or whether the operation is reversible). With no annotations or output schema, the description should provide more context, but it remains incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single 'id' parameter, so the schema already documents it as required and describes it as 'record id'. The description adds no additional meaning beyond that, which aligns with the baseline score of 3 for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '删除一条记录' clearly states the tool's purpose: deleting a record. It uses a specific verb ('delete') and resource ('record'). However, it does not explicitly differentiate from sibling tools like vault_take or vault_tidy, which might also remove or modify records, so it falls short of full distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention scenarios where vault_drop is preferred over vault_take or vault_tidy. There are no exclusions, prerequisites, or contextual cues, leaving the agent without decision-making information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vault_ingestB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, and the description fails to disclose key behaviors such as whether existing memories are overwritten, whether directory traversal is recursive, how duplicates are handled, or any formatting requirements. This is a mutation tool that modifies persistent state, so the lack of safety or side-effect disclosure leaves a significant transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler words. It directly states the action and target, achieving maximum efficiency for the content it provides.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that ingests files or directories, the description lacks essential context: expected file structure, recursion behavior, splitting semantics, error handling, and return behavior. With no output schema and minimal detail, an agent would be under-informed about how to invoke this safely and correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% because both parameters ('path' and 'split') have descriptions. The tool description adds no additional parameter context beyond the schema, so it meets the baseline for high coverage without adding extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'import' and the resource 'memories from markdown files or directories', which distinguishes it from sibling tools like vault_list or vault_ask. However, it doesn't explicitly name an alternative for contrast, so it falls slightly short of the highest tier.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for bulk imports from markdown sources, which contrasts with the single-item operations implied by vault_put or vault_take. However, no explicit guidance or exclusions are provided, so the agent must infer when to choose this over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vault_putA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does disclose a key behavior: automatic deduplication (merging identical or highly similar content into existing records). However, it does not explain what happens during the merge, whether the operation is reversible, or what the return value looks like, leaving gaps for a write operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the core purpose ('存入一条记忆记录') and then adds the deduplication behavior. Every word earns its place with no redundancy or unnecessary detail. This is an exemplary concise description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple store operation with no output schema and no annotations, the description gives a basic understanding but lacks important context such as the exact behavior during a merge, effects on existing records, or return format. Given the tool's five parameters and write nature, a bit more detail about deduplication and what the agent should expect would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents all five parameters (body, tags, title, source, weight). The description does not add any parameter-specific semantics beyond the schema, such as how deduplication interacts with tags or weight. This meets the baseline of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb '存入' (store/put) with the resource '记忆记录' (memory record), clearly indicating this tool stores a new record. It also mentions automatic deduplication, which distinguishes it from sibling tools like vault_ingest and vault_list. This is a clear and specific purpose statement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any explicit guidance on when to use this tool versus alternatives like vault_ingest or vault_take. It only states what the tool does, leaving the agent to infer usage context from the name and sibling set. No exclusions or alternative recommendations are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vault_listA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the behavioral trait of excluding collapsed content, which is useful. However, it does not explicitly state that the operation is read-only or safe, nor does it mention any side effects, permissions, or return format details. It is minimally transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, short sentence that is front-loaded with the primary action ('List recent records') and includes a valuable caveat ('excluding collapsed content'). Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with two well-documented parameters and no output schema, the description is reasonably complete. It communicates the core function and an important limitation. However, it does not describe the return structure or any pagination behavior beyond the parameter names, which is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents limit and offset with descriptions at 100% coverage. The tool description adds no additional parameter semantics beyond what the schema provides. Therefore, baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List recent records' with the specific verb 'list' and resource 'records', and adds a useful qualifier 'excluding collapsed content'. This distinguishes it from sibling tools like vault_put, vault_take, or vault_stats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention any exclusions or comparison to siblings like vault_ask or vault_stats. The only implied usage is that it lists records, but no explicit context or when-not-to-use is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vault_takeA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  As a read operation, the description implies no side effects, but it does not disclose behaviors like error handling for missing ids, authentication requirements, or rate limits. Without annotations, more could be expected, but the description does indicate the return is the 'complete content', which adds some context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short, front-loaded sentence that immediately conveys the purpose. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter read tool, the description is mostly complete. It indicates the result is the full record content. It doesn't cover error behavior (e.g., not found), but the low complexity makes this acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% as the only parameter 'id' is already described as 'record id (required)'. The description adds no new meaning beyond 'by id', so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'view the complete content of a single record by id' (按 id 查看单条记录的完整内容). The verb 'view' and resource 'single record by id' are specific, and it distinguishes from siblings like vault_list (listing records) and vault_ask (querying).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a single record's full content is needed by id, but it does not explicitly provide when-to-use vs alternatives or any exclusions. No mention of when to prefer vault_list or vault_ask instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vault_askA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the retrieval method (BM25 + semantic vector) and result ordering (relevance descending), which are useful behavioral traits. However, it does not explicitly state that the operation is read-only or discuss potential side effects, rate limits, or failure modes, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that immediately conveys the core mechanism and output ordering. Every word adds value, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two well-described parameters and no output schema, the description sufficiently explains the search algorithm and result ordering. It is complete for an agent to understand the tool's function without further elaboration.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage for both parameters (query and top_k) with clear descriptions. The description adds context about relevance sorting, which indirectly clarifies how top_k is used, but it does not provide additional parameter-level details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: it performs hybrid retrieval (BM25 + semantic vector) and returns records sorted by relevance. This clearly distinguishes it from sibling tools like vault_put (write) and vault_list (list all).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage is implied by the tool name 'ask' and the retrieval-focused description, but no explicit guidance is given about when to use it versus alternatives or when not to use it. The sibling context provides some signal but not direct instruction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vault_tidyA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It explicitly reveals a key side effect: original records are marked as folded, indicating they are not deleted. It does not mention reversibility, permissions, or return details, but the core state change is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the purpose and effect without any filler. Every word earns its place, making it highly concise and well structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description provides sufficient context: it explains the transformation and the fate of original records. It could mention the return value or when to use it, but given the low complexity, the description is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty input schema, so the baseline is 4. The description correctly implies that no inputs are needed and adds no conflicting or redundant parameter information; it fully covers the (non-existent) parameter requirements.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb (fold/compress) and a clear resource (mutually similar records), stating both the main action—folding similar records into a summary—and the side effect of marking originals as folded. This clearly differentiates it from sibling tools like vault_put, vault_list, or vault_drop.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when memory compression or deduplication is needed, but it does not explicitly state when to use this tool versus alternatives, nor does it mention any exclusions or prerequisites. The context is clear enough to infer the intended use, but explicit guidance is missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vault_statsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. The verb '查看' discloses that this is a read-only operation, and listing specific statistics (record count, vector backend, embedding config) gives the agent a clear picture of the output. It does not detail side effects or permissions, but for a zero-param stats tool, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the purpose ('查看存储统计') and includes relevant specifics. Every word contributes meaning; there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-param tool with no output schema, the description adequately covers the observable behavior. It lists three types of statistics and uses '等' (etc.) to hint at more, which is slightly open-ended but still gives enough context for an agent to understand the tool's function.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, and the schema confirms this. The description adds value by explaining what the tool returns without needing to describe parameter usage. Baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the explicit verb '查看' (view) and specifies the resource '存储统计' (storage stats), with examples like record count, vector backend, and embedding config. It clearly distinguishes this from sibling tools (vault_put, vault_take, etc.) which are mutation or retrieval operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use the tool (to view storage statistics) and provides concrete examples of the data shown. It does not explicitly mention alternatives or exclusions, but the scope is unambiguous enough for an agent to select it over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.