Server Quality Checklist

Tool Scores

• call_toolA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not mention potential side effects, required permissions, or the fact that this tool may execute arbitrary actions. This lack of transparency is risky for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no redundant information. The description is efficient and to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the basic purpose and links to search_tools, but it lacks details on error handling, return values, or behavioral constraints. Given the tool's generic nature, this is acceptable but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only the 'name' parameter has a description in the schema; 'arguments' is not described. The description adds minimal value by implying arguments are passed, but does not clarify structure or constraints, leaving significant ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (call a tool) and the target (by name with arguments), and it is distinct from the sibling tools which perform specific operations like remember or recall.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to use this for executing tools discovered via search_tools, giving a clear use case. It does not elaborate on when not to use it, but the guidance is sufficient for typical scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_toolsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It states that it returns matching tool definitions ranked by relevance, which is a behavioral trait. It also mentions the format is same as list_tools, which is useful. However, it doesn't disclose any side effects, rate limits, or other behavioral details. Given the tool is a search operation, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, two sentences, and front-loaded with the purpose. Every sentence adds value: the first states what it does, the second clarifies the return format. No waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and an output schema. The description explains the return format (same as list_tools) and ranking by relevance. Given the simplicity and the presence of an output schema, the description is complete enough. It could mention that it's a read-only operation, but that's not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter 'query' as a natural language query. The description adds no additional meaning beyond that. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: searching for tools using natural language. It specifies the action (search) and the resource (tools), and distinguishes it from siblings like list_tools by mentioning the return format. However, it doesn't explicitly differentiate from other sibling tools like call_tool, but the purpose is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: when you need to find tools by natural language query. It mentions the return format is same as list_tools, which gives some context. However, it doesn't explicitly state when to use this vs alternatives, nor does it provide exclusions or alternative tool references. The guidance is minimal but not misleading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forgetA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full disclosure burden. It does disclose the destructive scope and that data is removed from relational, graph, and vector databases, which is useful context beyond the schema. However, it omits important behavioral traits such as whether deletion is permanent/recoverable, whether permissions are required, and what happens if no targeting argument is supplied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short, front-loaded sentences plus a concise scope bullet. Every phrase adds value, with no repetition of schema fields or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has four optional parameters and no annotations, so a robust description should clarify edge cases like no-argument invocation, argument conflicts, and irreversible consequences. The existence of an output schema reduces the need to describe return values, but significant semantic guidance is still missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description paraphrases the targeting options (single item, dataset by name or id, everything) but adds no new parameter-specific semantics; for example, it does not clarify precedence or mutual exclusivity beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence 'Delete data from memory' uses a specific verb and resource, and the scope bullet (single item, dataset, or everything) clearly distinguishes it from siblings like remember, recall, and search_tools. It also states the underlying stores affected, leaving no doubt about the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this is the deletion counterpart to remember and is appropriate whenever stored data must be removed. However, it does not explicitly state when not to use it, mention alternatives for non-destructive operations, or give guidance about argument exclusivity and fallback behavior.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recallA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the disclosure burden and it does reveal non-obvious behavior: session-first keyword matching, fallback to the permanent knowledge graph, and automatic search-strategy selection. It does not discuss side effects, permissions, or rate limits, but for a search action the main routing behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tight and front-loaded, with the main purpose in the first sentence and exactly two supporting details about routing/fallback. No filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and thorough parameter documentation, the description covers the non-obvious routing behavior needed to understand the tool. It lacks explicit guidance about edge cases or alternatives, but the combination of description plus schema is adequate for confident invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema coverage is 100%, so the schema already documents all six parameters; the tool description adds useful conditional context around session_id, datasets, and search_type but does not explain query, top_k, dataset syntax, or system_prompt beyond the schema. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource ('Search memory') and adds distinguishing behavioral qualifiers ('auto-routing and session awareness'), which separates it from the write/remove siblings (remember, forget). Subsequent sentences clarify the scope with session-cache and knowledge-graph behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear invocation context: when session_id is supplied without datasets/search_type it searches the session cache first and falls through to the knowledge graph, and auto-routing applies when search_type is omitted. It does not explicitly name alternative tools or exclusion scenarios, but the parameter-condition guidance is sufficient for most recall usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rememberA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of disclosing behavior. It explains the pipeline variants, performance characteristics, error handling (errors via cognify_status), and constraints on file uploads. This is thorough for a memory storage tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections for modes and input constraints, but slightly verbose. It could be tightened while retaining key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (two modes, multiple parameters, file uploads) and the existence of an output schema, the description covers the essential nuances thoroughly. Minor omissions like return format are handled by the output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents parameters. The description adds value by explaining mode interplay (e.g., background ignored with session_id) and mutual exclusivity, but many details are already in schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool stores data in memory and distinguishes two modes (permanent vs session memory) based on session_id. It explicitly contrasts with sibling tools like recall and forget.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use each mode, when to omit session_id, and constraints on data vs file uploads (mutually exclusive). It clarifies that file uploads are permanent-only and that background mode is for avoiding deadline issues.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.