Server Quality Checklist

Tool Scores

• update_memoryB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It states 'overwrite' which implies mutation, but doesn't disclose critical behavioral traits: whether this is destructive (irreversible), requires specific permissions, has rate limits, or what happens on success/failure. The description is minimal and lacks operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it immediately scannable and appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a mutation tool with no annotations but a complete input schema and an output schema (implied by 'Has output schema: true'), the description is minimally adequate. It covers the basic purpose but lacks guidance, behavioral context, and doesn't leverage the output schema to hint at return values. It meets the bare minimum for a simple update operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are fully documented in the schema. The description adds no additional meaning beyond implying 'memory_id' identifies the target and 'text' is the replacement content, which is already clear from schema descriptions. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('overwrite') and resource ('existing memory's text'), making the purpose immediately understandable. It distinguishes from 'add_memory' (create new) and 'delete_memory' (remove), but doesn't explicitly differentiate from potential update-like siblings beyond the overwrite semantics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'add_memory' for new memories or 'get_memory' for retrieval. The description implies it's for existing memories but doesn't specify prerequisites (e.g., memory must exist) or exclusions (e.g., not for partial updates).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_memoryB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers limited behavioral insight. It states it 'fetches' a memory, implying a read operation, but doesn't disclose permissions, rate limits, error conditions, or what happens if the memory_id is invalid. The description adds little beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and key constraint ('once you know its memory_id'), making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (single required parameter) and the presence of an output schema (which handles return values), the description is reasonably complete. It covers the basic purpose and key usage hint, though more behavioral context would be beneficial given the lack of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the single parameter 'memory_id'. The description adds marginal value by emphasizing 'Exact' and 'once you know its memory_id', reinforcing it's for precise lookup, but doesn't provide additional syntax or format details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Fetch') and resource ('a single memory'), making the purpose understandable. It distinguishes from siblings like 'get_memories' (plural) by specifying 'single', but doesn't explicitly contrast with other retrieval tools like 'search_memories'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal guidance: 'once you know its memory_id' implies this tool is for direct lookup rather than discovery. However, it doesn't explain when to use this versus alternatives like 'get_memories' (bulk retrieval) or 'search_memories' (query-based), nor does it mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_memoryB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the deletion action and a confirmation step, but does not disclose critical behavioral traits such as whether the deletion is permanent, what permissions are required, error handling, or what the output schema returns. For a destructive tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently conveys the core action and condition without unnecessary words. It is front-loaded with the key information ('Delete one memory') and structured appropriately for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (destructive operation with one parameter) and the presence of an output schema (which handles return values), the description is minimally adequate. However, with no annotations and incomplete behavioral disclosure, it fails to fully address the tool's context, such as safety considerations or integration with sibling tools. It meets basic needs but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'memory_id' well-documented in the schema as 'Exact memory_id to delete.' The description adds minimal value by reiterating the need for 'memory_id' but does not provide additional context like format examples or validation rules. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and resource ('one memory'), making the purpose understandable. It specifies 'one memory' which distinguishes it from 'delete_all_memories', though it doesn't explicitly contrast with other siblings like 'delete_entities' or 'update_memory'. The description is specific but could be more precise about differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating 'after the user confirms its memory_id', suggesting this tool should be used when the user has identified a specific memory to delete. However, it does not provide explicit guidance on when to use this versus alternatives like 'delete_all_memories' or 'delete_entities', nor does it mention prerequisites or exclusions. The guidance is implied but incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_entitiesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the destructive nature ('Remove entirely') and cascade behavior, which is essential. However, it lacks details on permissions required, whether deletion is reversible, rate limits, error conditions, or what the output contains. For a high-risk deletion tool, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action and key behavioral detail (cascade-delete). Every word earns its place with zero waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the high-risk nature of this tool (permanent deletion with cascading effects), no annotations, and an output schema (which may cover return values), the description is minimally adequate. It covers the what and cascade effect but misses critical context like permissions, irreversibility, and error handling that an agent needs for safe invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter clearly documented in the schema itself. The description adds no additional parameter semantics beyond implying that exactly one of the four IDs should be provided (though not explicitly stated). Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Remove') and specifies the resources ('user/agent/app/run record entirely'), distinguishing it from sibling tools like delete_memory or delete_all_memories. It adds the critical detail of cascade-deleting memories, which is not obvious from the tool name alone.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for permanent deletion of entities with their memories, but does not explicitly state when to use this versus alternatives like delete_memory (for individual memories) or delete_all_memories (for bulk memory deletion without entity removal). No guidance on prerequisites, permissions, or recovery options is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_all_memoriesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While it states the destructive action ('Delete every memory'), it doesn't mention critical behavioral aspects like whether this operation is reversible, what permissions are required, confirmation requirements, rate limits, or what the output contains. The description is insufficient for a destructive bulk operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise - a single sentence that front-loads the core action and immediately clarifies what is preserved. Every word earns its place with zero redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a destructive bulk operation with no annotations but with an output schema, the description is moderately complete. It clearly states the action and scope but lacks important behavioral context about safety, permissions, and consequences. The existence of an output schema reduces the need to describe return values, but more operational guidance is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3. The description adds minimal value beyond the schema by implying these parameters define the scope ('in the given user/agent/app/run'), but doesn't explain parameter interactions, precedence, or what happens when multiple scopes are specified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Delete every memory') and resource ('in the given user/agent/app/run'), with explicit differentiation from sibling tools by specifying what is preserved ('but keep the entity'). This distinguishes it from delete_entities and delete_memory tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool - for bulk deletion of memories while preserving entities. However, it doesn't explicitly state when NOT to use it or name specific alternatives like delete_memory for individual deletions or delete_entities for entity removal.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_entitiesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It describes a read operation ('List') but does not disclose behavioral traits like whether it requires authentication, rate limits, pagination, or what the output format looks like, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose without unnecessary words, making it appropriately sized and zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple list operation), no annotations, and an output schema (which handles return values), the description is minimally adequate. However, it lacks details on behavioral context like permissions or output structure, leaving room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema coverage, so no parameter documentation is needed. The description does not add param info beyond the schema, but this is acceptable given the lack of parameters, warranting a baseline score above 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List') and the resource ('which users/agents/apps/runs currently hold memories'), distinguishing it from siblings like get_memories or search_memories that focus on memory content rather than entity holders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for identifying memory holders, but does not explicitly state when to use this tool versus alternatives like get_memories or delete_entities, nor does it provide exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_memoryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the storage action and a key requirement (needing at least one identifier), but doesn't address important behavioral aspects like whether this operation is idempotent, what happens on duplicate entries, permission requirements, rate limits, or error conditions. The description adds some value but leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise - two sentences that each earn their place. The first sentence states the core purpose, and the second provides a critical constraint. There's zero wasted language, and the most important information (the requirement) is appropriately placed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (8 parameters, write operation) and the presence of an output schema, the description provides adequate context. It covers the core purpose and a critical constraint. The output schema means the description doesn't need to explain return values. However, for a write operation with no annotations, it could benefit from more behavioral context about side effects or error handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the schema already documents all 8 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema - it mentions the identifier requirement but doesn't explain parameter interactions or provide usage examples. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'store' and the resource 'new preference, fact, or conversation snippet', making the purpose specific and actionable. It distinguishes from siblings like delete_memory or get_memories by focusing on creation rather than retrieval or deletion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('store a new...') and includes a critical requirement ('requires at least one: user_id, agent_id, or run_id'), which helps guide proper invocation. However, it doesn't explicitly mention when to use alternatives like update_memory or how this differs from other storage operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_memoriesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: the semantic search nature, automatic user_id injection into filters, and the enable_graph parameter's specific usage condition. It doesn't cover rate limits, authentication needs, or pagination, but provides substantial operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose. The filter examples are valuable but slightly lengthy; every sentence earns its place by providing practical guidance, though it could be slightly more streamlined.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (semantic search with filters), no annotations, and the presence of an output schema, the description is complete enough. It covers purpose, usage, key behaviors, and parameter semantics thoroughly, making it self-sufficient for an agent to understand and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds significant value beyond the schema by explaining filter patterns with concrete examples, clarifying that user_id is automatically added, and providing context for enable_graph usage. This elevates the score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Run a semantic search') and resource ('over existing memories'), distinguishing it from siblings like get_memories (likely a simpler retrieval) and add_memory/update_memory/delete_memory (write operations). It establishes this as a search-focused tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (semantic search with filters) and includes practical filter examples for common scenarios. However, it doesn't explicitly state when NOT to use it or name specific alternatives among siblings (e.g., get_memories for unfiltered retrieval), which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_memoriesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: automatic user_id injection, pagination mechanics (1-indexed), default page_size, and the distinction between filtering vs. searching. However, it doesn't mention potential rate limits, authentication requirements, or error conditions, leaving some behavioral aspects uncovered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose. Every sentence adds value: the initial statement of purpose, the usage guidance with examples, and the behavioral notes about pagination and automatic filtering. There's no wasted text, and the information is presented in a logical, scannable format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, filtering logic) and the presence of an output schema, the description provides excellent contextual completeness. It covers the purpose, differentiation from siblings, parameter usage with examples, and key behavioral aspects. The output schema will handle return values, so the description appropriately focuses on usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds significant value beyond the schema by providing concrete filter examples with JSON syntax, explaining the automatic user_id injection behavior, and clarifying pagination usage. This gives practical context that the schema alone doesn't provide, warranting a higher score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Page through memories using filters instead of search.' It specifies the verb ('Page through') and resource ('memories'), and explicitly distinguishes it from the sibling 'search_memories' tool by mentioning 'instead of search.' This provides excellent differentiation from alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool versus alternatives: 'Use filters to list specific memories' and 'instead of search.' It also offers practical examples of common filter patterns, helping the agent understand appropriate use cases. The mention of pagination parameters further clarifies operational context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.