Server Quality Checklist

Tool Scores

• get_usageB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'check current memory usage' which implies a read-only operation, but doesn't specify if it requires authentication, has rate limits, or what happens if quotas are exceeded. This leaves gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that are front-loaded and efficient. The first sentence states the purpose, and the second adds context without redundancy, though it could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema), the description is adequate but has gaps. It explains what data is retrieved but doesn't cover behavioral aspects like authentication or error handling, which are important for a monitoring tool in a server with siblings like delete_memory.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description adds value by explaining what information is retrieved ('plan, limits, and how many memories are stored'), which compensates for the lack of output schema, making it above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('check current memory usage') and resources ('plan, limits, memories stored'), distinguishing it from siblings like delete_memory or store_memory. However, it doesn't explicitly differentiate from get_user_context or list_memories, which might also involve memory-related queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance with 'useful for monitoring quota,' suggesting when to use it. However, it lacks explicit when-not-to-use criteria or alternatives among siblings, such as clarifying when to use list_memories instead for detailed memory content.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_memoriesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions visibility status and use cases but fails to cover critical aspects like pagination behavior (implied by limit/offset but not explained), authentication needs, rate limits, or error handling. For a read operation with multiple parameters, this leaves significant gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently combines purpose and usage guidelines without redundancy. Every part earns its place, making it appropriately sized and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (6 parameters, no output schema, no annotations), the description is adequate but incomplete. It covers purpose and usage well but lacks behavioral details (e.g., pagination, auth) and doesn't explain return values, leaving gaps that could hinder an agent's effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description adds minimal value beyond the schema by implying visibility filtering but doesn't provide additional syntax or format details. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as listing stored memories for a user with visibility status, which is specific and distinguishes it from siblings like 'search_memories' (filtered search) or 'get_user_context' (context retrieval). However, it doesn't explicitly contrast with all siblings (e.g., 'get_usage'), keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage contexts: when the user wants to see what's remembered, review team vs private memories, or find a memory to update/delete. This gives practical guidance but lacks explicit exclusions (e.g., when not to use it vs. 'search_memories') or named alternatives, preventing a score of 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_memoryA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it indicates this is a destructive operation ('Delete'), it lacks crucial details such as whether deletion is permanent, requires specific permissions, has side effects (e.g., cascading deletions), or provides confirmation feedback. For a mutation tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences that directly address purpose and usage. Every word earns its place, with no redundant information or fluff, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a destructive mutation tool with no annotations and no output schema, the description is incomplete. It doesn't address critical behavioral aspects like permanence, permissions, error handling, or what happens post-deletion. For a tool that permanently removes data, more context is needed to ensure safe and correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'id' fully documented in the schema as 'The memory ID to delete'. The description adds no additional parameter semantics beyond what the schema provides, so it meets the baseline for high schema coverage without adding value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Delete') and resource ('a specific memory by ID'), distinguishing it from sibling tools like 'list_memories', 'store_memory', or 'update_memory'. It precisely communicates the tool's function without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool ('when a user asks to forget something'), which helps differentiate it from read-only siblings like 'list_memories' or 'search_memories'. However, it doesn't explicitly state when NOT to use it or mention alternatives like 'update_memory' for modifying instead of deleting.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_memoryA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it implies a mutation operation ('Update'), it does not disclose critical behavioral traits such as required permissions, whether changes are reversible, error handling, or rate limits. For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding how the tool behaves beyond its basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by usage guidance and prerequisites in subsequent sentences. Each sentence earns its place by adding necessary context without redundancy, making it efficient and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity as a mutation with no annotations and no output schema, the description is adequate but incomplete. It covers the purpose, usage, and parameters indirectly, but lacks details on behavioral aspects like permissions, side effects, or return values. For a tool that modifies data, more contextual information would be beneficial to ensure safe and correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema by mentioning the parameters in context ('visibility (private/team) or importance'), but does not provide additional semantics like examples or edge cases. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Update a memory's visibility or importance') and the resource ('memory'), distinguishing it from siblings like delete_memory, store_memory, list_memories, and search_memories. It specifies exactly what can be modified (visibility and importance), making the purpose unambiguous and distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool ('when a user wants to make a memory private, share it with the team, or change its importance') and references sibling tools for prerequisites ('Find the memory ID via search_memories or list_memories first'). However, it does not explicitly state when NOT to use it or mention alternatives like store_memory for creating new memories, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_user_contextA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing key behavioral traits: the ranking method ('Ranked by Temporal Memory Scoring'), the optional query behavior, team memory inclusion logic, and the formatted output purpose. It doesn't mention rate limits, authentication needs, or error conditions, but provides substantial operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with zero waste - each sentence adds important information about functionality, usage patterns, and team behavior. The description is appropriately sized and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 6-parameter tool with no annotations and no output schema, the description provides good context about what the tool does and how to use it. It explains the formatted output purpose and team memory behavior. However, it doesn't describe the return format or structure, which would be helpful given the lack of output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds some value by explaining the query parameter's purpose ('to focus the context on a topic') and team inclusion behavior, but doesn't provide additional semantic context beyond what the schema already documents for most parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('retrieve a pre-formatted context string'), resource ('for a user'), and purpose ('ready to inject into an LLM system prompt'). It distinguishes from siblings like list_memories or search_memories by focusing on formatted context retrieval rather than raw memory operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage ('Can be called without a query to get the most important recent memories') and mentions team configuration behavior. However, it doesn't explicitly state when to use this tool versus alternatives like search_memories or list_memories, which could help differentiate more clearly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_memoriesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and adds valuable behavioral context: it explains the ranking algorithm ('balances semantic relevance with memory freshness and importance'), specifies default team memory inclusion behavior, and mentions configuration dependencies ('when a team is configured'). It doesn't cover error conditions, rate limits, or authentication needs, but provides substantial operational insight beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with zero waste: first states core purpose, second explains ranking methodology, third clarifies team behavior. Each sentence adds distinct value, and the description is appropriately front-loaded with the main functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 6 parameters, 100% schema coverage, and no output schema, the description provides good context about ranking methodology and team behavior. It doesn't describe return format or result structure, which would be helpful given no output schema, but covers the essential operational context well for a read-only search operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description adds minimal parameter-specific information beyond what's in the schema - it mentions team configuration defaults but doesn't elaborate on parameter interactions or search semantics. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs 'semantic search across stored memories for a user' with specific ranking methodology ('Temporal Memory Scoring'), distinguishing it from siblings like list_memories (which likely lists without search) or get_user_context (which might retrieve context without search). It specifies both the action (search) and resource (memories) precisely.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for semantic search of memories. It mentions default behavior for teams ('returns both private and team memories by default'), which helps differentiate from list_memories (which might not include team memories). However, it doesn't explicitly state when NOT to use it or name specific alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• store_memoryA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: the tool stores data, handles deduplication and contradiction detection automatically, and explains the impact of importance scores on decay rates. It does not cover aspects like error conditions, performance, or rate limits, but provides substantial context for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two sentences that efficiently convey core functionality and usage guidance. Every sentence adds value without redundancy, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity as a mutation tool with 6 parameters and no annotations or output schema, the description is largely complete. It covers purpose, behavioral traits, and usage guidelines, but could benefit from mentioning potential side effects, error handling, or return values to fully compensate for the lack of structured data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds some semantic context by explaining the importance parameter's practical usage (e.g., trivia vs. critical preferences), but does not provide additional meaning for other parameters beyond what the schema offers, aligning with the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Store') and resource ('a fact or preference about a user'), distinguishing it from siblings like delete_memory, update_memory, list_memories, and search_memories. It also mentions the system's automatic handling of deduplication and contradiction detection, which adds unique functional context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use the tool by detailing importance score ranges (e.g., 0.1-0.3 for trivia, 0.5 for general facts, 0.7-1.0 for critical preferences), which helps in decision-making. However, it does not explicitly mention when not to use it or name alternatives among siblings, such as update_memory for modifications.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.