Server Quality Checklist

Tool Scores

• get_episodic_memoryC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the action ('retrieve') without detailing aspects like authentication needs, rate limits, error handling, or what the return format looks like (e.g., list of memories with timestamps). This leaves significant gaps in understanding how the tool behaves in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized for a simple retrieval tool, though it could be more front-loaded with key details if expanded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what episodic memories are, how they're structured, or what the return values include, which is crucial for a tool with sibling memory types and no structured output documentation. This leaves the agent under-informed about the tool's context and results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for both parameters ('limit' and 'userId') in the input schema. The description adds no additional meaning beyond this, such as explaining the format of 'userId' or typical 'limit' values. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the verb ('retrieve') and resource ('episodic memories for a user'), which provides a basic understanding of the tool's function. However, it lacks specificity about what episodic memories are or how they differ from other memory types, and doesn't distinguish this tool from sibling tools like 'get_long_term_memory' or 'get_short_term_memory', making the purpose somewhat vague.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'search_episodic_memory' for filtered retrieval or differentiate from other memory types, leaving the agent with no context for tool selection beyond the basic purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_long_term_memoryC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral insight. It states what gets stored but doesn't disclose whether this creates new records or updates existing ones, what permissions are required, whether data persists permanently, or what happens on success/failure. For a data storage tool with zero annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point with no wasted words. It's appropriately sized for a simple storage operation, though it could be more front-loaded with critical behavioral information given the lack of annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a data storage tool with no annotations and no output schema, the description is insufficient. It doesn't explain what happens after storage (success response, error conditions), whether this overwrites existing data, or how it differs from sibling storage tools. The context demands more completeness for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 4 parameters thoroughly. The description mentions the same three data categories (demographics, contact, preferences) but adds no additional meaning beyond what the schema provides. The baseline of 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool stores user data (demographics, contact details, preferences), which is a clear verb+resource combination. However, it doesn't distinguish this from sibling tools like 'set_short_term_memory' or 'add_episodic_memory' - the purpose is clear but lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'set_short_term_memory' and 'get_long_term_memory', there's no indication of when this long-term storage is appropriate versus short-term or when to retrieve versus store data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_short_term_memoryC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves data, implying a read-only operation, but does not specify if it requires authentication, has rate limits, returns structured or unstructured data, or handles errors. This leaves significant gaps in understanding the tool's behavior beyond basic retrieval.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It is front-loaded with the core action and resource, making it easy to parse quickly. Every word earns its place by directly conveying the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a retrieval tool. It does not explain what data is returned, in what format, or any behavioral traits like error handling. For a tool with 2 parameters and no structured output, more context is needed to guide effective usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear documentation for both parameters: 'key' (optional, returns all if not provided) and 'sessionId' (required). The description does not add any meaning beyond what the schema provides, such as explaining the format of keys or session IDs, so it meets the baseline for high schema coverage without extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Retrieve data from short-term memory' clearly states the verb ('retrieve') and resource ('short-term memory'), distinguishing it from siblings like 'add_episodic_memory' or 'set_short_term_memory'. However, it lacks specificity about what kind of data is retrieved or the scope, making it somewhat vague compared to a fully detailed purpose statement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_long_term_memory' or 'get_episodic_memory'. It does not mention any prerequisites, exclusions, or specific contexts for usage, leaving the agent to infer based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_episodic_memoryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool adds a memory but doesn't disclose behavioral traits like whether this is a write operation (implied but not explicit), what permissions are needed, how duplicates are handled, or what happens on success/failure. For a mutation tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and includes clarifying parentheses that add value without verbosity. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (7 parameters, 3 required), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what the tool returns, error conditions, or behavioral nuances. For a mutation tool with rich parameters, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 7 parameters. The description adds no parameter-specific information beyond what's in the schema, such as examples or usage notes. With high schema coverage, the baseline is 3, as the description doesn't compensate but also doesn't need to.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Add') and resource ('episodic memory'), with additional clarifying context ('past experience or event'). It distinguishes from siblings like 'get_episodic_memory' by specifying creation rather than retrieval. However, it doesn't explicitly differentiate from other memory-setting tools like 'set_long_term_memory' beyond the memory type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'set_long_term_memory', 'set_short_term_memory', and 'search_episodic_memory', there's no indication of appropriate contexts, prerequisites, or exclusions for this specific tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_long_term_memoryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the retrieval action without detailing what 'long-term memory' entails (e.g., format, scope, or persistence), whether it requires authentication, or if there are rate limits. This is inadequate for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of memory retrieval tools and the lack of annotations or output schema, the description is incomplete. It doesn't explain what 'long-term memory' returns, how it differs from other memory types, or any behavioral traits, leaving significant gaps for an AI agent to understand its use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'userId' parameter clearly documented. The description adds no additional parameter semantics beyond what the schema provides, such as explaining the user context or memory retrieval specifics. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Retrieve') and resource ('long-term memory for a user'), making the purpose understandable. However, it doesn't distinguish this tool from its siblings like 'get_episodic_memory' or 'get_short_term_memory', which also retrieve memory types, so it lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to prefer long-term over short-term or episodic memory, nor does it reference sibling tools like 'set_long_term_memory' for updates or 'search_episodic_memory' for queries, leaving usage context unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_episodic_memoryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'search' but doesn't explain what constitutes an episodic memory, how results are ranked, whether it's read-only or has side effects, or any limitations like rate limits. For a search tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste—'Search episodic memories by content'—that directly conveys the core action. It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of searching memories, lack of annotations, and no output schema, the description is incomplete. It doesn't cover behavioral aspects like result format, error handling, or how it differs from siblings. For a tool with 3 parameters and no structured safety hints, more context is needed to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with clear descriptions for 'limit', 'query', and 'userId'. The description adds no additional meaning beyond the schema, such as query syntax or result format. With high schema coverage, the baseline is 3, as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('search') and resource ('episodic memories') with the scope 'by content', making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_episodic_memory' or 'get_long_term_memory', which might also retrieve memories, so it misses full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_episodic_memory' and 'get_long_term_memory' that might retrieve memories differently, there's no indication of context, prerequisites, or exclusions, leaving usage ambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_short_term_memoryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Store') but lacks details on permissions, idempotency, error handling, or what happens on conflicts. It mentions TTL via the schema but doesn't explain behavioral implications like automatic expiration.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse quickly without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't cover behavioral aspects like side effects, return values, or error conditions. Given the complexity of memory operations and lack of structured safety hints, more context is needed for safe agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description adds no additional semantic context beyond implying session-scoped storage, which is already covered by the 'sessionId' parameter in the schema. Baseline 3 is appropriate as the schema handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Store') and resource ('data in short-term memory'), specifying it's for a session. However, it doesn't explicitly differentiate from sibling tools like 'set_long_term_memory' or 'add_episodic_memory', which would require mentioning the specific memory type and session scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for session-specific storage but provides no explicit guidance on when to use this tool versus alternatives like 'set_long_term_memory' or 'add_episodic_memory'. There's no mention of prerequisites, exclusions, or comparative contexts with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.