Server Quality Checklist

Tool Scores

• add_memoryC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions storage behavior but lacks critical details: what permissions are needed, whether storage is permanent or temporary, rate limits, or what happens on success/failure. The 'AUTO-STORE' hint suggests automated behavior but doesn't clarify how it differs from manual storage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the core purpose, though the second part about 'AUTO-STORE' could be integrated more smoothly. It avoids unnecessary elaboration but could benefit from clearer structure to separate purpose from behavioral hints.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, 0% schema coverage, no annotations, and sibling tools, the description is insufficient. It doesn't explain the storage mechanism, return values (despite having an output schema), or how it differs from related tools. The mention of 'AUTO-STORE' adds some context but doesn't compensate for the overall gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate but provides no parameter information. It doesn't explain what 'messages', 'user_id', or 'metadata' represent, their formats, or how they interact. The description adds no value beyond the bare schema, leaving all three parameters semantically undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool 'Store important information to memory' which provides a basic verb+resource combination, but it's vague about what 'memory' refers to and doesn't distinguish it from sibling tools like list_memories or search_memories. The 'AUTO-STORE user preferences and decisions' adds some specificity but remains ambiguous about scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like list_memories or search_memories. The mention of 'AUTO-STORE user preferences and decisions' implies a context for automated storage but doesn't provide clear when/when-not criteria or prerequisites for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_conversationsC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions auto-run behavior which is useful, but doesn't describe what the analysis entails, what data sources are used, whether it's read-only or has side effects, what permissions might be needed, or any rate limits. The description is insufficient for a tool with behavioral implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the core purpose. The auto-run information is efficiently appended. However, the dash formatting could be clearer, and the description could be more structured with separate sentences for different aspects of functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (which reduces need to describe return values), no annotations, and simple parameters, the description provides basic purpose and auto-run behavior but lacks crucial details about what analysis is performed, how parameters affect results, and behavioral characteristics. It's minimally adequate but has significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and 2 parameters, the description provides no information about the 'user_id' or 'limit' parameters. The schema shows defaults (null for user_id, 20 for limit) and types, but the description doesn't explain what user_id filters, what limit applies to, or how these affect the analysis. The description fails to compensate for the complete lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool 'analyze conversation patterns and generate insights', which provides a general purpose but lacks specificity about what patterns or insights are generated. It doesn't distinguish this analysis tool from potential siblings like 'suggest_next_actions' that might also analyze conversations. The description is vague about the exact nature of the analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'AUTO-RUN at session start' which provides some usage context, but doesn't explain when to manually invoke this tool versus relying on auto-run, nor does it differentiate when to use this versus sibling tools like 'search_memories' or 'suggest_next_actions' for conversation analysis. No explicit alternatives or exclusions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_memoriesB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions autonomous use but doesn't disclose key behavioral traits: whether this is a read-only operation, if it requires authentication, potential rate limits, or what the output contains. For a search tool with no annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two clauses: one stating the purpose and another providing usage guidance. It's front-loaded with the core function. However, the second clause could be more clearly integrated, and there's room for slight improvement in flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema, the description doesn't need to explain return values. However, with no annotations, 3 parameters (1 required), and 0% schema coverage, the description is incomplete—it lacks details on behavioral traits and parameter meanings, making it only minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'natural language' for the query parameter but doesn't explain the semantics of 'user_id' (e.g., filtering by user) or 'limit' (e.g., result count). With 3 parameters largely undocumented, the description adds minimal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search conversation history using natural language'. It specifies the resource (conversation history) and action (search via natural language). However, it doesn't explicitly differentiate from sibling tools like 'list_memories' or 'analyze_conversations', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes usage guidance: 'USE AUTONOMOUSLY based on conversation triggers', which implies when to use it (in response to conversation triggers). However, it doesn't specify when NOT to use it or mention alternatives like 'list_memories' for non-search retrieval, leaving the guidance incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_memoryB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool deletes a memory, which implies a destructive mutation, but doesn't cover critical aspects like whether deletion is permanent, requires specific permissions, has side effects, or returns confirmation. This leaves significant gaps for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with zero wasted words. It's front-loaded with the core action and resource, making it highly efficient and easy to parse. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's destructive nature, no annotations, and an output schema (which should cover return values), the description is minimally adequate but lacks depth. It states what the tool does but omits important context like safety warnings, error conditions, or integration with siblings (e.g., using 'list_memories' first). The presence of an output schema helps but doesn't fully compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'by ID', which adds meaning to the 'memory_id' parameter beyond the schema's minimal coverage (0%). However, it doesn't specify what format the ID should be (e.g., UUID, numeric) or where to obtain it. With one parameter and low schema coverage, this provides some but incomplete compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and the resource ('a specific memory by ID'), making the purpose immediately understandable. It doesn't explicitly distinguish from sibling tools like 'list_memories' or 'search_memories', but the verb 'Delete' inherently differentiates it from those read operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid memory ID), exclusions (e.g., not for bulk deletion), or suggest alternatives like 'list_memories' to find IDs first. Usage is implied only by the action itself.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_memoriesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but only states the basic action. It doesn't mention whether this is a read-only operation, if it requires authentication, how results are paginated or sorted, or what the output format entails, leaving critical behavioral traits unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero wasted words. It's front-loaded with the core action and resource, making it highly efficient and easy to parse at a glance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one optional parameter) and the presence of an output schema (which handles return values), the description is minimally adequate. However, it lacks details on behavioral aspects like permissions or result structure that would enhance completeness for a tool interacting with stored data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description implies no parameters are needed to list all memories, but the schema includes an optional 'user_id' parameter. Since schema description coverage is 0%, the description should compensate but doesn't explain this parameter's purpose. However, with only one optional parameter, the gap is minor, warranting a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all stored conversation memories'), making the tool's function unambiguous. However, it doesn't differentiate from sibling tools like 'search_memories' or 'analyze_conversations' that also involve memory retrieval, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_memories' (which might filter memories) or 'analyze_conversations' (which might process them). It lacks any context about prerequisites, timing, or exclusions, leaving usage decisions entirely to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• suggest_next_actionsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions the tool provides 'personalized recommendations' and suggests usage timing, it lacks critical behavioral details: what data sources it uses (e.g., memories, conversations), whether it requires specific permissions, how it generates recommendations, or what the output format looks like. For a recommendation tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—just one sentence with no wasted words. It's front-loaded with the core purpose ('Get personalized recommendations') followed by usage context. Every part of the description adds value, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (recommendation generation), no annotations, and an output schema exists (which should document return values), the description is partially complete. It covers purpose and usage timing but misses parameter semantics and behavioral details like data sources or algorithm behavior. The presence of an output schema helps, but the description should do more for a tool that likely involves personalization logic.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter (user_id) with 0% description coverage in the schema itself. The tool description provides no information about this parameter—it doesn't explain what user_id represents, whether it's required, or how it affects the recommendations. With low schema coverage, the description fails to compensate, leaving the parameter's meaning unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get personalized recommendations' with the specific trigger condition 'when user seems stuck or asks 'what now?''. It uses a clear verb ('Get') and specifies the resource ('personalized recommendations'), though it doesn't explicitly distinguish from sibling tools like 'analyze_conversations' or 'search_memories' which might also provide insights.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'AUTO-SUGGEST when user seems stuck or asks 'what now?''. This gives clear guidance on when to invoke the tool. However, it doesn't specify when NOT to use it or mention alternatives among the sibling tools, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.