Server Quality Checklist

Tool Scores

• memory_statsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves statistics, implying a read-only operation, but doesn't disclose any behavioral traits such as performance characteristics, rate limits, authentication needs, or what happens if the memory store is empty. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It's front-loaded with the core action and includes specific metrics, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is minimally adequate. It explains what the tool does but lacks details on behavioral aspects and usage context. Without annotations or output schema, it should ideally provide more on return values or operational constraints, but it meets the basic need for a simple stats tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameters need documentation. The description doesn't add parameter details, which is appropriate here. Baseline is 4 for 0 parameters, as the schema fully covers the absence of inputs, and the description doesn't need to compensate for any gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('statistics about the memory store'), specifying what metrics are retrieved ('counts, categories, avg confidence'). It distinguishes from siblings like memory_store (store vs. stats) and memory_recall (retrieve content vs. stats), though not explicitly. However, it doesn't fully differentiate from all siblings (e.g., memory_feedback might also involve stats), so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to use memory_stats over other tools (e.g., for monitoring vs. retrieving content) or any prerequisites. This leaves the agent without explicit usage context, scoring low as it offers no when/when-not/alternatives information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_storeB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions 'provenance tracking' (which hints at metadata recording), it doesn't describe critical behavioral aspects like whether this is a write operation (implied by 'Store'), what permissions are needed, whether it overwrites existing data, error handling, or response format. For a tool with 6 parameters and no annotations, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—just two sentences that are front-loaded with the core purpose. Every word earns its place: the first sentence states the action and key feature, while the second provides concrete usage examples without redundancy. There's zero waste or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 parameters, no annotations, no output schema, and 50% schema coverage), the description is incomplete. It doesn't address behavioral traits like mutation effects, error cases, or return values, and it leaves parameter semantics largely undocumented. For a tool that stores data with provenance, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%, meaning half the parameters lack descriptions in the schema. The description doesn't add any parameter-specific information beyond what's implied by 'provenance tracking' (which might relate to source_type, confidence, etc.). It doesn't explain the meaning of parameters like 'category' enums or 'tags', nor does it compensate for the low coverage gap. With 6 parameters and partial schema documentation, this is minimally adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Store' and resource 'memory' with the specific purpose of 'with provenance tracking'. It provides concrete examples of use cases (lessons learned, strategic insights, etc.), making the purpose specific and actionable. However, it doesn't explicitly distinguish this tool from its siblings like memory_feedback or memory_recall.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance through the examples ('Use for lessons learned, strategic insights...'), which suggests appropriate contexts. However, it doesn't explicitly state when to use this tool versus alternatives like memory_feedback or memory_recall, nor does it provide any exclusion criteria or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_recallB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: ranking by 'decay-weighted relevance', support for filtering, and that it 'Returns memories with provenance and confidence scores.' This covers output format and ranking logic, but misses details like pagination, error handling, or performance characteristics (e.g., rate limits). For a retrieval tool with no annotations, this is adequate but has gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and well-structured: one sentence states the core purpose and ranking, another adds filtering support, and a third specifies the return format. Every sentence earns its place with no wasted words, and information is front-loaded effectively.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, 60% schema coverage, no annotations, and no output schema, the description is moderately complete. It covers the tool's purpose, filtering behavior, and output format, which is sufficient for basic understanding. However, it lacks details on error cases, performance, or deeper usage context, leaving room for improvement in a tool with multiple parameters and no structured output documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 60%, so the description must compensate. It adds value by explaining that the tool 'Supports keyword search and category/tag filtering,' which clarifies the purpose of 'query', 'category', and 'tags' parameters beyond their schema descriptions. However, it doesn't address 'min_confidence' or 'limit' parameters, leaving them reliant on schema coverage. The description provides partial compensation, aligning with the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Recall memories ranked by decay-weighted relevance.' It specifies the verb ('recall') and resource ('memories'), and mentions ranking methodology. However, it doesn't explicitly differentiate from siblings like 'memory_store' (which presumably stores memories) or 'memory_stats' (which likely provides statistics), though the 'recall' verb implies retrieval versus storage/statistics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'memory_feedback' or 'memory_stats'. It mentions support for 'keyword search and category/tag filtering', which hints at usage for filtered retrieval, but lacks explicit when-to-use or when-not-to-use statements, prerequisites, or comparisons to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_feedbackA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses behavioral traits by explaining the effect on memory prioritization ('memories marked useful bubble up, unused ones decay away'), which adds context beyond basic functionality. However, it lacks details on permissions, rate limits, or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two concise sentences that directly state the purpose and impact. Every sentence earns its place by providing essential information without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, no annotations, and no output schema, the description is reasonably complete. It covers purpose and behavioral impact but could improve by addressing usage guidelines more explicitly or detailing response expectations, though it suffices for a feedback tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the parameters. The description does not add meaning beyond the schema, as it does not explain parameter usage or constraints. Baseline 3 is appropriate since the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Record') and resource ('whether a recalled memory was useful'), and distinguishes it from siblings by focusing on feedback rather than recall, storage, or statistics. It explains the action and its impact on the memory system.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning 'recalled memory' and the learning flywheel, suggesting it should be used after memory_recall. However, it does not explicitly state when not to use it or name alternatives like memory_store for storing new memories, leaving some guidance gaps.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.