Server Quality Checklist

Tool Scores

• rememberA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond annotations by disclosing non-idempotency (calling twice creates two memories), eviction behavior under max_memories, and the permanent nature of pinning with no unpin mechanism. These are valuable behavioral traits not evident from the annotation hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately sized but every sentence contributes meaningful information: purpose, return value, dedup behavior, eviction, and pinning permanence. It is front-loaded with the action and result, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and detailed parameter schema, the description covers all necessary behavioral edge cases and context: return id, duplicate creation, eviction, and pinning. No important gaps are evident.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides thorough descriptions for both text and pinned, so the baseline is 3. The description adds extra semantics by explaining the permanent effect of pinned=True and that no unpinning is possible later, which is not fully explicit in the schema. This elevates it to 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Save' and resource 'a new note to this agent's memory', and explicitly notes that it always inserts, returning the new memory's id. This distinguishes it from sibling read tools like search and recall.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context about when to use the tool (saving notes) and behavior to expect (always inserts, no deduplication, possible eviction), but it does not explicitly name alternatives or say 'use this instead of search/recall'. The context is sufficient to infer appropriate use relative to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recallA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint and non-destructive, but description adds behavioral specifics: results ranked by hybrid keyword+semantic similarity, not chronological order, and edge-case handling when k exceeds memories. This goes beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four concise sentences, each adding distinct information: purpose, alternative, behavior, edge case. No redundant fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given output schema exists, description doesn't need to explain return values; it covers all behavioral aspects and usage context, making it fully sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers both params with descriptions, but the description adds meaning about `query` being matched by hybrid similarity and about `k`'s edge-case behavior (returns all if exceeded). This adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Recall notes previously saved via remember' – a specific verb (recall) with resource (notes from agent's own memory), and explicitly distinguishes from 'search' for the knowledge base. This differentiates it from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Directly instructs to use 'search' instead for the knowledge base, providing an explicit alternative. Also names 'remember' as the counterpart for saving, giving clear context for when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that results are ranked best-match-first by a hybrid of keyword and semantic similarity, not necessarily indexing order, and that exceeding k returns all documents rather than raising an error. This adds meaningful behavioral context beyond the annotations' read-only and idempotent hints, and explicitly states no side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tight and front-loaded, with the primary purpose in the first clause, followed by a sibling alternative and key behavioral details. Every sentence earns its place without redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present and robust annotations, the description covers the tool's scope, alternatives, ranking behavior, and error handling. It provides all necessary context for an agent to select and invoke the tool correctly, making it complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides clear descriptions for both parameters (query and k), covering 100% of them, so the baseline is 3. The description adds extra value by explaining the edge-case behavior of k exceeding document count and elaborating on the hybrid matching logic for query, going slightly beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches documents indexed via Index.add_files()/add_text() and explicitly distinguishes this from the agent's own memory. It names the specific resource and verb, and differentiates from the 'recall' sibling, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to use 'recall' instead for notes the agent chose to remember, providing a clear alternative for a different use case. It also establishes that this tool searches a reference knowledge base, giving the agent context for when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.