Server Quality Checklist

Tool Scores

• sqliteA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It does disclose that docs_* results are pre-chunked and advises against file reads. However, it fails to mention that the 'exec' command can perform arbitrary SQL writes, and does not warn about destructive actions or the need for the 'write' flag. This is a significant transparency gap for a tool that can mutate the database.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured: a brief overview, a list of commands, then detailed explanations for each key command. Every sentence serves a purpose, and the use of bold command names and concise phrases makes it scannable. The density is justified given the tool's complexity, though it could be slightly trimmed for the most basic database commands.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The docs_* commands are thoroughly explained, including behavior, params, and when to use them. However, the basic database commands (catalog, exec, list_databases, open_database, health_check) are only listed without any explanation of their purpose, return values, or side effects. Given that there is no output schema, the description fails to fully compensate for the complexity of the database operations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds substantial meaning beyond the schema. It clarifies that the 'sql' param doubles as a search query for docs_search, and it details command-specific params like {chunk_id, before, after} for docs_context and {query, limit, threshold} for docs_search_in_doc. While the schema already covers 60% of parameters, the description fills gaps and provides practical semantics for the other 40%.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's dual purpose: 'SQLite database operations + document-corpus search.' It then enumerates specific commands, and explicitly distinguishes between keyword search, semantic search, hybrid search, full-document retrieval, context expansion, and in-document search. This level of specificity and differentiation from siblings (retrieve, memory, kg) is excellent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance for each docs_* command: 'Use for exact terms, API names, phrases' for docs_search, 'Use for general queries' for docs_semantic_search, and '*PREFER THIS*' for docs_hybrid. It also advises against file reads and recommends alternatives, making the usage intent crystal clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• retrieveA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It explains key internal behaviors: KG-powered query expansion with an example, reciprocal rank fusion, source attribution, and compact mode token estimates. However, it does not explicitly state whether the operation is read-only or side-effect-free, though the nature of retrieval implies it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is about 80 words and front-loads the core purpose in the first sentence. Every sentence adds value: purpose, usage scenario, expansion behavior, fusion method, output attribution, and compact defaults. It is dense but not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 10 parameters and no output schema, the description provides a strong high-level overview, including expansion, fusion, and sourcing. However, it does not describe the return result structure beyond 'source attribution,' nor does it explain how parameters interact (e.g., rerank requiring rerank_top_n). This is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds some extra context for specific parameters: compact mode token counts (~500-800 vs ~2000) and expand_query behavior, but these details are also partially in the schema. The description does not meaningfully improve understanding of the more complex parameters like rerank or semantic_threshold.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Unified retrieval across KG, Memory, and Docs' with a specific verb ('retrieve') and resource scope. It also distinguishes itself from sibling tools by focusing on conceptual queries and knowledge-graph expansion, making its role unique.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells the agent when to use the tool: 'Use for conceptual queries where keyword search fails.' It provides a clear context but does not explicitly mention when to avoid it or direct users to alternatives like the sibling tools (sqlite, memory, kg), so it misses an explicit exclusion list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kgA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description carries full burden and delivers substantive behavioral context: limits (10K entities, 100 obs/entity), audit log for destructive ops, storage format (JSONL), default inference behavior for entity_type, and hybrid search weighting. Lacks detail on specific action side effects, but substantial coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is dense but justified given 15 actions and multiple behavioral notes. Structured logically: purpose, usage, search mode, limits, actions, critical warning. No filler or redundancy despite length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a high-complexity tool with 22 params, 15 actions, nested objects, and no output schema, the description covers purpose, usage, limits, audit, storage, search behavior, and a key input pitfall. Lacks per-action behavior and return formats, but provides a strong operational overview.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 59%, so this is moderate. The description adds critical semantics absent from schema by warning that omitting entity_type defaults to 'unknown', which pollutes the graph. Also mentions hybrid search weights and batch operation patterns (partially in schema). Adds value beyond structured definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly defines the tool as an 'Entity-relationship graph for structured facts' and states it's for entities with relationships, listing example domains. It distinguishes from siblings by explicitly contrasting with memory ('free-form text/learnings') and positioning it before semantic_search, making purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: 'Use for entities with relationships - concepts, tools, people, patterns. Check here before semantic_search. For free-form text/learnings, use memory instead.' This directly addresses alternatives and search precedence.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memoryA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and does so thoroughly. It discloses RAM-backed async disk persistence, server-side concatenation and re-embedding for append, semantic search with synonym expansion ('fast'->'quick'), full-overwrite semantics for store, and marker behavior for replace_section. It even mentions preprocess=False for exact match, adding genuine behavioral nuance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Although lengthy, the description is densely packed and logically structured: it opens with the most critical startup routine, then defines the tool's purpose, summarizes actions, lists key conventions, and ends with targeted usage tactics. Every sentence earns its place given the tool's complexity (7 actions, 16 params).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main use cases, return shape ({success, results/value}), persistence semantics, and key conventions. It lacks detailed per-action return schemas and explicit error-handling behavior. Given no output schema and high complexity, it is mostly complete but not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by clarifying action semantics (store=full overwrite, append=add fragment, replace_section=in-place edit), explaining the purpose of markers, and providing key conventions like 'learn:<topic>'. The reference to preprocess=False is extra, though the parameter is not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is 'semantic memory for cross-session persistence' and enumerates seven specific actions (store, append, replace_section, search, get, delete, list), along with key naming conventions. This distinguishes it from sibling tools by explicitly framing its role as a persistent memory system.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides strong when-to-use guidance: 'On session start: get session:latest + project:overview', 'When encountering issues or problems search for related context', and 'On milestones: store session:latest'. It also advises using append for large rolling docs and replace_section for in-place edits. However, it never explicitly contrasts with sibling tools like sqlite/retrieve/kg, so it lacks explicit when-not-to-use alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.