Server Quality Checklist

Tool Scores

• knowledge_graphB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses temporal validity and that code structure edges are created by knowledge-ingest, which is useful. However, it does not warn about destructive side effects of unlink/invalidate or the scope of bulk operations, leaving behavioral uncertainty.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise, using three sentences to cover operations, relationship types, and code structure nuance. It could tighten by omitting the redundant relationship list from the enum, but it remains front-loaded and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema is present, and the description does not describe return values for actions like list or traverse. It also omits details about state mutations (e.g., irreversibility of unlink). Given the tool's complexity and no annotations, this is a significant gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds minimal value: it re-lists relationship types (already in the enum) and notes the 'code:' prefix for node IDs. It does not explain parameter interactions beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs knowledge graph operations: create/remove edges, traverse via BFS, bulk-import code edges. It also lists relationship types and code structure specifics. However, it does not explicitly contrast with sibling tools like knowledge_search or knowledge_analyze, so differentiation relies on the implicit 'graph' focus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus siblings. It does not mention when not to use it, nor does it reference alternative tools. The code structure note implies a use case, but there is no clear routing or exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• knowledge_adminB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, and it does disclose destructive effects for prune_orphans ('delete embeddings') and vacuum ('reclaim free pages'). However, it understates rebuild_embeddings' destructive rewriting of existing embeddings, and it completely omits the 'promote' action's behavior (apply vs explain modes, write behavior vs read-only). For a tool with several destructive operations, the disclosure is partial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The core operations are front-loaded in the opening sentence, with each action's purpose expanded in a single subsequent sentence. The structure is efficient and scannable. Minor deduction for the slightly run-on single-sentence expansion and for leaving out promote, which makes the list feel incomplete rather than intentionally concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex admin tool with 10 parameters, 6 actions, no output schema, and no annotations, the description should be comprehensive — but it covers only 5 of 6 actions and omits the entire promote workflow with its 4 dedicated parameters (promote_mode, min_score, min_recall_count, min_unique_queries). An agent reading only the description would never learn a whole callable mode exists, which is a significant completeness gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all 10 parameters, giving a baseline of 3. The description adds marginal value by linking rebuild_embeddings to the provider-switching use case, but it adds no semantics beyond the schema for the config params (git_url, memory_dir, auto_distill) and leaves the promote-related parameters entirely unexplained in prose despite them being a substantial cluster of the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear purpose — administrative operations on the knowledge/vector store — and enumerates five concrete actions (status, config, rebuild_embeddings, prune_orphans, vacuum) with their effects. It clearly distinguishes itself from the read/search siblings. However, it silently omits the 'promote' action that exists in the schema enum, and the umbrella phrase 'Admin operations' is vague until the action list clarifies it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives actionable when-to-use context for rebuild_embeddings ('useful when switching providers') and explains what each action accomplishes, which implicitly routes the agent to the right action. But it provides no guidance for the 'promote' action (its dedicated params min_score, min_recall_count, min_unique_queries, promote_mode are never narrated), and it never states when NOT to use the tool or how it differs from siblings beyond 'admin' framing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• knowledgeA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden — and it does disclose meaningful side effects: write/delete trigger auto git sync, sync is a manual git pull+push, and wakeup returns a token-budgeted section-priority bundle. This is solid disclosure for a mutation-capable tool; it only omits reversibility (e.g., whether delete is recoverable from git history) and confirmation behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The purpose is front-loaded and the content is dense and informative, but it is presented as one long run-on sentence without line breaks or structural separation, making the six actions and their caveats harder to parse at a glance. Fewer words could be used; the parentheticals are useful but poorly delimited.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a high-complexity tool (9 params, 6 actions, enums, nested objects) with no output schema, the description explains wakeup's return bundle but not the return values for the other five actions (e.g., what list returns, what read returns on success/failure). The schema covers parameter semantics well, but the absence of output descriptions for the remaining actions leaves gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the input schema already documents all nine parameters with action-scoped descriptions. The description adds value on top by explaining wakeup semantics — section ordering, defaults, budget behavior — but does not substantially enrich the other parameters beyond what the schema states. A baseline of 3 is appropriate given the high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear purpose ('Knowledge base CRUD, sync, and session-start hydration') and enumerates six concrete actions with brief one-line definitions. It implicitly distinguishes from siblings (search, graph, analyze) by being the CRUD/sync/wakeup orchestrator, though it never names the siblings or their differing responsibilities explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is conveyed per-action ('call once at session start' for wakeup, 'auto git sync' for write/delete) and the verb definitions imply when each is appropriate. However, with five siblings present, there is no explicit when-to-use-versus-alternative guidance (e.g., when to prefer knowledge_search over this tool's list action), and no exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• knowledge_sessionA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It states the actions but does not mention whether operations are read-only, side effects, required permissions, error behavior (e.g., invalid session_id), or any rate limits. For a tool with three read-like actions, this is a notable gap; the agent is left to infer safety and failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with zero waste. It front-loads the core concept 'Session operations' and immediately explains the three actions. Every sentence contributes to understanding the tool, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides enough to invoke the tool correctly for the three actions, but it omits the response format entirely. Since there is no output schema, the agent is unaware of what each action returns (e.g., list returns an array, get returns messages). For a multi-action tool, this is a moderate gap, though the actions are simple enough that an agent might infer typical behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All seven parameters are fully described in the schema (100% coverage), so the schema carries the heavy lifting. The description adds context for how the action parameter drives behavior (list vs get vs summary) and clarifies the purpose of some parameters indirectly (e.g., tail for get, limit for list), but it does not add new semantic detail beyond the schema. This aligns with the coverage baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Session operations: list sessions, get a full conversation, or get a summary.' It identifies three distinct actions (list, get, summary) and explicitly differentiates from sibling tools by focusing on session operations, which none of the sibling names (knowledge, knowledge_search, knowledge_admin, knowledge_graph, knowledge_analyze) cover.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides per-action guidance: 'Use action list to browse sessions, get to retrieve messages, summary for a quick overview.' This clarifies when to use each action, but it does not mention when to favor this tool over siblings such as knowledge_search or knowledge_analyze. There is no explicit exclusion or alternative routing at the tool level, though the action-level guidance is useful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• knowledge_analyzeA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears the full disclosure burden, yet it is internally contradictory on the consolidate action: the verb implies merging/mutation while the gloss '(find duplicates)' implies read-only discovery. There is no statement about whether any action mutates entries, what output shape results, or performance implications — a real gap for a multi-action analysis tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Purpose is front-loaded in the first sentence and the whole description runs roughly 60 words with no filler. The trailing 'Actions:' enumeration is partially redundant with the schema enum but serves as a useful name→semantic mapping, earning its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a complex tool — 9 parameters, 8 actions, no output schema, no annotations. All action semantics are enumerated, but the description is silent on per-action return formats and on whether any action (notably consolidate) has side effects, both material for an analysis surface of this breadth.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the schema's parameter descriptions are unusually rich (per-action defaults, the Jaccard rationale for group_similarity, min_count semantics). The tool description adds little beyond re-listing action names, so the schema-carrying baseline of 3 is correct.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description enumerates eight distinct analyses with denotative glosses (duplicates, unconnected entries, god nodes, bridges, gaps, zero-result queries, stale-by-code-activity, brief), clearly binding the tool to knowledge-base entry analysis. This differentiates it from siblings like knowledge_search (querying) and knowledge_admin (admin actions) without needing to open a schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Each action carries clear context, e.g., 'gaps (entries with 0-1 edges)', 'bridges (cross-cluster connectors)', and search_gaps is explicitly framed as 'the single best signal for what entries should I write next?'. Missing, however, are explicit when-not or alternative statements that route selection against sibling tools at the tool level.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• knowledge_searchA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure, and it excels. It reveals the return shape ({mode, sessions, knowledge}), explains the hybrid ranking (TF-IDF + semantic), discloses fallback behavior (pure TF-IDF if embeddings unavailable), details the boost/filter semantics of category, and even mentions the 'score_components' breakdown when explain=true. This is comprehensive transparency for a read-only search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but well-organized: it leads with the core action and return format, then systematically explains the two modes and key behaviors. Every sentence adds distinct information (modes, ranking, fallback, category handling) with no filler. It is long, but each phrase earns its place given the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 12 parameters, 4 enums, and no output schema, the description covers the critical decision points: mode selection, source scope, ranking approach, category application, and MMR parameter meaning. It explains the return envelope and the score breakdown for explain mode. No essential usage context appears missing; the schema handles parameter-level details, and the description supplies the higher-level orchestration model.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so each parameter is already described. The description adds meaningful context beyond the schema by explaining how scope switches modes, how category boost/filter interacts with category_mode, the MMR tradeoff (lambda 1.0 = pure relevance, 0.0 = diversity), and that 'role' is ignored in scoped mode. This is valuable semantic glue, though it doesn't cover every parameter explicitly – some are only in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Search across sessions AND knowledge entries.' It immediately clarifies the dual-source nature and then distinguishes two modes (general vs. scoped) with clear scope semantics. This cleanly differentiates it from sibling tools like knowledge_session or knowledge_graph, which are likely more specialized.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use general mode (no scope) vs. scoped mode ('scope' set), including that scoped mode is sessions-only and filtered by domain. It mentions the behavior of parameters like 'role' being ignored when scope is set. However, it does not name any sibling tools as alternatives or give exclusion conditions, so the guidance is strong but not complete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.