Server Quality Checklist

Tool Scores

• distillery_gh_syncA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions batched processing and async operation, and notes that background mode returns a job_id. However, it omits crucial details such as whether the sync updates existing entries, creates duplicates, or requires authentication. It also lacks information on error handling or rate limits, so the behavioral picture is incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a single opening sentence states the purpose, followed by a bulleted list of parameter explanations. Each line adds value and the format is easy for an agent to parse. There is no waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While parameter semantics are well covered, the description lacks details on the return value (except in background mode), error conditions, prerequisites such as authentication tokens, and the exact semantics of the sync operation. Given the absence of an output schema and annotations, the description should provide more context about side effects and expected outcomes. It is sufficient for a basic call but not fully complete for autonomous decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero descriptions (0% coverage), so the description must compensate. It does this thoroughly by explaining each parameter: url is a repository slug or full URL, author has a default, project is optional, and background enables async mode with an immediate job_id return. This fully clarifies the meaning and defaults beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool syncs GitHub issues and PRs into the knowledge base using a batched pipeline. It identifies the specific verb and resource, making the purpose clear. While it doesn't explicitly differentiate from sibling tools, the 'sync' action and GitHub focus distinguish it from store, get, search, and other operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that this tool is for syncing GitHub data into the knowledge base, which gives a general sense of when to use it. However, it does not explicitly state when to prefer this over alternatives like distillery_store or distillery_ingest_doc, nor does it provide exclusions or cautionary notes. The usage context is adequate but not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_sync_statusA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the tool's primary behavior (checking sync job status, filtering by job_id or source_url, and listing all recent jobs when neither is given). However, it does not explicitly state that the tool is read-only, nor does it describe the response format, pagination, or potential errors. This leaves some ambiguity about side effects and output structure, though the name and purpose imply a non-mutating status check.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured. It opens with the primary purpose, then lists each parameter in a clear format, and concludes with the default behavior. Every sentence contributes to understanding the tool without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The core usage is covered, but for a status tool with no output schema and no annotations, the description omits details about the response structure (e.g., what fields are in each job status), whether there are any access limitations, or what constitutes a valid job_id or source_url format. Given the simplicity of the tool, this is partially acceptable, but an agent might still need to infer expected output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clearly explains the semantic meaning of both job_id and source_url, including the fallback behavior when neither is provided. It does not detail whether both can be combined or the expected data format, but it adds significant value beyond the bare type information in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ('Check the status of background sync jobs'), which clearly distinguishes it from siblings like distillery_status (general status) and other distillery tools. The purpose is unambiguous and the parameter examples reinforce the scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains exactly when to use each parameter (job_id, source_url, or none) and what result each produces. It does not explicitly mention alternatives or when not to use this tool, but the context of 'background sync jobs' differentiates it from other distillery tools. The lack of a direct contrast with distillery_status is a minor gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_statusA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it details the return schema (which implies a read-only probe), it does not explicitly state side effects, authentication requirements, or error conditions. For a status tool, this is acceptable but not fully transparent about whether any modification occurs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (USE WHEN, PARAMS, RETURNS, RELATED). The main purpose is front-loaded in the first line, and every section adds relevant information without redundancy. It is appropriately sized for a status probe.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Since there is no output schema, the description provides a detailed return schema covering all expected fields and types. It also gives usage guidance and related alternatives. For a tool with no parameters, this is comprehensive and leaves no ambiguity about what the agent can expect when calling it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the description states 'PARAMS: (none)' while the schema is an empty object. The baseline for 0 params is 4; the description adds no extra meaning but confirms the absence of required inputs, which is mildly helpful.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Return a lightweight in-protocol health/metadata probe.' This clearly distinguishes it from siblings like distillery_list (entry counts/filtering) and distillery_configure (runtime configuration). The related section also reinforces the distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'USE WHEN: verifying MCP connectivity... without relying on the HTTP-only /health endpoint.' It also names alternatives (distillery_list, distillery_configure) and explains what they are for, providing clear routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_ingest_docA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden of behavioral disclosure. It explains key behaviors: chunking of large text, idempotent deduplication by content hash, and the exact response on re-ingest (persisted=false, dedup_action='skipped'). It does not explicitly mention permissions or failure modes beyond simple error codes, but the core side effects are clearly disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (intro, USE WHEN, PARAMS, RETURNS, RELATED) and front-loaded with the core purpose. It is somewhat long, but every section adds necessary detail for correct usage, so the length is justified. The use of headings improves scannability for an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 9 parameters, no annotations, and no output schema, the description is remarkably complete. It explains the dedup/chunking behavior, provides detailed parameter guidance, explicitly lists success and error return shapes, and references related tools. Nothing an agent needs to call it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates. It documents every parameter (text, author, doctype, source, external_id, title, project, tags, metadata) with explanations of defaults, valid values, and how they are stored (e.g., doctype applied as tag and metadata, external_id defaults to SHA-256 hash). This goes far beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Ingest an arbitrary document (ADR, spec, decision, customer feedback)'. It uses a specific verb and resource and explicitly distinguishes itself from siblings like distillery_store (single entry) and PreCompact transcripts, so an agent can easily identify when to use it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'USE WHEN' section explicitly states when to use this tool: 'importing a standalone document... so it becomes queryable knowledge with provenance'. It also contrasts with distillery_store and PreCompact transcripts, providing clear alternatives and the conditions that select this tool over them.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_getA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior. It does so by specifying the return structure for both success (id, content, entry_type) and error (code, message). While 'Retrieve' implies a read-only operation, it does not explicitly state that no mutation occurs, but the provided return/error formats give substantial behavioral insight for a simple GET.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (USE WHEN, PARAMS, RETURNS, RELATED), front-loading the purpose. Every sentence provides essential information without redundancy. The format is scannable for an agent and not overly verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter retrieval tool with no output schema and no annotations, the description is complete. It covers when to use it, the parameter meaning, expected return types, and error handling, and even suggests sibling tools for related use cases. Nothing an agent needs to call it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the schema by explaining that entry_id is a 'UUID of the entry to retrieve.' The schema only defines it as a string, so this clarifies the expected format and purpose, fully compensating for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Retrieve') and resource ('knowledge entry') with a unique ID, clearly distinguishing from siblings like distillery_search and distillery_list. The description explicitly notes the tool retrieves a single entry by ID, which is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides an explicit 'USE WHEN' section that tells the agent exactly when to use this tool (fetching full content/metadata of a specific entry). It also lists related tools (search, list) as alternatives, giving clear routing context without needing to inspect other schemas.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_updateA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It clearly describes success and error return shapes, including error codes like NOT_FOUND and INVALID_PARAMS. It also implies partial-update semantics by saying 'one or more fields'. However, it does not explicitly state that non-provided fields remain unchanged, and it does not detail permission or validation behavior beyond error codes. Still, it gives substantial behavioral context for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (USE WHEN, PARAMS, RETURNS, RELATED). It front-loads the core purpose and usage, then enumerates all parameters in a compact list, followed by return formats and related tools. Every sentence contributes valuable information with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 11 parameters, no output schema, and no annotations, the description covers all essential information: parameter names and valid values, return formats, error codes, the requirement of at least one updatable field, and relationships to sibling tools. An agent has everything needed to call the tool correctly without further research.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully explain each parameter. It does so by listing every parameter with its type, purpose, and valid enum values (e.g., for entry_type, status, verification). This adds meaningful semantics that the schema alone cannot convey.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Update') and a clear resource ('existing knowledge entry'), and it explicitly distinguishes itself from sibling tools like distillery_correct (for superseding) and distillery_get (for reading). This removes ambiguity about when this tool applies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'USE WHEN' section explicitly tells the agent when to use the tool (modifying fields) and even specifies a precondition (at least one updatable field must be provided). The RELATED section names the alternative tool (distillery_correct) and the condition that selects it, leaving no ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_listA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden. It discloses default status filtering, pagination ordering ('newest first'), response modes (summary/full/ids/review), content preview behavior, source-to-feed_url aliasing, mutual exclusions (group_by/output='stats'), stale_days semantics, include_evergreen backfill handling, and structural filter behavior. It also details return fields and error codes. This is exhaustive and beyond what any annotation could provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but necessary given 24 parameters. It is well-structured with sections (PARAMS, RETURNS, RELATED) and front-loads the core purpose and usage. Some redundancy exists (status behavior repeated in the intro and status parameter, include_archived explanation overlaps), but it remains efficient and each parameter earns its place. A minor trim would push it to 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a highly complex tool with no output schema and no annotations, the description is exceptionally complete. It explains return shape, error codes, mutual exclusions, edge cases (e.g., include_evergreen, structural orphan behavior), and references related tools. An agent would have everything needed to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all 24 parameters. It does so thoroughly: each parameter explains its purpose, valid values (e.g., entry_type enums, status options, source origins), defaults, and special behaviors (e.g., source URL alias to feed_url, structural filter combination, content_max_length truncation). This adds substantial meaning far beyond the bare schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'List knowledge entries with optional filters and pagination (newest first).' It immediately distinguishes from distillery_search by stating it is for browsing/filtering without a semantic query, and it names the alternative explicitly. The purpose is unambiguous and differentiates from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'USE WHEN: browsing or filtering entries without a semantic query' and directs to 'Use distillery_search instead when you have a natural-language question.' It also explains the default status behavior (active/pending_review with archived hidden) and how to override it, giving clear when-to/not-to guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_relationsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Given there are no annotations, the description carries the full burden of behavioral disclosure. It details side effects (upsert on re-assert), idempotency of promote_entities, error codes (NOT_FOUND, INVALID_PARAMS, INTERNAL), return structures for every action, and even explains the orphan_rate formula. No contradictions with annotations (none provided).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured with sections (USE WHEN, PARAMS, RETURNS, ERROR, RELATED). It is front-loaded with the core purpose and then provides exhaustive details. While some verbosity exists (e.g., repeating default values that are already in the schema), the density is justified given the tool's complexity (6 actions, 19 params). A 5 would require even tighter prose; a 3 would be too short. 4 reflects a strong but not perfect balance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 19 parameters, 6 actions, no schema descriptions, and no output schema, the description is comprehensive. It documents all input parameters, return shapes for every action, error handling, and even extra requirements (e.g., metrics requiring the [graph] extra). Nothing an agent needs to correctly invoke the tool is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must explain every parameter. It does: each parameter's type, required condition, valid values (e.g., action values, relation_type enum), defaults, and semantic meaning (e.g., weight as edge strength, valid_at/invalid_at bi-temporal window, metric behaviors like betweenness vs. Adamic-Adar). This is a textbook example of description compensating for schema gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Manage typed relations between knowledge entries.' This is a specific verb and resource. It then enumerates concrete use cases (linking entries, walking the graph, computing metrics) and names two sibling tools (distillery_correct, distillery_find_similar) that it is not, distinguishing it clearly from the related tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'USE WHEN' section explicitly tells the agent when to employ this tool (linking, traversing, metrics) and the 'RELATED' section points to alternatives: distillery_correct for auto-creating 'corrects' relations and distillery_find_similar for discovering related entries. This is explicit when/when-not guidance, leaving no ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_watchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and does so thoroughly. It discloses return payloads for success (list/add/remove) and error codes with structure, explains async behavior (sync_history returns immediately with job_id), soft-delete semantics (purge archives entries), probe behavior (HEAD with GET fallback, INVALID_PARAMS on failure), and even the thresholds constraint (digest <= alert). No contradictions exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long due to the 12 parameters and rich behavioral detail, but it is well-organized with clear sections (USE WHEN, PARAMS, RETURNS, RELATED). Every sentence earns its place, though a few lines (e.g., thresholds rationale) could be tighter without losing value. Still, it remains efficient relative to the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 12 params and no output schema, the description covers everything an agent needs: full return shapes for all actions, error handling, per-parameter constraints, async job linkage, and cross-references to related tools. No information is missing for correct invocation, making it complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero property descriptions, so the description must compensate entirely. It does: every parameter (action, url, source_type, label, poll_interval_minutes, trust_weight, thresholds, sync_history, purge, probe, force, mode) gets a meaning, valid values, defaults, and often context (e.g., why use thresholds for noisy aggregators). This is far beyond a baseline, fully bridging the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Manage monitored feed sources' and immediately narrows scope with 'USE WHEN: listing, adding, or removing RSS/GitHub feed sources'. It distinguishes itself from siblings by mentioning distillery_configure (thresholds) and distillery_store_batch (bulk ingestion) in RELATED, making the tool's unique role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'USE WHEN' section defines the exact conditions for invoking this tool. It also provides exclusions and alternatives: distillery_configure for threshold adjustment, distillery_store_batch for bulk ingestion, and distillery_sync_status for checking background sync progress. The criterion for choosing this tool over siblings is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_classifyA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full behavioral disclosure. It reveals that entries below threshold (default 0.6) go to pending_review, that invalid types trigger a suggestion in the error response, and that successful calls return the full updated entry. It also explains tag merging and project assignment side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with clear sections (USE WHEN, PARAMS, RETURNS, RELATED) and front-loads the core purpose. Each sentence earns its place: parameter explanations, return formats, error codes, and related tools are all information-dense with no repetition or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters, no output schema, and no annotations, the description is remarkably complete. It covers input semantics, success/error return structures, error codes with suggestion details, and names related tools. The only minor omission is whether existing classifications are overwritten, but this is not critical given the overall thoroughness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description compensates fully. For entry_type it lists all valid values and explicitly notes that aliases like 'note' are rejected but the error includes a suggestion pointing to canonical type 'inbox'. For confidence it explains the threshold behavior. All six parameters are described with semantics beyond their schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise verb-resource pair: 'Apply a pre-computed classification to an existing entry.' It then clarifies the intended context (persisting results from LLM/heuristic analysis) and distinguishes this from sibling tools by focusing on classification persistence rather than storage, retrieval, or review.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'USE WHEN' section gives a clear precondition: when you have determined type and confidence and want to persist it. It also names related tools with their specific purposes, distillery_resolve_review for acting on pending_review entries and distillery_list for seeing the queue, effectively telling when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_configureA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states the tool can read or write, specifies the return shapes for read, write, and error cases, and mentions the `disk_written` boolean in the write return, exposing the side effect of persisting changes. It also notes that `value` 'must satisfy type and range constraints for the given key,' implying validation. This is thorough, honest disclosure with no contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every section earns its place: USE WHEN, PARAMS, RETURNS, RELATED. It is front-loaded with the core purpose, then structured so the agent can quickly scan for the information it needs. No redundant filler or restatement of the schema – each line adds new value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no output schema, and no annotations, this description is encyclopedic. It covers when to use the tool, all valid input combinations, the exact return contracts for success and failure (including error codes), and points to relevant sibling tools. An agent has everything it needs to invoke this tool correctly without further inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides zero descriptions (coverage 0%), so the description must compensate – and it does brilliantly. For `section` it lists the four valid dotted-notation paths. For `key` it gives a per-section breakdown of valid keys. For `value` it clarifies optionality and type constraints. This is more actionable than most descriptions, leaving no ambiguity about parameter meaning or allowed inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Read or update a runtime configuration value' – a specific verb-resource pair that defines the tool's scope. It then enumerates valid sections and keys, making the exact target clear. Among the sibling tools (store, ingest, classify, watch, etc.), this is the only one that reads/updates runtime config, so it is unambiguously distinguishable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit 'USE WHEN' clause: 'reading current thresholds/settings, or adjusting them at runtime without editing the config file directly.' It also names related tools in the RELATED line: distillery_watch (to manage feed sources) and distillery_status (to review current system state), giving the agent clear guidance on when to prefer those alternatives. This is textbook usage differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_correctA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: it supersedes the original, preserves audit trail, returns archived_entry_id, and includes a detailed error code list. It also explains inheritance defaults, making side effects and edge cases transparent to the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but efficiently structured with clear headers (USE WHEN, PARAMS, RETURNS, RELATED). The core purpose is front-loaded in a single sentence, and each section adds essential information without redundancy. Bullet-like formatting makes it scannable for an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all operational aspects: purpose, usage conditions, parameter semantics, return values (success and error), and related tools. It also explains inheritance behavior and error codes. No missing information that an agent would need to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description carries the entire burden. It describes every parameter, marks required vs optional, explains inheritance for optional fields, and enumerates valid values for entry_type. This fully compensates for the lack of schema descriptions and exceeds what a schema alone would provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence states a specific verb ('store a correction') and resource ('an existing entry') with clear intent to supersede. The RELATED line explicitly differentiates from distillery_update and distillery_relations, eliminating ambiguity about which tool to call.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides an explicit 'USE WHEN' condition that describes the exact scenario (wrong information, need to replace while preserving audit trail) and names alternative tools for other cases. The RELATED section gives clear routing guidance, so an agent knows exactly when to pick this over its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_find_similarA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden — and it does so comprehensively. It discloses embedding-budget implications ('reuses the entry's STORED embedding (no re-embed, no embedding-budget spend)'), idempotency via the unique (from_id, to_id, relation_type) index, the nuanced excluded_linked_count semantics (counts both linked-source exclusions and self-exclusion), batch-mode edge cases (seed with no embedding maps to empty results, excluded_count best-effort reported as 0), and the full error-code surface. Genuinely rich behavioral context beyond any structural field.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long, but appropriately so for a 10-parameter, three-return-mode tool at 0% schema coverage. It is disciplined in structure: front-loaded purpose, then USE WHEN, PARAMS, RETURNS (single/batch/error), and RELATED. Each sentence carries information — batch embedding reuse, idempotency, excluded_count semantics — with no filler or repetition. The length earns its place given the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 10 parameters, three distinct return shapes, a standalone batch mode, idempotency, and embedding-budget side effects — with zero schema descriptions and no output schema — the description is remarkably complete. It documents every parameter, every return shape including the batch variant and error codes, plus edge cases (missing embeddings, best-effort counts). The minor mention of 'progressive disclosure modes' without elaboration is the only slight gap, and it does not undermine completeness for a tool this complex.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description fully compensates for every one of the 10 parameters. Each param gets purpose and constraints: content's precedence rule ('When both are set, content wins'), threshold's cutoff range (0-1), limit's bounds (1-200), the batch-mode exclusivity rule for source_entry_ids ('cannot be combined... INVALID_PARAMS'), and accept_action's valid enum plus idempotency note. At zero schema coverage, this is maximal compensation — no parameter is left without semantic grounding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with a specific verb+resource statement, 'Find stored entries similar to the given text (cosine similarity),' and immediately distinguishes itself from siblings via the USE WHEN section ('checking for duplicates or conflicts before storing, finding entries related to arbitrary text, or surfacing hidden connections'). The RELATED block explicitly contrasts against distillery_search (natural-language) and distillery_relations (inspecting links), leaving no ambiguity about what this tool is versus its siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides an explicit USE WHEN list with concrete selection scenarios (duplicate/conflict checking before storing, relating arbitrary text, surfacing hidden connections), and names alternatives in RELATED: distillery_store for storing with automatic checks, distillery_search for natural-language queries, distillery_relations for inspecting existing links. Both positive selection conditions and exclusions are articulated; an agent can route correctly without inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_resolve_reviewA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it delivers: it details success response format, error codes, and the no-op behavior (already_in_state, version not bumped, timestamps not rewritten). It also explains the conditional requirement for new_entry_type, giving agents a complete picture of what happens.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with clear sections (USE WHEN, PARAMS, RETURNS, RELATED) and uses bullet points for readability. Every sentence carries needed information; there is no padding. The purpose is front-loaded, and the length is justified by the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, conditional logic, and no output schema, the description is comprehensive. It covers all parameter semantics, return shapes, error scenarios, and related tools. An agent has everything needed to invoke it correctly without additional lookup.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The PARAMS section goes far beyond the bare schema. It lists valid values for action and new_entry_type, marks new_entry_type as required when action='reclassify', and describes reviewer as audit metadata. Since the schema has zero parameter descriptions, this text is essential and fully compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise verb and resource: 'Resolve a pending-review entry by approving, reclassifying, or archiving it.' It explicitly ties the tool to the review queue and low-confidence classifications, clearly distinguishing it from the many sibling tools that handle storage, retrieval, or classification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'USE WHEN' section explicitly states the condition for using this tool (entries with status=pending_review). It also provides alternatives in 'RELATED' by naming distillery_classify and distillery_list with output_mode='review', making the choice unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_searchA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and meets it thoroughly. It discloses the default status candidate set, the graph expansion mechanism (BFS hops, scoring formula parent_score * 0.5 ** depth, provenance tagging, additive merging), the output_mode effects on result shape, and the exact error code list. Nothing about the tool's runtime behavior is omitted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long, but it is extremely well-structured: a one-line purpose, then USE WHEN, then a clearly formatted PARAMS block, then RETURNS (success and error), then RELATED. It is front-loaded with the core semantics. Every section earns its place given the tool's complexity (19 params, graph expansion, multiple output modes); there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 19 parameters, no output schema, and no annotations, the description is fully self-sufficient. It explains every parameter, the exact return envelope shape for each output_mode, the graph_expansion summary fields, and the error contract. It also covers edge cases like archived status handling, published windows, and evergreen inclusion. There is nothing an agent would need to infer or look up elsewhere.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate, and it does comprehensively. Every single parameter is described with its purpose, defaults, and sometimes extra context (e.g., include_evergreen explains the backfill flag and references issue #444, published_after notes it is used by /radar). The PARAMS section adds meaning far beyond the schema's bare types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence states a precise verb (Search), a specific resource (knowledge entries), and the method (semantic similarity using cosine distance, ranked descending). It explicitly differentiates from related tools at the end (distillery_list for filter-based browsing without semantic ranking, distillery_find_similar for comparing against arbitrary text), so an agent can immediately tell this apart from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives an explicit USE WHEN condition ('finding entries that match a natural-language question or topic') and then details the default status filtering behavior with concrete instructions for overriding it via status or include_archived. It also names the two related tools and their purposes, making alternatives clear. No gaps in when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_storeA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It covers return formats for both success and error cases, explains dedup behavior, conflict_prompt inclusion (with size and default rationale), metadata requirements per entry type, expiry semantics, and output_mode effects. It also notes the performance tradeoff of include_conflict_prompt. This is exceptionally transparent for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Though lengthy, the description is highly structured with clear sections (USE WHEN, PARAMS, RETURNS, RELATED). Every sentence provides necessary information—there is no filler. The purpose is front-loaded, and the parameter documentation is organized and scannable. Given the tool's complexity (14 parameters with conditional logic), the length is justified and the layout enhances usability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers everything an agent needs to invoke the tool correctly: all required and optional parameters, enum values, defaults, metadata requirements, return structures for both success and error, error codes, and related tools. With no output schema, it fully documents return values. It even includes edge-case details like default conflict_prompt behavior. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully explain every parameter. It does: each parameter is listed with its type, required/optional status, defaults, valid enums, and special conditional requirements (e.g., metadata schemas per entry_type). This adds immense value beyond the raw input schema, which only provides types and defaults without semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise verb+resource: 'Store a new knowledge entry and return its ID with dedup/conflict information.' It clearly identifies the action and output. The RELATED section names sibling tools (distillery_find_similar, distillery_correct) and states their different purposes, effectively distinguishing this tool from its siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'USE WHEN' section explicitly states the intended usage scenarios (capturing new knowledge like session notes, bookmarks, meeting minutes). It also provides guidance on when to use the summary output_mode for bulk imports and names alternatives (distillery_find_similar for pre-store dedup checks, distillery_correct to supersede an entry), giving clear when-to-use vs. when-not-to-use direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• distillery_store_batchA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With zero annotations, the description carries the full burden, and it delivers: explains batch semantics (no dedup/conflict), per-item failure handling (valid entries persisted, failures reported in 'results'), top-level error conditions, and the return shape. It also mentions budget exhaustion and issue #364, showing up-to-date behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but modularly structured with clear sections (USE WHEN, PARAMS, RETURNS, RELATED). Every sentence adds value—the length is warranted by the tool's complexity. Front-loading the purpose and using headers makes it scannable and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and a complex per-item batch API, the description covers all essential aspects: usage context, parameter specifications, success and error return formats, and related tools. An agent has everything needed to invoke it correctly without ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description provides exhaustive detail for each entry field: content, author, entry_type with valid values, tags, metadata, source, project, plus defaults. It also explains the top-level 'project' parameter. This fully compensates for the sparse schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening line 'Batch-store multiple knowledge entries in one call (no dedup/conflict checks)' clearly states the verb, resource, and key distinguishing feature. It immediately contrasts with single-entry storage, making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Includes a dedicated 'USE WHEN' section specifying exact scenarios (bulk import, migration, backfill) and the trade-off (no dedup, throughput matters). Also names related tools (distillery_store, distillery_watch) and their differentiation, fully routing the agent to the correct alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.