Server Quality Checklist

Tool Scores

• chamber_verifyA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds behavioral detail beyond the annotations by explaining that the tool reports sources that no longer exist or whose text changed, and that it does not change conclusions. It explicitly states 'Read-only' matching annotations and clarifies the meaning of results, providing useful context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, with the main action front-loaded. The second sentence adds conceptual context without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This tool has a simple optional param, no output schema, and annotations cover read-only/idempotent. The description explains what the tool reports and the meaning of results, making it complete for an agent to understand when and how to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes the only parameter 'since' with its purpose and default behavior. The description adds no additional parameter information, but since schema coverage is 100%, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool re-checks stored beliefs' pinned sources against the current corpus and reports those whose evidence moved, specifying outcome types (not_found, hash_mismatch). It identifies this as drift detection, which distinctively separates it from sibling tools like chamber_ask and chamber_corpus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that the tool is for drift detection, providing clear context for when it should be used. However, it does not explicitly state when not to use it or mention alternatives, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chamber_askA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite annotations indicating no read-only guarantee (readOnlyHint: false), the description goes beyond by explicitly warning 'this WRITES' and detailing the commit-gate effects: claims with verified citations are recorded as beliefs with pins, unsourced assertions mint citation debt, and spend is recorded. It also explains what it cannot do (bypass gate, activate skill, approve pending write). This is rich behavioral disclosure beyond the annotation flags.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but each sentence contributes unique information: purpose, citation judging, reference format, corpus-only scope, and the write behavior. It is front-loaded with the main action. The note about 'exactly as `chamber ask` does' is slightly redundant but clarifies equivalence. Overall, it is well-structured for the complexity, though a bit verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description fully explains the return format (ALLOWED/UNSUPPORTED claims with file#passage references) and the side-effects of the write. It covers the tool's scope (corpus-only), its limitations (cannot bypass gate, etc.), and its relationship to verification. This is complete for a complex write tool with several behavioral nuances.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% coverage for all four parameters with clear descriptions. The tool description adds no additional parameter-specific semantics beyond the schema, except indirectly noting the commit gate implications for the strict parameter (not explicitly). With high schema coverage, the baseline of 3 is appropriate; the description does not need to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Ask a question of the local Chamber corpus and get an answer...' This clearly differentiates the tool from siblings by focusing on question-answering with citation-based claims. It also states the output format (ALLOWED/UNSUPPORTED claims with file#passage references), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes its usage context clear: it is the tool to ask questions of the indexed corpus and receive cited answers. It also hints at the relationship with chamber_verify by noting that verified claims are 'what chamber_verify later checks for drift,' implicitly distinguishing answer generation from verification. However, it does not explicitly say 'use this when you want to ask a question' or provide direct when-not-to-use guidance for siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chamber_corpusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds behavioral detail beyond the readOnlyHint and idempotentHint annotations by specifying that the tool identifies files 'far above the median passage count' and interprets them as 'the signature of an export rather than a note.' It also explicitly states 'Read-only,' reinforcing the annotation without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first enumerates the tool's report contents, the second gives usage guidance and a safety note. Every clause adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no parameters or output schema, the description thoroughly sets expectations: it lists the report contents, explains the outlier heuristic, and tells the agent when to invoke the tool. The 'Read-only' statement also confirms safety, making this a self-contained description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, and the input schema already reflects this with an empty properties object. The description doesn't need to explain parameter usage, so the baseline of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Report what is actually in the index'—a specific verb and resource—and then enumerates the exact report contents (counts, source kinds, top folders, outliers). This clearly differentiates it from the sibling tools chamber_ask and chamber_verify, which are about querying rather than inspecting the corpus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides an explicit trigger for use: 'Use this before trusting a "nothing matches" answer.' This is a clear context. However, it doesn't mention when not to use it or reference alternatives by name, so it's one step below full guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.