Server Quality Checklist

Tool Scores

• memory_statsD
  1.8/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It merely lists output fields and does not state that it is a read-only operation, describe any side effects, or explain the output format. The agent cannot infer safety or side effects from this description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single fragment, not a full sentence, and omits the verb. While it is short, it is under-specified and lacks the structure of an effective description. The fragment does not earn its place because it fails to communicate the tool's action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, no parameters, and no annotations, the description is minimal. It lists fields but does not explain what 'substrate size', 'entities', 'sources held', or 'mode' mean, nor does it state the purpose or expected usage. An agent has insufficient information to invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the baseline is 4 per the rubric. The description does not need to add parameter meaning because none exist. However, it also does not enrich the tool's overall semantics beyond the schema, so no higher score is warranted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description is a noun phrase listing data fields ('Substrate size, entities, sources held, and mode') without a verb or explicit resource. It does not state that this tool retrieves or returns these statistics, so the agent cannot be sure what action the tool performs. It also does not differentiate this from sibling tools like recall or export_memory.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus the sibling tools. No context about prerequisites, conditions, or typical scenarios (e.g., checking memory state before an operation) is provided. The agent is left entirely to guess when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_sourceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure, yet it discloses almost none. It does not state what happens if source_id already exists, whether the operation is idempotent, what side effects occur, or what the response contains. The purpose is described, but not the behavioral consequences of calling the tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with minimal waste, and it front-loads the primary action ('Register a source document') before the purpose clause. The sentence is slightly convoluted and abstract, but it earns its length and is appropriately compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With zero schema description coverage, no annotations, and no output schema, the description must carry the full explanatory burden but leaves too much unstated. Key operational details — what text should contain, how source_id is used, what 'verifiable byte ranges' means in practice, and what happens on duplicate registration — are absent, making the tool hard to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description must compensate, but it does not. The schema has two bare string parameters (source_id and text) with no descriptions, and the description never explains what 'text' represents (the document content?) or what 'source_id' is (an identifier, a URL?). For a registration tool with an abstract 'verifiable byte ranges' concept, this is a real gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ('Register a source document') and explains the purpose (bind claims to verifiable byte ranges). It is not a tautology of the name and the concept of registering a source before claiming against it is reasonably distinguishable from the sibling tools. However, the 'verifiable byte ranges' phrasing is abstract and leaves some ambiguity about what exactly the tool accomplishes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is only implied, not explicit. The phrase 'so claims remembered against it bind' suggests one registers a source before remembering claims against it, which hints at sequencing relative to the sibling 'remember' tool. But there is no explicit statement of when to use this tool versus alternatives, no prerequisites, and no exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_memoryB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the blob is portable and readable without the server or model, which is useful behavioral context. However, it does not mention side effects (e.g., whether the export is read-only), output size, error handling, or any performance implications, leaving gaps for an agent assessing risk or expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two short sentences that immediately state the action and the key differentiator (portability and independence). Every word earns its place, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and no output schema, the description is missing critical details: the purpose of 'path' and what the output actually looks like beyond 'portable open-format blob'. It does not explain the return format, size limits, or any requirements for calling it, leaving the agent to guess.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema defines a single 'path' parameter with no description, and the tool description makes no mention of it at all. With 0% schema description coverage, the description completely fails to clarify what the path parameter means or how to use it, providing no added value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action (export) and resource (the whole substrate) as a portable blob. It is distinct from sibling tools like remember or recall by its export operation, but it does not explicitly name an alternative or contrast it with siblings, so it falls short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when you need to export the entire substrate, but it gives no explicit guidance about when to use it over alternatives or when not to use it. There is no mention of prerequisites, permissions, or contexts where another tool would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recallA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does real work: it discloses the return shape (grounded claims with byte ranges) and, critically, the abstention behavior — telling the agent to treat an ungrounded-term disclosure as a real answer, not empty results. This prevents a serious misinterpretation. It stops short of covering error conditions or permission needs, but the abstention quirk is the highest-value disclosure for this tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, front-loaded with the core purpose first, then return format, then behavioral caveat. Every sentence earns its place with no filler. Slightly dense but tightly constructed — no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter search tool with no output schema, the description covers the essential behavioral contract (search, grounded return format, abstention semantics). However, it is vague on what 'the substrate' is, and beyond the byte-range note it gives no sense of result volume, ordering, or pagination — minor gaps for such a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the undocumented query parameter — and it does not. Nothing clarifies query format, length, phrasing guidance, or how a good query is constructed. The single parameter's meaning is somewhat self-evident from 'Search memory,' but the description adds no value beyond the schema itself.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb-resource pair ('Search memory') that cleanly separates it from every sibling (remember, add_source, verify_receipts, forget, export_memory, memory_stats — none of which search). The byte-range grounding detail further differentiates it as a retrieval-with-evidence tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The verb 'search' implies the tool is for retrieval, which is enough to distinguish it from mutation siblings like remember and forget. But the description never explicitly says when to use it over alternatives or states any exclusions — usage guidance is only implied, not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_receiptsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It does reveal a key property: tamper-evidence (changing a source breaks receipt verification). However, it does not state whether the operation is read-only or mutating, whether it can fail, or what output it produces. The disclosed property is useful but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The primary action is front-loaded, and the tamper-evident detail is added efficiently. Every sentence contributes meaningful information, earning a high score for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no parameters and no output schema, so the description must convey enough context for correct invocation. It explains the action but omits side effects, error conditions, and return values. It also does not place the tool in the broader workflow (e.g., when to verify receipts). This leaves some ambiguity about the expected behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, and schema coverage is trivially 100% given the empty schema. Per the rubric, 0 parameters earns a baseline of 4. The description adds no parameter-specific detail, but none is needed since there are no parameters to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('re-hash every held source and re-slice every receipt') and a resource (sources and receipts). The tamper-evident note adds a distinctive property that separates it from siblings like remember and recall. It is not a tautology and clearly communicates what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to use this tool versus the sibling tools (e.g., after adding a source, before exporting memory). It does not mention prerequisites or exclusions. An agent is left to infer the appropriate invocation context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rememberA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden, and it delivers: it discloses that admission is conditional on evidence, that refusals are typed and actionable, and that the tool is a write operation ('commit'). The schema adds the byte-range binding and receipt behavior. Gaps: it does not describe what a successful call returns (the receipt format) or explicitly state the response envelope beyond typed refusals.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences, zero wasted words. The primary verb is front-loaded, the gating mechanic follows immediately, and the verbatim constraint closes it out. Every clause earns its place and no structured-field content is redundantly repeated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool of moderate complexity — a gated mutation with typed refusals and byte-range receipt binding — the description plus schema covers the essentials: purpose, evidence requirement, failure mode, and the optional source_text binding. The main omission is the success response shape (what the receipt actually is on the happy path), which is hinted at in the schema but never fully described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds real interpretive value beyond the schema: framing the caller as 'proposer' subject to a 'deterministic gate' explains WHY claim and evidence matter and how they interact, and the verbatim emphasis reinforces the constraint on the evidence parameter. This contextual rationale is more than the schema's terse field descriptions provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with a specific verb+resource construction — 'Commit a fact to memory' — and immediately distinguishes itself from the sibling set: it is not recall (reading), not forget (deleting), not verify_receipts (checking), and not add_source (storing a source document). The added detail about the deterministic gate and verbatim evidence requirement makes the tool's identity unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool — when an agent holds a fact supported by verbatim quoted evidence — and the gate/verbatim constraints effectively rule out using it for paraphrased claims. However, it never explicitly names alternatives or states when-not-to-use, leaning entirely on implicit context. No exclusion guidance such as 'for uncited facts use add_source first' is present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forgetA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It goes well beyond a simple 'erase' by describing the output as a SIGNED CERTIFICATE, the requirement that a probe battery must all abstain, and that bystanders are left intact. This gives the agent a strong understanding of what to expect and the guarantees provided. Minor gaps: no mention of reversibility, permissions, or side effects, but for an erasure tool these are adequately covered by the explicit guarantees.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action and the unique deliverable (signed certificate). It packs all essential details — closure, probe battery, bystanders — into a compact, scannable format without any filler words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter and no output schema, the description fully explains what the tool does, what it returns (the signed certificate and its guarantees), and the trigger context. There is no missing information an agent would need to decide to call this tool and understand its result. The description is self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one required string parameter 'subject' with 0% description coverage. The tool description contributes the meaning that 'subject' is the entity to be erased, which is essential. However, it does not specify the expected format (e.g., name, ID, or any canonical form) or constraints beyond being a string. Given the simplicity of the parameter, the addition of meaning earns a 3, but it could be more explicit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action (erase), the resource (subject), and the unique guarantee (issue a SIGNED CERTIFICATE with exact closure, probe battery all abstain, bystanders intact). It clearly differentiates from siblings like recall or verify_receipts by explicitly framing this as the artifact for a 'delete me and prove it' request.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear use case: 'delete me and prove it' requests. It implicitly guides the agent to use this tool when deletion with verifiable proof is needed, and by contrast other tools (remember, recall, etc.) serve other purposes. However, it does not explicitly name alternative tools or discuss when not to use it, so it falls short of a perfect 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.