Server Quality Checklist

Tool Scores

• evidence_append_lessonC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description bears full responsibility for disclosing behavioral implications. It mentions 'append' which is a mutation, but does not disclose side effects, idempotence, or any constraints on the ledger.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler or redundant information, making it highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has four parameters and no output schema, yet the description omits parameter meanings, return behavior, and error conditions. The minimal detail may leave an AI agent uncertain about how to correctly invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, and the description does not explain the meanings of subject, lesson, verified_evidence, or source_refs. The agent must infer semantics from parameter names alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies the verb 'append', the resource 'evidence-backed lesson', and the destination 'configured ledger', making the primary function clear. However, it does not explicitly differentiate from sibling tools like evidence_create_note, though the resource type differs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no instructions on when to use this tool in preference to siblings such as evidence_read or evidence_search. No prerequisites, exclusions, or alternative scenarios are mentioned, leaving the agent without explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evidence_searchB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of behavioral disclosure. It notes the use of 'optional local embeddings' and 'deterministic lexical fallback', which adds insight into how the search operates. However, it does not disclose return format, permission requirements, or whether the search is purely read-only, though 'Search' implies it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single compact sentence that front-loads the primary action ('Search allowlisted local notes') and adds relevant implementation detail. Every word earns its place; no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has only two simple parameters and no output schema. The description covers the core functionality and even adds fallback behavior, but omits any information about return values, result ordering, or error conditions. For a search tool, this is moderately complete but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description does not mention any parameters. The 'query' parameter is implicitly required, but 'max_results' is not addressed. The description adds no value beyond the schema, leaving parameter semantics under-explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Search' and the resource 'allowlisted local notes', making the tool's function clear. It doesn't explicitly distinguish from sibling tools like evidence_read, but the search intent is evident from the name and resource description.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives such as evidence_read or evidence_append_lesson. The description doesn't mention contextual signals or exclusions, leaving the agent to infer usage from the tool's name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evidence_create_noteB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses that the tool never overwrites existing notes, which is a key behavioral trait. However, it does not explain what happens on duplicate slugs, permission requirements, or the return value, missing important details for a creation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently communicates the core purpose and a critical guarantee. No wasted words; it is appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with eight parameters and no output schema, the description is too sparse. It omits field meanings, on-conflict behavior (beyond 'never overwrite'), and return details, leaving agents without enough context to invoke it reliably. Sibling tool names help, but the description itself under-delivers.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero coverage, and the description does not mention any of the eight parameters (e.g., slug, summary, decision, next_actions). The agent gets no additional semantic context for required fields, making it difficult to construct valid inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create a new evidence note') and resource, and adds a distinguishing guarantee ('never overwrite'). This separates it from siblings that read, search, or append lessons, providing a specific and unambiguous purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies creation context via 'Create a new,' which distinguishes it from read/search/append siblings, but it doesn't explicitly state when to use this tool vs alternatives or mention exclusions. There is no mention of when not to use it or references to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evidence_readA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It mentions redaction of credentials and challenge flags, which is valuable, but does not cover error handling, path validation, or what happens if the note is not allowlisted. The description provides some behavioral context beyond the raw schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the verb and resource, with no redundant phrases. Every word adds meaning, making it an exemplary model of conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with one parameter and no output schema, the description covers the essential functionality and important behavior (redaction). While it could mention error conditions or path resolution, the tool's simplicity means this level of detail is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter, relative_path, with no description and 0% coverage. The tool description implies the path identifies the note but does not explicitly explain its format, what 'relative' refers to, or allowlist criteria. The parameter name is self-explanatory, and 'allowlisted' adds a constraint, providing marginal compensation for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool reads a note, with specific qualifiers (allowlisted UTF-8, redaction) that distinguish it from sibling tools like evidence_create_note, evidence_search, and evidence_append_lesson. The verb 'Read' and resource 'note' make the purpose immediately clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that this tool is used for reading existing notes by relative path, but it does not explicitly state when to use it versus alternatives. Sibling tool names provide context, but the description itself offers no guidance such as 'use evidence_search to find notes before reading.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evidence_statusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. The verb 'show' indicates a non-destructive read operation, which is a useful behavioral cue. However, it does not disclose other potential behaviors such as authentication requirements, data source details, or what happens if no configuration exists. For a zero-parameter status tool, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence with no filler. It front-loads the action ('Show') and specifically lists the three configuration aspects covered. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (no parameters, no output schema, no annotations), the description provides sufficient context: it states what information is shown. It could ideally clarify the output format or whether it returns a summary, but for a simple status tool, the description is nearly complete. Minor gap is absence of return-value details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is trivially covered. Per baseline, 0 params rate 4. The description adds no parameter-specific meaning because there are none to describe.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: to show configured evidence boundary, retrieval mode, and write policy. It uses a specific verb ('show') and resource ('configured evidence...'), and it distinguishes from siblings like evidence_read (which likely reads evidence content) and evidence_create_note (which creates notes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (when needing to inspect current configuration) but provides no explicit guidance on when not to use it or alternatives. There are no exclusions or comparisons to sibling tools, so usage context is inferred rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.