Server Quality Checklist

Tool Scores

• get_fragmentA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool returns full text and source position, which is useful, but it does not mention error handling (e.g., invalid id), potential empty results, or any permission requirements. For a read-only fetch, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core function, and adds value with the sibling differentiation. There is no redundant or filler text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter fetch tool without an output schema, the description adequately covers purpose, return content, and usage context relative to siblings. It could optionally describe the exact response structure, but that is not essential given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% coverage for the single 'id' parameter, including an example. The description reinforces the parameter's role ('by id') but does not add further semantic details beyond what the schema already provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Fetch one fragment by id') on a specific resource ('a source fragment') and adds what is returned (full text and location). It also distinguishes the tool from siblings by noting that other tools cite fragment ids and this is how to open one.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when you have a fragment id and need to view its content or context. The phrase 'Every other tool cites fragment ids; this is how you open one' provides clear usage context, though it does not name a specific alternative or explicit 'when not to use' scenario.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_gapsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It explicitly discloses the deliberate absence of citations ('returns no citations — there is nothing to cite, which is the point') and describes the content of results (questions, answering role, blockers). This adds meaningful context, though it stops short of declaring read-only semantics or other operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, both concise and front-loaded. The first sentence states the core purpose, and the second explains a key behavioral nuance. Every sentence earns its place without unnecessary filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema and no annotations, the description covers the essential details: what the tool returns (unanswered questions, responsible role, blockers) and a notable behavior (no citations). It could be more explicit about the output format (e.g., array of objects), but it is largely complete for simple list retrieval.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the input schema is complete. Per the baseline for 0 params, a score of 4 is appropriate. The description adds no parameter-specific information because none exist, and none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Questions no source answers, with the role that can answer each and what it blocks.' This clearly explains what the tool does and distinguishes it from sibling list tools like list_sources and list_contradictions, which focus on different aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to find unanswered questions, but it does not explicitly state when to use it versus alternatives like list_contradictions or list_sources. No exclusions or alternative comparisons are provided, so usage guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_brainA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior on its own. It explicitly states the tool returns 'nothing answers that' rather than guessing when the brain has not read the source, and that results include the source. This covers the key behavioral trait of avoiding fabrication.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action, and includes essential behavioral details without any redundant text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description covers the input scope, output format, and empty-result behavior. It could mention potential pagination or ordering details, but for a simple query tool with a limit parameter, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents both parameters with coverage 100%, so the baseline is 3. The description adds meaningful context for the 'question' parameter by specifying the kind of questions allowed (history, decisions, constraints, customer feedback), and clarifies the output format (fragments with sources), which enriches the parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Ask') and identifies the resource ('the product's own history') and scope ('decisions, constraints, what customers said'). It distinguishes from sibling tools like get_fragment and list_sources by focusing on natural-language question retrieval rather than direct retrieval or enumeration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear context for when to use: asking plain-language questions about the product's history. It does not explicitly name alternatives or exclusions, but the context is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• draftA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility. It discloses key behaviors: every line traced to the source fragment, open questions listed below, and refusal when no supporting source. These go beyond the schema and give the agent an accurate mental model of the tool's operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler. The first sentence immediately states the action and key output properties, and the second adds a decisive rejection condition. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description provides sufficient insight into the return structure (traced lines, open questions) and refusal case. It could have elaborated on where sources come from or how the draft is presented, but for a two-parameter tool, this is adequately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes both parameters (kind and topic) with 100% coverage, so the description adds little beyond restating the kinds in prose. It does not introduce new syntax or format details beyond what the schema provides, matching the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Draft') and names clear resources ('PRD section, user stories or acceptance criteria'). It also distinguishes itself from siblings by highlighting traceability to fragments and listing open questions, which none of the sibling tools imply.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys clear context: use this tool to draft from sources with traceability. It also gives an implicit when-not by stating it refuses when no source supports the topic, though it does not explicitly mention alternative tools or exclusion scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sourcesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It clearly implies a read-only listing operation and adds context about the content types and fragment counts. While it doesn't explicitly state 'read-only' or mention ordering/pagination, the behavior is transparent enough for a simple list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first describes functionality, the second explains the use case. No fluff, perfectly front-loaded and concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no params or output schema, and the description covers what it returns and why to use it. It doesn't detail exact return shape or ordering, but that's acceptable for a listing tool with such clear intent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds value by explaining the output (fragment counts), which is informative but not parameter-related.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool lists all sources the brain has read, with examples (interviews, PRDs, tickets, threads, docs) and fragment counts per source. This specific verb+resource is distinct from sibling tools like list_contradictions or list_gaps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage context: 'Use it to see what an answer could possibly be based on before you trust one.' It does not explicitly exclude cases or name alternatives, but the intended use is evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_contradictionsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes on the full burden of explaining behavior. It reveals that the tool outputs quoted sides and a resolution question for each contradiction, giving a clear picture of return content. It does not mention side effects, but 'list' implies a read-only operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, with the core functionality front-loaded in the first sentence and usage context in the second. No unnecessary detail is included, earning a top score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter tool, the description provides sufficient context: it states the inputs (project sources), outputs (quoted contradictions and resolution questions), and a prioritization hint. Despite lacking an output schema, the description covers the essential return structure, making it complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is empty (100% coverage), so parameter description is unnecessary. The baseline for zero parameters is 4, and the description correctly avoids fabricating parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines the tool's purpose: listing contradictions from project sources, including quotes and resolution questions. It distinguishes itself from sibling tools like list_gaps and list_sources by focusing specifically on contradictions, making its role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The statement 'This is usually the highest-value call' provides strong guidance on when to invoke this tool, suggesting it should be prioritized to surface conflicts. However, it does not explicitly mention alternatives or when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.