Server Quality Checklist

Tool Scores

• query_intelligenceA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the burden. It discloses the output format (keyword-matched results with timestamps and excerpts) and the LEDGER growth from report_outcome. However, it does not explicitly state that the tool is read-only or that it has no side effects, relying on the word 'search' to imply this.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences front-load the purpose, then add useful behavioral context without any fluff. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple 2-parameter schema and no output schema, the description adequately explains the data source, result format, and data population via report_outcome. It lacks explicit read-only status and ordering/pagination details, but these are minor given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context about the search target (LEDGER and Wisdom Codex) and result content, but does not add syntax or format details for query and limit beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the Boardroom LEDGER and Wisdom Codex for precedents, past decisions, and insights, with a specific verb and resource. This distinguishes it from siblings like report_outcome (which writes) and analyze (which likely analyzes current data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives is provided. The description implies use for searching past decisions but does not mention check_governance, trust_lookup, report_outcome, or analyze, nor any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyzeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals the tool's multi-step internal behavior (routes, loads, searches, provides structured analysis) and important context like mandatory tension and demo/full mode differences. This goes beyond the bare 'analyze' name and gives a good sense of what the tool does internally, though it stops short of stating side effects or return structure explicitly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with 'Run a Boardroom consultation.' The first sentence efficiently lists the core actions, and the second adds relevant mode distinctions. Every sentence contributes value with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one well-documented parameter and no output schema, the description is largely complete. It explains the internal process and demo/full protocol modes, giving sufficient context for an agent to invoke it. However, it does not describe the return value structure beyond 'structured analysis,' which is a minor gap for such a high-level consultation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%: the only parameter 'task' is described as 'The decision, question, or task to analyze.' The description's mention of 'your question' adds no new semantic detail beyond the schema. Since the schema already fully documents the parameter, a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Run a Boardroom consultation' and enumerates the specific steps (routing questions, loading philosophies, searching precedents, structured analysis). This is a specific verb+resource with enough detail to distinguish it from sibling tools like check_governance or query_intelligence.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for decisions/questions and describes the internal routing process, but it does not explicitly state when to use this tool versus alternatives or provide exclusions. The phrase 'your question' suggests the intended input, but no clear 'use this when' guidance is offered.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_governanceA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the return values and the 'fast classification' behavior, but does not explicitly state whether the tool is read-only, requires authentication, or has any side effects. It adds some behavioral context but leaves room for ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the primary action, and every sentence adds value. It lists outputs and a key differentiator (speed) without unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (one parameter, no output schema), the description is fairly complete: it states the purpose, lists the return types, and notes the speed advantage. It would benefit from explicit guidance on usage versus alternatives, but overall it covers the essential context for this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'task' is already described in the schema with the same phrasing ('The task or decision to classify'). The description adds little beyond the schema, but it does imply that the task is used to produce governance outputs. With 100% schema coverage, no additional compensation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool classifies a task and determines governance advisors for review, with a specific list of outputs (decision type, advisors, risk level, constitutional constraints). It uses a specific verb ('classify') and resource ('governance advisors'), distinguishing it from siblings like query_intelligence or analyze.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context on when to use this tool ('Fast classification without running the full session'), implying it is a quicker alternative to a full governance session. However, it does not explicitly name alternative tools or state when not to use it, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• report_outcomeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing side effects (feeds Knowledge Flywheel, updates Trust Oracle if entity specified) and a failure condition (returns warning if outcome could not be persisted). It does not cover auth, rate limits, or idempotency, which are useful but not essential for this tool's core behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences, front-loaded with the main purpose and then adding side effects and error behavior. Every sentence provides distinct, necessary information with no redundant fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, no annotations, and no output schema, the description covers the core purpose, side effects, entity-conditional behavior, and failure warning. The only minor gap is that it does not describe the success return value, but this is not critical for invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description adds some context (e.g., entity triggers Trust Oracle update, followedRecommendation corresponds to whether the original recommendation was followed), but it does not significantly expand on the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Report') and resource ('outcome of a decision for the Boardroom learning system'). It distinguishes itself from sibling tools like check_governance and trust_lookup by emphasizing the write/reporting nature and its side effects on the Knowledge Flywheel and Trust Oracle.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: to record a decision outcome, what happened, and whether the recommendation was followed. It also explains conditional behavior with entity. However, it does not explicitly mention exclusions or alternative sibling tools, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• trust_lookupA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It explains the return format (6-dimension vector, composite score, recommendation), and importantly discloses that new entities return a default 'unknown' profile, which is a key behavior. However, it does not mention any permissions, data source, or potential side effects, though for a read-only lookup these are less critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, front-loaded with the main action, and no filler. Each sentence adds distinct value: what it does, what it returns, and the special case for new entities.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of annotations and output schema, the description provides a thorough overview: purpose, return structure, recommendation values, and the default behavior for new entities. It also names the companion tool for building trust data. No critical missing information for a simple lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions cover both parameters fully (100%), so the baseline is 3. The description does not add significant meaning beyond the schema; it echoes the entity types and context usage but offers no new details like format constraints or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Look up' and identifies the resource as 'trust profile for any entity', enumerating the output components. It clearly differentiates from siblings like report_outcome (building trust) and check_governance (governance checks).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states the primary use case (look up trust profile) and explicitly directs to 'report_outcome' for building trust data, providing an alternative and a when-not scenario. It also implies the tool is for lookup, not for governance or analysis.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.