Server Quality Checklist

Tool Scores

• ask-gptA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint, destructiveHint, openWorldHint), the description discloses that it calls an external provider 'via the Codex CLI' with rate limits, and returns 'a text-wrapped JSON envelope { result }.' This adds meaningful behavioral context about side effects and output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences plus a brief note on return format. Every sentence adds value—purpose, expert usage, provider details, rate limit caveat, and output structure. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 6 parameters (all described in schema) and no output schema, the description covers the return format and a key behavioral detail (rate limits). However, it lacks usage context and guidance on when to pick this tool over siblings, leaving the agent without a complete picture for selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds only minimal parameter semantics by mentioning that `expert` applies personas, but does not elaborate on `prompt`, `files`, or other parameters beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides a 'Single-provider second opinion via codex (advisory, single-shot),' specifying the provider and nature. It distinguishes itself from sibling 'ask-*' tools by naming 'codex' as the provider, but does not explicitly contrast with other single-provider alternatives like ask-gemini or ask-one, so it lacks full sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its many siblings (e.g., ask-gemini, ask-all, or expert tools). It mentions that on a named expert tool the persona is ignored, but this is a behavioral note, not a usage guideline. There is no when-not-to-use or alternative recommendation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• consensus-stepA
  3.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description transparently discloses critical behavioral traits: it mutates server loop state, is advisory but mutates on every call, state is ephemeral (lost on server restart), and only dispatch_peers calls external providers. This goes well beyond the annotations (readOnlyHint=false, destructiveHint=false, openWorldHint=true) by adding context about state persistence and external calls.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is packed with necessary detail, but it is somewhat dense and could benefit from clearer structure (e.g., breaking into sections). It front-loads the overall purpose but the subsequent list of actions and constraints is a bit sprawling. Still, every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (10 parameters, no output schema), the description provides a comprehensive overview of the workflow, state management, and expected replies (text-wrapped JSON envelope). It covers edge cases (sessionId required after init, convergence loop) and parameter constraints per action. However, it does not explain return values in detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaningful context beyond the schema by grouping parameters by action (e.g., 'init only', 'record_blind only') and explaining semantics like 'blindVerdict: your pre-commit verdict text, written before the panel is revealed'. This helps the agent understand usage constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool implements a consensus loop with a specific sequence of actions (init, record_blind, dispatch_peers, submit_adjudication, submit_revision). It explains the role of the host model as arbiter. However, while it distinguishes the tool from abstract siblings like 'consensus' by detailing the step-by-step protocol, it doesn't explicitly contrast with related tools in the sibling list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies the required order of actions and that only one action per call is allowed. It implies usage context (client-driven loop) but does not provide explicit guidance on when to use this tool vs alternatives like the simpler 'consensus' tool. No exclusions or when-not-to-use scenarios are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask-grokA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds context beyond annotations: it labels the tool as 'advisory, single-shot' and notes external provider call, API key requirement, rate limits, and return format (text-wrapped JSON envelope). No contradiction with annotations (readOnlyHint=true, destructiveHint=false, openWorldHint=true).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with only two sentences, front-loading the core purpose. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters and no output schema, the description covers purpose, behavior, parameter details, and external dependencies. It lacks explicit error handling but is otherwise complete for a well-annotated tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3. The description adds meaning: it explains that 'expert' applies a persona and is ignored on named expert tools, 'developerInstructions' overrides persona, 'reasoningEffort' applies only to certain providers, and 'files' attachment behavior per provider.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is a 'Single-provider second opinion via grok (advisory, single-shot),' identifying the provider and the one-time advisory nature. It distinguishes from sibling tools by specifying 'single-provider' and 'grok,' and mentions 'expert' personas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for a 'second opinion' but does not explicitly state when to use this tool versus alternatives like 'ask-gpt' or 'ask-gemini.' It mentions external dependencies (XAI_API_KEY, rate limits) but lacks explicit when-not or alternative tool comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• debuggerA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the tool 'fans out to the configured provider panel' with a specific persona, that each provider needs key/CLI and rate limits apply. This adds behavioral context beyond the annotations (readOnlyHint=true, destructiveHint=false, openWorldHint=true). No contradictions with annotations; the description provides useful transparency about external dependencies and response format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: first clearly states the purpose and output, second provides usage guidance and behavioral notes. It is concise with no unnecessary words, effectively front-loading the key information. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (debugging analysis using external providers), the description covers purpose, usage, behavioral notes (provider dependency, rate limits), and output format. It mentions return structure (JSON envelope with results array) but does not detail the results schema, which is acceptable without an output schema. Generally complete for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 6 parameters with 100% description coverage, so the schema already explains each parameter. The description does not add new meaning beyond what the schema provides (e.g., it doesn't elaborate on how 'prompt' should be structured for best results). Baseline score of 3 is appropriate since the schema carries the burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is a debugging specialist that produces ranked root-cause hypotheses and small safe fixes from bug reports, logs, and code. It specifies the output format (JSON envelope with results array) and when to use it (crashes, failing tests, wrong output). This distinguishes it from sibling tools like 'code-reviewer' or 'researcher', which have different focus areas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use for crashes, failing tests, or wrong output', providing clear usage context. It does not explicitly state when not to use, but the tool's name and specialization implicitly exclude other tasks. It mentions the tool returns 'says honestly that the evidence shows no bug', which guides appropriate usage. No direct comparison to alternatives, but the scope is reasonably defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• plan-reviewerA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and non-destructive behavior. The description adds value by disclosing that it fans out to a provider panel, requires keys/CLI, has rate limits, and returns a JSON envelope with results. This goes beyond the annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively concise (2 sentences plus a brief return format note). It front-loads the purpose and usage. The technical details about fan-out and return envelope could be slightly more structured but do not hinder readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description mentions the return format as 'text-wrapped JSON envelope { results[] }', which is helpful. However, it lacks details on error handling, provider configuration, and edge cases. Given the tool's complexity (fan-out, multiple providers), more completeness would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear parameter descriptions. The description adds contextual meaning, such as the 'expert' parameter being ignored on named expert tools and the persona being advisory. This supplements the schema without redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a plan reviewer that validates implementation plans before building. It uses specific verbs ('verifies', 'validate') and distinguishes itself from siblings like 'architect' or 'code-reviewer' by focusing on plan review before execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises using the tool 'to validate an implementation plan before starting significant work', providing clear context. However, it does not explicitly state when not to use it or mention alternative tools for other phases of the development process.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scope-analystA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Disclosed behaviors include 'fans out to configured provider panel', 'advisory persona', 'each provider needs its key/CLI, rate limits apply', and 'returns text-wrapped JSON envelope { results[] }'. This adds context beyond annotations (readOnlyHint, openWorldHint) without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences only: first sentence states purpose and use condition, second explains behavior and output. Front-loaded, no redundancy, every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description covers the return envelope shape. It also mentions provider dependencies. However, the structure of results[] is not detailed, but acceptable given the tool's advisory role.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description does not add any meaning or usage guidance for individual parameters beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is a 'pre-planning consultant' that 'catches ambiguities, hidden requirements, and pitfalls before planning begins.' It specifies when to use ('vague or could be interpreted multiple ways'), distinguishing it from sibling tools like architect or plan-reviewer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Use when a request is vague or could be interpreted multiple ways.' Provides clear context but does not explicitly mention when not to use or suggest alternative sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• panelA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds value by confirming it is 'local and read-only (no provider calls)' and describing the return envelope, though annotations cover the safety profile well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero waste. First sentence delivers purpose and behavior, second gives usage guidance. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-param optional tool with rich annotations, the description covers purpose, behavior, return format, and usage flow. No gaps given the context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description repeats the schema text for parameters without adding new meaning or syntax details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it 'returns the names of the providers... WITHOUT calling them', distinguishing it from ask-all. It identifies the specific verb and resource (discover panel) and differentiates from siblings like ask-one.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'Use this to discover the panel, then issue one ask-one call per provider in parallel'. Provides clear context but lacks explicit exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session-revisitA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate non-read-only and non-destructive, but the description adds important behavioral details: it persists a linked child record, requires persistence, and returns a JSON envelope or error. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with four sentences, each adding value: core action, requirements and process, return format, and error case. Front-loaded with the essential purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (2 parameters, no output schema), the description covers all necessary context: the action, prerequisites, side effects, return format, and error conditions. Sufficient for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description does not add new information about parameters beyond what the schema provides; the schema already includes good descriptions for both sessionId and cwd.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Re-run a persisted session's ORIGINAL question with the CURRENT providers/config'), the resource ('persisted session'), and differentiates from siblings like session-get by mentioning linking via parentId and the requirement for persistence.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use (to re-run a session with current config) and notes prerequisites (sessions.persist must be on), but does not explicitly contrast with other tools in the sibling list beyond stating it uses the original tool path.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyzeA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint: true), the description adds that the tool is local, makes no provider calls, and writes nothing. It also reveals the return format (text-wrapped JSON envelope) and the slash command rendering. This provides comprehensive behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph of four sentences, front-loaded with the main action. It is concise but uses technical terms that may be dense. Every sentence adds value, but a more structured format (e.g., bullet points) could improve readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (two lenses, advisory suggestions) and the absence of an output schema, the description adequately explains the return format, prerequisites, and constraints. It covers all essential aspects for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters. The description provides a brief paraphrase ('how many recent session records', 'tail size of debug log') but does not add substantial new meaning beyond the schema descriptions. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes recent runs from debug log and session store to return advisory tuning suggestions. It specifies the two lenses (timing and agreement) and their independence, which distinguishes it from generative tools like ask-all. The purpose is unambiguous and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions a prerequisite (debug.enabled for the timing lens) and implies usage context for making tuning decisions. However, it does not explicitly state when to avoid this tool or provide alternatives among the sibling tools. The context is clear but lacks exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask-oneA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly, non-destructive, and open-world. The description adds behavioral details about external API calls, key/CLI requirements, rate limits, and the return envelope format with error handling. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat lengthy but every sentence adds information. It front-loads the core purpose and provides structured details. Minor redundancy could be trimmed, but overall it is clear and well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 parameters, no output schema, and moderate complexity, the description adequately covers all parameters, return format, error cases, and usage notes. It lacks an explicit list of return fields but is sufficient for an advisory tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions. The description adds value by explaining the provider parameter with examples, clarifying the expert parameter's behavior with named tools, and describing file attachment modes per provider type. This goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides a second opinion from ONE named provider in the active panel, lists example provider names, and distinguishes from sibling tools like ask-all and provider-specific tools. It also notes the advisory, single-shot nature.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises issuing N calls in parallel for multiple providers and clarifies the need for provider keys/CLI and rate limits. It also explains the return format and error case. While it doesn't explicitly state when not to use, it provides sufficient contextual guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask-openrouterA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds context that it calls external provider, requires API key env, and has rate limits. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences covering purpose, key feature, technical requirements, and return format. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, external dependency, return format, and key parameters. Could mention what the result envelope contains but sufficient for a simple call.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds value by explaining that expert persona is ignored on named expert tools and that files attachment works differently.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it's a single-provider second opinion via openrouter, advisory and single-shot. Distinguishes from sibling tools like ask-all and specific expert tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Indicates advisory, single-shot usage and mentions external dependency with API key. Implicitly contrasts with multi-provider tools but lacks explicit when-not-to-use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• code-reviewerA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds beyond that by revealing the tool fans out to external providers (each needing keys/CLI and subject to rate limits), returns a text-wrapped JSON envelope with results[], and operates in an advisory persona. This informs the agent about dependencies and output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at three sentences, each serving a distinct purpose: stating the persona and scope, specifying when to use, and explaining the behavior and output. It is front-loaded and contains no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the moderate complexity (6 parameters, no output schema), the description covers all essential aspects: purpose, usage, behavior, and output format (JSON envelope with results[]). It does not detail error handling or edge cases, but the annotations and schema fill remaining gaps, making it sufficiently complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining that on a named expert tool like this, the tool's own persona wins over the 'expert' parameter and that 'developerInstructions' overrides the built-in persona. This clarifies parameter behavior beyond the schema's field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs code review focusing on bugs, security holes, and maintainability, distinguishing it from style nitpicks. It specifies the action (reviewing a diff or file before merging) and the resource (code), making the purpose immediately understandable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'to review a diff or file before merging.' It also clarifies what not to do ('not style nitpicks'). However, it does not mention when to use alternative sibling tools like security-analyst or architect, missing an opportunity to guide the agent on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• researcherA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true and openWorldHint=true. The description adds valuable behavioral details: it fans out to a provider panel with advisory persona, requires keys/CLI per provider, rate limits apply, returns a text-wrapped JSON envelope with results, and includes evidence and honest unverified flags. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences long, front-loaded with the main purpose, and includes necessary technical details without redundancy. It is efficient but could be slightly more concise by combining some clauses.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple providers, six parameters, no output schema), the description thoroughly covers behavior: provider fan-out, return envelope format, file attachments, reasoning effort, and persona override. It provides sufficient context for an AI agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the input schema already fully describes each parameter. The description adds minimal value for parameter semantics, only mentioning that the expert parameter is ignored on named expert tools. It does not explain parameter details beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a research specialist for external libraries, frameworks, APIs, and open-source code, and specifies concrete use cases like 'how do I use X' and 'why does this dependency behave this way'. This distinguishes it from sibling tools such as architect, code-reviewer, and debugger.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (e.g., best-practice questions) and mentions that on a named expert tool the tool's own persona wins. However, it does not explicitly list when not to use it or recommend specific alternatives among the many sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• security-analystA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant context beyond annotations: it explains the fan-out to providers, advisory persona, provider requirements (keys/CLI), rate limits, and return format as a JSON envelope. This is valuable as annotations only indicate readOnly and openWorld hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with purpose, then use cases, then behavior. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, use cases, behavior, and return format. While it could detail the results structure more, it sufficiently complements the annotations and schema, leaving minor ambiguity about the exact output shape.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already describes all parameters well. The description does not add substantial parameter-specific guidance beyond the schema, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is a security engineer for threat modeling and vulnerability assessment, with specific use cases like auth/authorization changes and security audits. This distinguishes it from sibling tools such as code-reviewer or architect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly lists when to use: for auth/authorization changes, untrusted input handling, new endpoints, or a focused security audit. It does not mention when not to use or alternative tools, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• architectA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behaviors beyond annotations: it fans out to a configured provider panel with an advisory persona, requires each provider's key/CLI, has rate limits, and returns a text-wrapped JSON envelope with a results array. This complements the readOnlyHint (true), destructiveHint (false), and openWorldHint (true) annotations effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (three sentences) and front-loaded with purpose, followed by usage and behavioral details. Every sentence adds value without redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, no output schema, 18 sibling tools), the description is remarkably complete. It covers purpose, usage scenarios, behavioral model, return format, and prerequisites (keys, rate limits). No gaps are apparent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all parameters, so the baseline is 3. The tool description adds context about the advisory persona and fan-out behavior, but does not significantly enhance the meaning of individual parameters beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a software architect for system design, tradeoff analysis, and complex decisions. It lists specific use cases (architecture, API/schema design, multi-service interactions, or after two failed fixes) and distinguishes itself from siblings by noting that the 'expert' parameter's persona is overridden on named expert tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use the tool (e.g., architecture, complex decisions, fresh perspective after two failed fixes). It also indirectly advises against using it when a more specific expert tool is appropriate (since the persona is overridden). However, it lacks explicit 'when not to use' statements, which would improve clarity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask-geminiA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the tool returns a 'text-wrapped JSON envelope { result }' and that it calls an external provider with rate limits. Annotations already indicate readOnlyHint=true and destructiveHint=false, so the description adds value without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences front-loading the purpose and key guidance. Every sentence adds value with no redundancy. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters, no output schema, and annotations providing safety context, the description covers return format, rate limits, and single-shot nature. It does not detail error handling or pagination, but for a single-shot advisory tool this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the schema: it explains that 'expert' applies personas, 'developerInstructions' overrides built-in persona, and that 'reasoningEffort' is ignored for Gemini. Since schema coverage is 100%, the description enriches understanding of parameter behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Single-provider second opinion via gemini (advisory, single-shot)' which includes a specific verb (ask/consult), resource (gemini), and scope (single-provider). It distinguishes from sibling tools like ask-all, ask-gpt, etc. by emphasizing it is only for Gemini.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises passing 'expert' for personas and notes that for named expert tools the tool's own persona wins. It also mentions rate limits and that reasoningEffort is ignored for Gemini, providing context on when to use. However, it does not explicitly state when not to use this tool versus alternatives, though it is implied by the single-provider nature.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session-annotateA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description details the additive, non-destructive nature of the operation and states that it is a local write with no provider calls. This adds valuable context beyond the annotations (which already indicate non-read-only and non-destructive). It also explains error conditions and return format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences that efficiently convey purpose, prerequisites, and return behavior. Every sentence serves a distinct purpose with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (two required params, no output schema), the description provides complete information: it explains the action, prerequisites, side effects (additive local write), error conditions, and return format. No additional context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and both parameters have clear descriptions. The description does not add additional parameter-level detail beyond what the schema provides. As per guidelines, with high schema coverage the baseline is 3, and the description does not increase it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Append'), the resource ('persisted session's audit trail'), and key characteristics ('additive local write, no provider calls'). It distinguishes from sibling tools, which are about analysis, consultation, or retrieval, not annotation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly mentions the prerequisite 'Requires sessions.persist', which provides a clear condition for when to use this tool. It also notes that an error occurs when persistence is off. However, it does not provide explicit alternatives or when-not-to-use scenarios, which would elevate the score to 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session-getA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false. The description adds details about being local, no provider calls, error cases, and that cwd is ignored, which goes beyond annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action, then return format and error handling. No wasted words, every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple fetch tool, the description covers prerequisites, behavior, return format, and error scenarios. With good annotations and no output schema needed, it is fully informative.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining that cwd is irrelevant for session-get and clarifying the role of sessionId, enhancing understanding beyond the schema text.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the action (Fetch), resource (persisted consensus/ask-all session record), and method (by id). It lists the contents (opinions, verdict, arbiter, annotations) and prerequisites, making the purpose unambiguous and differentiated from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states it is read-only, local, and requires sessions.persist, giving context on appropriate usage. It lacks explicit comparison to siblings like session-annotate or session-revisit, but the read-only nature and mention of ignoring cwd provide implicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask-allA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses external provider calls, key/CLI requirements, rate limits, return format (JSON envelope with results/omitted), and session persistence condition. Adds significant value beyond readOnlyHint annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two well-structured sentences front-load the core action and include all key caveats without superfluous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters, no output schema, and complex multi-provider behavior, the description adequately covers purpose, behavior, output format, and configuration details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all 6 params (100% coverage). Description adds context: expert applies persona, reasoningEffort ignored by some providers, cwd resolution, and default persistence behavior. Enhances understanding beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool fans out a question to multiple LLM providers (GPT, Gemini, Grok, OpenRouter) in parallel for independent second opinions, distinguishing it from single-provider siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Indicates use for independent second opinions and advisory results. Implicitly contrasts with single-provider tools via sibling names, but lacks explicit when-not or alternative references.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• consensusA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses important behavioral traits: calls external providers with rate limits, returns a text-wrapped JSON envelope with split verdict/synthesis, and persists session only when enabled. No contradiction with annotations (readOnlyHint, destructiveHint, openWorldHint).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense with information and front-loaded with the core purpose. Every sentence provides necessary context. The NOTE at the end is somewhat audience-specific but does not detract significantly. Could be slightly trimmed, but overall efficient for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 parameters, no output schema, and moderate complexity, the description covers purpose, usage, parameters, behavior, and return format (JSON envelope). It is self-contained and eliminates ambiguity for the agent invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all 8 parameters. The description adds value beyond schema by explaining default behavior (e.g., maxRounds default from config, synthesizeAlways overrides loop, expert persona overriding, file delivery modes). Reduces cognitive load for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it runs a 'multi-round consensus convergence loop server-side' and returns a converged verdict. It distinguishes from sibling tool 'consensus-step' via the NOTE about using the /consensus slash command for the host-arbiter loop, and from other single-provider tools by the loop mechanism.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly describes when to use the full loop vs. the single synthesis pass (synthesizeAlways), and advises on open questions vs. plan convergence. Also mentions the alternative /consensus slash command for host-arbiter loop, providing clear selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.