Server Quality Checklist

Tool Scores

• consult_architectC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It says it 'directly queries' an external AI model, but does not describe the return format, whether the call is synchronous, potential costs, rate limits, authentication requirements, or any side effects. An agent calling this tool would have no idea what to expect in the response.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately names the action and the target. It is concise and to the point, with no filler. It could be considered slightly terse but is appropriately sized for the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool queries an AI model and has no output schema or annotations, the description should clarify what the agent can expect back (e.g., text, structured steps, reasoning). It also fails to connect the tool to its siblings, leaving the agent in the dark about when this is the appropriate consultation route. These gaps make the description incomplete for safe and correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes all four parameters (model, effort, prompt, context) with 100% coverage, so the baseline is 3. The description adds no parameter-specific information beyond what the schema provides, so it neither improves nor detracts from the schema's clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'queries' and the resource 'The Architect (OpenAI gpt-5.6-sol)', and enumerates the kind of output it provides (system decomposition, modular design, API interfaces, structured implementation steps). This is specific and distinguishes it from generic query tools, but it does not reference any sibling tools, so an agent cannot tell how this differs from consult_contrarian or deliberate_council without additional context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus its siblings. The description only states what it does, not under what circumstances it should be chosen over consult_contrarian, deliberate_council, offload_task, or fast_context_reader. An agent is left to guess which consultation tool fits a given request.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• offload_taskB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description bears full responsibility. It discloses that an external worker is used and mentions default models and latency, but it doesn't clarify whether the call is synchronous, what happens on failure, network dependencies, or cost implications. It also doesn't describe the return value or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action and resource, then adds the purpose. It's not overly verbose and avoids redundancy, though it could be slightly more structured by separating the purpose from the resource list.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 5 parameters, no output schema, and no annotations, the description is too thin. It doesn't explain expected output format, error conditions, when to choose specific providers/models, or how the result is returned. An agent would need to inspect the schema or rely on defaults without clear guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all parameters. The description adds no additional meaning beyond what the schema provides; it doesn't explain relationships between parameters or edge-case behaviors, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('offloads') and lists concrete resources ('focused implementation subtask, utility function, unit test suite, regex, or refactoring step'), clearly distinguishing this tool from siblings like consult_architect or deliberate_council, which are about thinking rather than execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states the motivating purpose ('to save host agent context and execution limits') but doesn't explicitly say when to use this vs alternatives, nor when not to use it. It implies usage for small, well-defined tasks but no explicit exclusions or comparison with doing the work inline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• consult_contrarianA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that this makes a direct query to a named external model and describes the nature of the analysis (adversarial, edge-case, race-condition checks). However, it does not mention return format, potential latency/cost of an LLM call, or whether state is modified, leaving some important behavioral context unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that names the action, the resource, and the outcome. There is no filler or repetition; every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter tool with no annotations and no output schema, the description is somewhat thin. It makes the purpose clear and the schema documents parameters, but it does not explain the response shape or provide explicit usage boundaries relative to siblings. Adequate, but with clear gaps an agent must fill by inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%—every parameter has a description, including the enum values for effort and the semantics of prompt and context. The tool description adds high-level purpose context but no parameter-specific detail beyond what the schema already provides, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('queries') plus a named resource ('The Contrarian (DeepSeek-V4 Pro)') and a concrete set of purposes (adversarial code review, bug-hunting, edge cases, race conditions, over-engineering checks). This clearly differentiates it from siblings like consult_architect or deliberate_council, which imply different consultation modes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool—when adversarial critique or bug-hunting is needed—but it never explicitly states when not to use it or names alternatives. Sibling names suggest different roles (e.g., architect vs. contrarian), yet the description leaves that routing entirely to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deliberate_councilA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it discloses key behaviors: concurrent queries, specific model identities, and that the tool returns raw perspectives for the Host Agent to synthesize rather than a final answer. It does not address cost, latency, or side effects, but the core operational behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with the main action front-loaded and no filler. Each sentence earns its place: what the tool does, how it works (concurrent queries), and what it returns. The size is appropriate for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main purpose and the shape of the output, and the absence of an output schema means return values need not be detailed. However, for a six-parameter tool with no annotations, it omits when to prefer this over its single-model siblings and gives no caution about cost or latency. Adequate but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds some context by mapping the model parameters to their roles (structural blueprint vs adversarial critique), but it does not provide syntax, format, or effort guidance beyond what the schema already documents. The marginal contribution does not raise the score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ('Runs a full multi-model LLM Council deliberation') and details the two sub-queries (Architect for structural blueprint, Contrarian for adversarial critique). It clearly differentiates from the single-perspective sibling tools by emphasizing concurrent execution and the return of both perspectives. No tautology or ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for getting both perspectives at once, but it never explicitly names alternatives or states when-not-to-use. An agent must infer that consult_architect or consult_contrarian would be the right choice for a single-perspective need. Guidance is left to inference rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fast_context_readerA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that a high-speed worker (DeepSeek-V4 Flash) is used, gives expected latency (~1-2 seconds), and states the context-bloat prevention effect. It stops short of describing error cases or output format, but for a simple read/filter operation the disclosed behavior is meaningful and non-obvious.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The first sentence front-loads the action and resource, the second explains the benefit. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter tool with no output schema and no annotations, the description covers the what, the why, and the performance characteristics. It could add example extraction targets or clarify how 'focus' interacts with the parse/filter/summarize modes, but the core calling guidance is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds context about the overall purpose and the effect on context window, but it does not provide parameter-specific details beyond what the schema already states for 'content' and 'focus'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs (parses, filters, summarizes) tied to explicit resources (raw file contents, large logs, cache dumps, complex schemas). It clearly differentiates itself from the sibling consultation/offload tools by positioning itself as a context-preserving extraction utility rather than a deliberation or task-delegation tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Prevents bloating host agent context window' gives a clear usage context: use this when content is large and only a focused extraction is desired. It does not explicitly name alternatives or exclusion conditions, but the sibling names and the description make the intended scenario reasonably clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.