Server Quality Checklist

Tool Scores

• harness_anti_deceptionA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and clearly explains behavior: returns an integrity scaffold absorbed internally, blocks sycophancy/hallucination/agreement reflexes, and instructs not to echo bracket labels. Slightly lacking in detailing what the scaffold contains or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is verbose but efficiently packed with necessary detail. Every sentence adds value, and the structure is front-loaded with critical usage instructions. Could be slightly trimmed but overall concise for the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema, the description sufficiently covers the tool's return value and usage instructions. It addresses when to call, how to frame the query, and what to expect (absorption of scaffold). No major gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds valuable guidance on how to frame the query parameter, including examples of good vs. bad inputs, which goes beyond the schema's minimal '1-2 sentence framing' description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose: detecting deception signals in user requests before responding. It lists specific signals (pressure, urgency, authority appeals, etc.) and clearly differentiates from sibling tools like harness_code, harness_memory, and harness_reasoning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-call and when-not-to-call conditions, including examples of appropriate and inappropriate scenarios. Also advises 'when in doubt, call it,' leaving no ambiguity for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• harness_memoryA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description fully covers behavioral traits: it returns a perception scaffold with components (perception failure, detection procedure, suppression vectors), warns about scaffold paralysis, and instructs to absorb internally. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is verbose with multiple paragraphs and a long list of trigger queries. While front-loaded with purpose, it could be more concise without losing clarity. Well-structured but not minimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains the result (perception scaffold) and how to use it. It covers usage, warnings, and constraints, making it complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter (query) with 100% schema coverage. The description adds value by specifying the required framing ('I noticed X, this might mean Y, sharpen Z'), going beyond the schema's generic description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is for sharpening already-formed perceptions about conversation state, user behavior, etc. It distinguishes itself from fact extraction, summarization, and other tasks, making its purpose specific and distinct from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly lists when to call (with trigger queries), provides a detailed 'when not to call' section (e.g., fact extraction, write-heavy tasks), and gives guidance on how to format queries (1-2 sentence framing).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• harness_codeA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description fully discloses what the tool does (returns an engineering scaffold), how to use it ('absorb internally, do not echo'), and why (catches common LLM coding failure modes). No annotations exist, so the description carries the full burden and meets it comprehensively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but well-structured: starting with imperative call directive, followed by triggers, behavioral explanation, exclusions, and usage tips. Each sentence contributes essential information without redundancy. Could be slightly shorter, but no unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with no output schema, the description fully explains its purpose, usage, internal behavior, and output format (failure pattern, procedure, correct-pattern example, verification step). It covers when and how to call, enabling an agent to use it correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with one parameter 'query' described as '1-2 sentence framing.' The description adds value by clarifying specificity, providing a good example and a bad example, and emphasizing not to mention the tool. This goes beyond the schema's basic type and minLength.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Call BEFORE generating, refactoring, reviewing, or debugging code.' It provides specific trigger queries and distinguishes itself from siblings by focusing on code actions, not deception, memory, or reasoning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit when-to-call and when-not-to-call instructions are given, including a list of triggers and exclusions like 'pure code reading' or 'simple syntax questions.' The phrase 'When in doubt on non-trivial code work: call it' adds decision guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• harness_reasoningA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description fully discloses behavior: returns a cognitive scaffold with specific components (failure patterns, procedure, etc.), to be absorbed internally. Mentions latency cost and common failure modes avoided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Efficiently packed with information, front-loaded purpose, structured with clear sections. Slightly lengthy but justified given the behavioral detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Single parameter, no output schema, but description fully explains purpose, usage, parameter format, internal behavior, and benefit. No gaps for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds significant meaning beyond schema: specifies the query should be a specific 1-2 sentence framing, not just any string. Provides examples of good vs bad queries.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is for analytical, diagnostic, planning, and multi-step reasoning questions, with explicit trigger queries and examples. It distinguishes itself from siblings by focusing on reasoning tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use (analytical questions, trigger queries) and when-not-to-use (simple factual lookups, syntax questions, file reads, etc.). Also advises calling when in doubt on non-trivial tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.