Server Quality Checklist

Tool Scores

• harness_reviewA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must disclose behavior. It mentions what is scanned but fails to indicate if the tool is read-only, has side effects, performance impacts, or output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. First sentence packs the core purpose, second provides usage guidance. Ideal conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Missing essential details: no output schema, no mention of return values, no prerequisites (e.g., git repository). For a scan tool, this is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context about the review types but does not enhance understanding of individual parameters beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs multi-perspective code review covering security, correctness, and style, with specific examples. It differentiates from sibling tools like harness_classify and harness_init.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises periodic calling (every 3rd turn) to catch missed issues. While it doesn't list when not to use, the sibling tools have distinct purposes, making usage context clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• harness_resetA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must disclose behavioral traits. It states it resets state but lacks details on reversibility or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first explains purpose, second gives usage context. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but tool is simple. Could elaborate on return values or error scenarios, but adequate for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for both parameters. Description adds no additional semantic value beyond what schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'Reset' and specific resources 'PGO baselines and shown-errors state' distinguish it from siblings like harness_classify or harness_init.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit when to use: 'when switching to a new task or after significant dependency changes.' No explicit exclusions or alternatives mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• harness_pgoA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It explains the incremental behavior (only new errors, reset on success) well. Lacks details on side effects or permissions, but for a verification tool, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each adding unique value: action, key concept, usage guidance. No waste, effectively front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool (one param, no output schema), the description provides sufficient context: what it does, its incremental nature, and when to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a single parameter 'cwd' described as 'Working directory'. The description adds no extra meaning beyond the schema, so baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs incremental PGO verification and highlights the key concept of returning only new errors. However, it does not distinguish itself from sibling tools beyond the incremental aspect, which could be more explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends use after each agent turn that modifies code for incremental verification. No when-not or alternatives provided, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• harness_classifyA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that simple tasks skip PGO and review, but does not detail other behavioral traits (e.g., idempotency, state changes). Adequate for a simple classification but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the verb, examples, and usage guidance. Every sentence is purposeful with no extraneous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage, and behavioral implications. However, it does not describe the output/return value (e.g., expected format like 'simple' or 'complex'), which is missing since no output schema exists. This leaves an important gap for an AI agent invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema documents one parameter 'message' with description 'User's task description'. The tool description adds meaning by defining the classification outcomes and context, significantly enriching the parameter semantics beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Classify a task as simple or complex' with explicit examples. It distinguishes from sibling tools (harness_init, etc.) by focusing on classification rather than execution or review.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies 'Call before starting work' and explains that simple tasks skip overhead, providing clear contextual guidance. However, it does not explicitly discuss when not to use or give direct comparisons to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• harness_initA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that the tool detects commands, establishes baselines, classifies complexity, and returns session info. However, it does not mention side effects, prerequisites, or whether it overwrites existing state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each serving a distinct purpose: purpose, usage, return. No redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's role as an initializer with three parameters and no output schema, the description adequately covers what it does, when to call, what to provide, and what it returns. Could elaborate on 'baselines' and 'PGO readiness', but sufficient for typical usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline 3. Description adds value by explaining the purpose of taskMessage for complexity classification and that commands can override auto-detection. Adds context beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Initialize LiDAR Harness for a project' and lists three specific actions: detect commands, establish baselines, classify complexity. It distinguishes from siblings like harness_classify and harness_reset by being the initialization step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Call this ONCE at the start of a session' and advises to provide the user's task message. Provides clear context for usage, though no explicit when-not-to-use or alternative tools are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.