Server Quality Checklist

Tool Scores

• groundB
  3.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, but the description adds important behavioral details: it reports low confidence as UNKNOWN and never asserts them. This helps an agent understand the tool's reliability and handling of uncertain facts.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no redundancy. Each sentence adds value: the first defines the core action, the second clarifies a key behavioral rule. It is efficiently structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one parameter, no output schema, and no annotations, the description covers the main behavior and a critical detail (UNKNOWN handling). However, the loop context is vague, and return format is not described, leaving some ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description only mentions the query parameter implicitly via 'Exa deep search' but adds no specifics on query format, constraints, or examples. The parameter semantics are underexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool performs Exa deep search to retrieve cited, confidence-scored facts and mentions it's used inside a loop. This distinguishes it from siblings like 'doubt' and 'verdict', though not explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal usage context: 'used inside the loop' implies a specific workflow but fails to specify when to use this tool versus alternatives or when to avoid it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verdictA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It describes the decision logic (proceed or return questions) but does not specify whether the tool has side effects, requires authentication, or how it processes the input. As a gate function, it likely only inspects state, but this is not confirmed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise—only two sentences. It front-loads the core action ('Gate a doubt() result') and follows with clear conditions. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, no annotations, and a single parameter with no description, the description is minimally complete: it explains the tool's function and decision condition. However, it lacks details on error handling, the expected input structure, and the format of the output (blocking questions), which may confuse an agent unfamiliar with the system.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for the single parameter 'result', the description must compensate. It only says 'Gate a doubt() result', implying the parameter should be a doubt result object, but provides no structure or constraints (e.g., required fields, type expectations) for this generic object.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Gate' and resource 'doubt() result', clearly stating the tool's purpose: to either proceed if the plan converged without doubt or return blocking questions. It distinguishes itself from siblings 'doubt' and 'ground' by referring to the output of doubt().

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: after a doubt() result, to decide based on convergence. It implies the condition for proceeding and what to do otherwise, but does not explicitly mention when not to use or alternative tools, though the sibling context provides some differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• doubtA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It thoroughly explains the iterative behavior, convergence condition, hard ceiling on max_passes, and the role of context. It also lists return fields, giving a clear picture of what the tool produces.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a concise overview, followed by clear Args and Returns sections. Every sentence adds value without redundancy. It is front-loaded with the core concept.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (iterative doubt resolution, 3 parameters, no output schema), the description is complete. It covers behavior, parameter semantics, return format, and usage context. No gaps are apparent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description fully documents all three parameters in the Args section. It adds meaning beyond the schema: prompt is the task/plan, context is real evidence, max_passes is the convergence ceiling clamped to 20. This is comprehensive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to doubt every decision in a plan, iteratively resolve doubts with evidence, and converge on a hardened plan. It uses specific verbs and resources, and distinguishes itself from siblings (ground, verdict) by its iterative doubt-resolution process.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use the tool: for hardening plans against doubts. It specifies how to use context (codebase for code doubts, Exa for world doubts) and to flag for human when needed. It mentions the hard ceiling of 20 passes. However, it does not explicitly exclude cases where alternatives like ground or verdict would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.