Server Quality Checklist

Tool Scores

• delete_traceA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description extensively details behavior beyond the annotations: SQL DELETE scoped to tenant, cascade to spans, SET NULL for eval_results, non-idempotent behavior (`deleted: false`), lack of audit logging, rate limit of 20 req/min, and cross-tenant silent failures. These are valuable, non-obvious traits disclosed clearly. No contradiction with annotations—destructiveHint=true aligns with the described SQL DELETE.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections, but it suffers from redundancy. For example, the eval_results SET NULL behavior is repeated in the purpose sentence, the behavior section, and the 'Don't use' list. The cross-tenant 'deleted: false' appears in both behavior and parameter sections. While front-loaded and organized, the verbosity exceeds what a concise description needs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description fully explains the output shape ('{ "deleted": boolean, "trace_id": string }') and enumerates error modes (400, 429, 500). It covers security (tenant isolation), rate limits, audit behavior, and the nuances of cascade vs. nullify. For a destructive tool with complex side effects, this is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema already covers trace_id with a description and regex pattern, the description adds crucial context: the Zod validation fails with 400 before storage, trace_id provenance (from log_trace/get_traces), and cross-tenant behavior returning `deleted: false`. This goes well beyond the schema's raw type and pattern.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Remove a single trace by id' — a specific verb and resource — and immediately distinguishes it from siblings by noting cascading behavior and explicitly contrasting with delete_rule: 'delete_trace is the DESTRUCTIVE single-row remove for traces.' This leaves no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use scenarios ('captured in error, contains sensitive data... compliance') and when-not-to-use ('Don't use to clean up OLD data in bulk... use retention config', 'Don't use to PAUSE a trace', 'Don't use to delete eval_results'). Also suggests combining with get_traces for candidate discovery, giving the agent clear decision guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evaluate_outputA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite annotations already marking idempotentHint and non-destructive, the description adds valuable context: writes one eval_result row to Iris storage, deterministic in-process scoring, no external network calls in heuristic mode, rate limits (20 req/min HTTP, unlimited stdio), runtime ~5-50ms, and detailed output shape including insufficient_data semantics. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-organized with clear sections (siblings, behavior, output shape, usage, parameters, error modes). It front-loads the purpose, and each paragraph provides distinct value. Some minor redundancy exists between the output shape prose and the JSON example, but overall it's efficiently structured for a tool with this complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 parameters, nested custom_rules, no output schema, and multiple sibling tools, this description is exceptionally complete: covers return shape, error modes (Zod, ReDoS, rate limits, storage failures), parameter interactions, and performance characteristics. It leaves few open questions for an agent deciding to invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds crucial conditional semantics: expected is required only for relevance, cost_usd/token_usage only for cost, custom_rules always fires regardless of eval_type, and eval_type defaults to completeness. This clarifies parameter interactions beyond the schema's per-field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Score agent output against configurable eval rules and return a 0..1 score + per-rule breakdown.' It clearly distinguishes from siblings by naming evaluate_with_llm_judge, verify_citations, log_trace, etc., and positions evaluate_output as the fast heuristic path.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'Use when' and 'Don't use when' sections provide direct guidance: use for quality scoring after log_trace, avoid for empty outputs or JSON schema validation. It also names alternatives for semantic scoring and citation verification, giving clear decision criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tracesA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds substantial context beyond those: rate limits ('Rate-limited to 20 req/min on HTTP MCP, unlimited on stdio'), tenant scoping ('queries only the caller's tenant rows'), pagination limits, and the subtle idempotency nuance ('new traces logged after the call obviously show up on subsequent calls'). No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long, but appropriately structured with headers (Behavior, Output shape, Use when, Don't use, Parameters, Error modes) and front-loaded with purpose. There is minor redundancy (read-only stated twice, limits repeated in schema and text), but every sentence earns its place given 11 parameters and no output schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description fully documents the return shape ('{"traces": [...], "total": number, "limit": number, "offset": number, "summary"?: ...}'), error modes (400 for invalid enum/limit, 429 rate limit, 500 storage failures), pagination behavior, and empty-result semantics. This is complete for a complex 11-parameter read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description elevates this by adding behavioral semantics not in the schema: the min_score/max_score apply to 'the LATEST eval per trace', agent_name/framework are exact-match with 'no wildcards in v0.4', limit >1000 returns 400, and canonical combined queries like 'since + sort_by="latency_ms" + sort_order="desc"' are demonstrated. This far exceeds what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with a specific verb+resource: 'Query stored agent-execution traces with filters, pagination, and optional dashboard summary.' It differentiates itself from siblings explicitly by stating 'get_traces is the READ path for historical agent executions — never mutates anything' and lists what each sibling does for contrast.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance ('Use when you need historical data: investigating a past failure, computing quality trends, comparing agents...'), concrete query patterns (e.g., 'Set min_score / max_score to surface outliers'), and clear alternatives: 'Don't use to score a trace (use evaluate_output). Don't use to create a trace (use log_trace). Don't use as a live event stream.' This is exemplary usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_ruleA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark destructiveHint=true and idempotentHint=false, but the description goes far beyond by detailing exactly what happens: rewrites ~/.iris/custom-rules.json, appends to audit log, returns deleted:false on already-deleted rules, stops firing immediately, preserves historical eval_results, tenant-scoped behavior, and rate limits. No contradiction with annotations; the description adds substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately structured with labeled sections (Behavior, Output shape, Use when, Don't use, Parameters, Error modes), making it scannable. Every sentence provides value, and it is front-loaded with the core action. Despite its length, it is concise for the complexity it covers.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all relevant aspects: behavior, output shape, usage conditions, parameter details, error modes, and relationship to sibling tools. Given the tool's destructive nature and lack of output schema, this level of detail is complete for an agent to invoke it correctly. No gaps are apparent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  While the schema already covers rule_id with its pattern and description, the description enriches parameter semantics by explaining the Zod format mismatch (400 before store touch) and cross-tenant silent false. It also clarifies that the rule_id is exactly what list_rules or deploy_rule returns. This adds meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Remove a deployed custom evaluation rule,' a specific verb+resource that clearly states the action. It further distinguishes itself from siblings by explicitly positioning delete_rule as the destructive remove path for the custom-rule store, noting it does NOT touch traces, eval_results, or built-in rules. This fully disambiguates it from delete_trace and other management tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance ('Use when a custom rule is obsolete...') and a typical flow (list_rules → delete_rule). It also gives clear exclusions: don't use to pause (use dashboard toggle), don't use on built-in rules, and don't use for traces/eval_results. This is exemplary usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deploy_ruleA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only declare readOnlyHint=false, idempotentHint=false, destructiveHint=false. The description goes far beyond by disclosing that the tool writes to ~/.iris/custom-rules.json via atomic write, appends to audit log, activates immediately, persists across restarts, is not idempotent (mints fresh rule_id), is tenant-scoped, rate-limited to 20 req/min, and surfaces specific error modes (400, 429, 500). No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every sentence earns its place. It is front-loaded with purpose, then differentiates siblings, explains behavior, output shape, usage guidance, parameter semantics, and error modes. The structure is logical and scannable, with no repetition or filler. For a tool with 6 parameters and nested objects, this complexity warrants the length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is fully complete for a write-path tool with no output schema. It covers purpose, exact file write behavior, non-idempotency, tenant scoping, rate limits, output JSON shape, parameter semantics, when to use/avoid, and all error modes. Even without an output schema, the agent knows exactly what to expect and how to handle failures.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description adds substantial meaning: name length constraints and its appearance in eval_result rule_results, evalType must match evaluate_output eval_type with concrete examples, severity affects sorting/audit but not scoring, definition type/config pairing examples (regex_match needs config.pattern, cost_threshold needs config.max_cost, etc.), and invalid configs rejected at deploy time. This is far beyond the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource+scope: 'Deploy a new custom evaluation rule that will fire on every future evaluate_output call of its eval category.' It explicitly distinguishes from siblings by naming them (list_rules, delete_rule, evaluate_output, log_trace, get_traces, delete_trace, evaluate_with_llm_judge, verify_citations) and states deploy_rule is the 'WRITE path that grows the custom-rule library.' This is unambiguous and clearly differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Use when an agent observes a recurring failure pattern and decides to enforce it as a standing rule.' It also gives explicit when-not-to-use guidance: don't use to validate (use the preview endpoint) and don't use to edit (use delete_rule + deploy_rule). It further maps a concrete workflow combining evaluate_output + get_traces + deploy_rule. Alternatives are explicitly named and contextually placed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evaluate_with_llm_judgeA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Even though annotations already declare readOnlyHint=false and openWorldHint=true, the description adds substantial behavioral context: external API costs money, 1-10s latency, cost cap via IRIS_LLM_JUDGE_MAX_COST_USD_PER_EVAL, non-determinism, writes an eval_result row, rate limits, and one retry on 429. Error modes are enumerated in detail. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but highly structured with clear section headers (sibling tools, behavior, output shape, use when, parameters, error modes). Every paragraph serves a distinct purpose, and it front-loads the core function. No redundancy with schema or annotations was found.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 12-parameter tool with no output schema, the description fully compensates: it details the exact output JSON structure, template-specific requirements, cost guardrails, rate limits, prerequisites, and all error modes. It leaves no operational ambiguity and is self-contained for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description goes far beyond individual field descriptions by explaining cross-parameter constraints: expected is required only for correctness template, source_material only for faithfulness, model has no default and cost varies 100x, max_cost_usd is pre-computed before the call, and explicit defaults for temperature, max_output_tokens, and timeout_ms. This is high-value semantic enrichment.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with a specific verb+resource: 'Score agent output using an LLM as the judge (Anthropic or OpenAI).' It immediately distinguishes itself from siblings by naming evaluate_output as heuristic and verify_citations as a specialized form, making its unique role unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance ('Use when heuristic rules are too coarse...'), explicit don't-use cases ('Don't use for simple regex...', 'Don't use without an API key', 'Don't use on very large outputs'), and names alternatives (evaluate_output, verify_citations). This is exemplary usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_rulesA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, but the description adds substantial behavioral context: pure read, in-memory caching, no disk read per call after boot, no external network, tenant scoping, rate limits (20 req/min HTTP, unlimited stdio), and <5ms response time. It also discloses error modes (429, empty list for missing file). No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured with clear sections (Sibling tools, Behavior, Output shape, Use when, Don't use, Parameters, Error modes). It front-loads the core purpose and each section earns its place by adding non-redundant information. No fluff or repetition of schema field names beyond what is necessary for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity and the absence of an output schema, the description fully compensates by documenting the output shape, empty-array behavior, rate limits, error modes, and scoping nuances. It covers all contextual dimensions an agent would need for correct invocation and result interpretation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema covers both parameters (100% coverage), the description adds crucial semantics: eval_type is exact-match with no wildcards, enabled_only excludes deployed-but-disabled rules, both filters are AND-combined, and defaults are explicit (undefined / false). This goes well beyond the schema's field descriptions, providing operational detail that an agent needs to invoke correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Enumerate deployed custom evaluation rules from the local rule store.' It further distinguishes the tool from siblings by explicitly stating 'list_rules is the READ path for the custom-rule store; nothing else exposes the inventory,' making its unique role unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Use when you need to know what custom rules are currently live' followed by concrete scenarios. It also provides strong exclusions and alternatives: 'Don't use to count traces or evals (that's get_traces)', 'Don't use to inspect built-in rules', and names deploy_rule/delete_rule for mutations. This is textbook usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• log_traceA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Goes far beyond annotations by detailing persistence backends (SQLite/Postgres), best-effort OTel export with fire-and-forget semantics, auth/rate limits by mode, non-idempotency, output shape, and error modes (Zod rejects, 500 on storage failure, ~50ms latency). These details are not present in the annotations and materially affect agent behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but efficiently organized with labeled sections (Behavior, Output shape, Use when, Parameters, Error modes). It front-loads the core purpose with no filler; each section addresses a distinct question an agent would have, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a highly complex tool with nested objects, OTel export, and 11 parameters, the description covers purpose, usage timing, output format, error behavior, parameter nuances, and immutability constraint. The lack of an output schema is compensated by the explicit output shape description. This is a comprehensive and actionable specification.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema covers all 11 parameters, the description adds rich semantics not in the schema: summary fields are authoritative (no auto-aggregation), span start_time fallback and duration derivation, metadata is opaque and not filterable via get_traces, missing latency means 'not reported', and defaults for span.kind/status_code and timestamp. This resolves ambiguities an agent would otherwise face.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Persist a single agent execution trace' — a specific verb and resource. It enumerates the stored data (input, output, spans, tool calls, cost, latency, token usage) and explicitly distinguishes log_trace from siblings as 'the WRITE path' while naming what each other tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Contains a dedicated 'Use when' section listing preconditions (after agent output) and follow-up tools (evaluate_output, get_traces), and a 'Don't use when' section contrasting with console logging and noting the lack of an update path due to immutability. Explicit exclusions and alternatives are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_citationsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are minimal (readOnlyHint=false, openWorldHint=true), leaving the description to carry nearly all behavioral context—and it does. It discloses SSRF-guarded fetch mechanics (scheme allowlist, IP blocking, redirect chase with re-checks), default refusal of outbound HTTP unless allow_fetch=true or IRIS_CITATION_ALLOW_FETCH=1, cost caps that stop the pipeline mid-citation, 20 req/min MCP rate limit, and that it writes one eval_result row. These details align with and enrich the open-world hint without contradicting any annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but front-loaded and ruthlessly organized: opening summary, then labeled sections for Sibling tools, Behavior, Output shape, Use when, Don't use, Parameters, and Error modes. Each sentence carries distinct operational facts with no filler; the length is proportionate to the tool's 10-parameter, network-facing, judge-calling complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema present, the description supplies a complete JSON response shape, covering every field. It also documents all throw-vs-collect error semantics, per-citation error kinds, graceful degradation (null overall_score), and safety/cost/time limits. For a tool of this complexity, everything needed for correct invocation and interpretation is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema already covers 100% of parameters with descriptions, the tool description adds substantial operational meaning: defaults for every parameter, the iri_citation_allow_fetch env-var interaction, suffix-matching for domain_allowlist with union merge over IRIS_CITATION_DOMAINS, silent skip of citations beyond max_citations (hard cap 50), mid-call cost-cap behavior, and the full resolve_error.kind taxonomy. This goes well beyond the structured schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific three-phase verb chain ('Extract citations from agent output, fetch the cited sources, and use an LLM judge to check...') and names the exact return value (per-citation verdicts + overall support ratio). It then explicitly distinguishes from siblings evaluate_with_llm_judge (general semantic scoring) and evaluate_output (cheap fabricated-citation heuristic), making the tool's unique scope unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'Use when' paragraph gives concrete preconditions (factual claims with [1]-style references, DOIs, or URLs; research/legal/medical domains). The 'Don't use' paragraph lists three explicit exclusions (no citations, no allow_fetch, open allowlist with untrusted output). Alternatives are named directly: evaluate_with_llm_judge for generic scoring, evaluate_output for cheap detection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.