Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, covering the safety profile. The description adds no additional behavioral context (e.g., auth, rate limits) beyond what the annotations provide. It does mention the output components, but that relates to return value, not behavior. No contradictions exist.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, information-dense sentence that front-loads the core purpose ('Returns the score of one test') and then lists the specific components and the test id format. No wasted words; every clause adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has a complete input schema, an output schema, and a description that enumerates the return components and provides the test id format. An agent has everything needed to call it correctly: required testId, optional repo, and expectations for the response. No gaps are apparent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for both parameters (repo and testId). The description repeats the test id format already present in the schema, adding no new semantics. Since the schema fully documents parameters, the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Returns') and resource ('score of one test'), and enumerates the components that make up the score (flip rate, rerun recovery, etc.). This clearly differentiates it from sibling tools like flake.rank (which ranks multiple tests) and flake.quarantine.list (which lists quarantined tests) by focusing on a single test's explanation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies a use case (explaining a single test's flakiness score) but does not explicitly state when to use this tool versus alternatives. No exclusions or alternative routing are mentioned, leaving the agent to infer based on the name and single-test focus. Clear context is present, but explicit guidance is missing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint and idempotentHint, so the safety profile is covered. The description adds the key behavioral detail: the verdict and per-failure decision logic. It does not contradict annotations and provides meaningful context beyond them, though it does not detail the exact response format (already covered by output schema).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that front-loads the primary action and then explains the decision logic and return value. There is zero fluff; every clause contributes meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple (two optional params, both well-documented) and the output schema exists, so the description does not need to explain return values in detail. It fully covers the core logic and provides enough context for correct invocation. No critical information is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters are already well-described in the schema. The description adds a concrete example ('target/surefire-reports') for the reports parameter, which is useful but not substantial. It does not compensate for any gaps because there are none, so a baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('runs flake-detector's gate') and a specific resource ('directory of JUnit XML reports'), and explains the exact decision logic (which failures block, which do not). It clearly differentiates from siblings like flake.rank and flake.explain by focusing on build gating.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly conveys the context of use (deciding whether failed tests block the build) through both the title and the explicit behavior description. However, it does not explicitly mention alternatives or exclusions, so it stops short of a 5. An agent can infer this is the tool for build-block decisions, but there's no guidance on when to prefer it over other flake tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds a meaningful behavioral consequence beyond the readOnlyHint and idempotentHint annotations: expired entries fail the build until renewed or removed. It does not conflict with annotations and gives the agent useful context about downstream impact without restating the annotation-provided safety profile.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences front-load the core purpose and then add one important behavioral note. There is no filler or redundant restating of the tool name or schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema, annotations, and simple optional parameter, the description is complete for invocation: it identifies the source file, the returned fields, expiration semantics, and the build-failure consequence. No critical information needed to call the tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single optional repo parameter, so the schema already explains it fully. The description adds no param-specific details, but the baseline of 3 applies because the schema carries the semantic burden adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Reads') and names an exact resource (the quarantine ledger at .flake/quarantine.yaml), then spells out exactly what is returned: reason, owner, expiry, and expiration status. This clearly distinguishes it from sibling tools like flake.gate or flake.rank, whose purposes are different even without explicit comparison.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to prefer this tool over siblings or when it should not be used. There is no mention of alternatives or exclusions, so an agent must infer its role purely from the title and subject matter.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, lowering the bar. The description adds valuable behavioral context: it reads a specific database file (`.flake/history.db`), computes scores, and returns metadata including signals and explanations. It also describes output order. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with zero redundancy: the first sentence states the action, data source, and output in detail; the second gives a clear usage instruction. Front-loaded and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose, output contents, and usage context. Since an output schema exists, detailed return types are unnecessary. The only minor gap is not explicitly stating that the history database must exist, but it's implied by 'ingested run history.' Overall adequate for the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description doesn't add parameter-specific meaning beyond what the schema already documents ('top' counts, 'repo' root). It implicitly relates to ordering but that's already in the schema's parameter descriptions. The baseline applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb ('runs flake-detector's scorer over the ingested run history') and a specific resource (the history database), and it lists the output (ranked tests with score, run count, signals, explanation). It doesn't explicitly differentiate from sibling tools like flake.explain or flake.quarantine.list, but the function is unambiguous enough that an agent can tell it apart.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage context: 'Use it before retrying a failing test or deciding whether a red build is a regression.' This gives clear when-to-use guidance, though it doesn't mention exclusions or alternatives. That matches a 4: clear context without explicit when-not-to-use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, covering the safety profile. The description adds meaningful behavioral context by explaining the comparison logic (score drop > threshold or pass->fail flip) and implying dependence on the output of llmeval.run. It does not contradict the annotations and goes slightly beyond them by detailing the filtering condition, though it does not disclose potential performance characteristics or file-format requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with zero filler. The first sentence states the core action and output condition, front-loading the most important information. The second sentence provides usage guidance, also concise. The structure is efficient and easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has four parameters, two required, and an output schema exists (per signals). The description clearly explains the purpose and usage, including a prerequisite ('after llmeval.run'). It doesn't need to explain return values since an output schema exists. It could mention file-format expectations or error conditions, but these are either implied or covered by the schema. Overall, it's sufficiently complete for an agent to use correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all four parameters (repo, current, baseline, threshold) with clear descriptions. The tool description mentions 'threshold' but only repeats what the schema says ('Score drop that counts as a regression; the harness default is 0.10'). It adds no new meaning beyond the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Compares'), the resource ('two llm-eval-harness result files'), and the exact output condition ('cases whose score dropped by more than the threshold or that flipped from pass to fail'). It also differentiates from the sibling llmeval.run by explicitly saying 'Use it after llmeval.run', so an agent can tell it apart without ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use the tool: 'Use it after llmeval.run to see whether a prompt change or model upgrade regressed anything.' This provides clear context and names the prerequisite tool, effectively guiding the agent on when this tool is appropriate and when it is not (e.g., before a run). No other exclusions are needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare non-read-only and idempotent, while the description adds valuable behavior: replay serves recorded responses with no API key, record invokes the model and saves recordings, and the run returns per-case checks, score, tokens, latency, pass rate, and a result file path. This goes well beyond the structured hints and does not contradict them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences pack the core behavior, mode choice, returned metrics, and downstream hook with no filler. The central purpose is front-loaded, and the mode and result-file details follow in a natural order.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with five parameters, one enum, an output schema, and annotations, the description covers the input resources, the mode alternatives, the returned metrics, and how the result feeds llmeval.drift. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and every parameter already has a description, so the tool description is not required to add parameter-level detail. It reinforces the mode distinction in prose and explains the output, but it does not introduce new meaning for goldenSet, prompt, version, or repo beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a concrete action ('Runs llm-eval-harness') and a precise resource ('a golden dataset with one prompt version'), and the title reinforces the exact unit of work. It also signals the downstream relationship to llmeval.drift, which helps distinguish this runner from the TQL/flake sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear decision-relevant guidance: replay is the default, serves recorded responses, and needs no API key, while record calls the model and saves recordings. It does not explicitly say 'use replay when X, use record when Y' or contrast with llmeval.drift, but the mode semantics are clear enough for an agent to choose correctly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is covered. The description adds behavioral context beyond annotations by specifying exactly what the returned documentation contains (bad/fixed example, options), which helps the agent anticipate the output. It does not describe error handling for invalid rule ids, but that is minor given the schema pattern constraint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences of minimal length. The primary purpose is front-loaded in the first sentence, and the rule id format is a single brief follow-up. There is no redundancy or filler; every word contributes.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and an output schema present, the description adequately covers what the agent needs: it explains the tool's output content and the rule id format. It does not mention potential errors or edge cases, but the output schema likely conveys return structure, and the simple nature of the tool makes this gap acceptable. Overall, it is sufficiently complete for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with the ruleId property fully documented (pattern '^TQL\d{3}$' and example). The description's mention that 'Rule ids look like TQL003' merely repeats the schema information without adding new meaning. Thus, the description does not compensate or expand on the schema, so a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Returns') and resource ('documentation of one rule'), and enumerates the content ('what it catches, a bad example, a fixed example and its options'). It clearly distinguishes from list-type tools like tql.rules and lint tools like tql.lint by focusing on a single rule's explanation. The rule id format is also given, which removes ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is used when one needs detailed documentation of a specific rule, but it does not explicitly contrast it with sibling tools such as tql.rules (likely lists all rules) or flake.explain (explains a different linter). There is no 'use this instead of X when...' guidance, leaving the agent to infer the use case from the name and description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint and idempotentHint, so the description's job is reduced. It adds value by detailing the output structure (rule id, severity, location, message, fix hint) and the categories of issues detected, which helps the agent understand what to expect without contradicting the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero fluff. The primary action and output are front-loaded in the first sentence, and the second sentence enumerates concrete examples of what the linter catches. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description need not detail return values. It covers the core behavior, output fields, and relative path nuance. It doesn't explain config or failOn, but these are fully documented in the schema, so nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds a key clarification: 'Paths are relative to the repository,' which directly aids correct invocation of the 'paths' parameter. It also implies the default scope (test sources) aligns with the default 'src/test/java'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'runs' a linter, the resource 'test sources', and the output format. It also lists specific issue categories it catches, making the purpose unambiguous and distinct from generic linting tools. While it doesn't explicitly contrast with siblings like tql.rules, the specificity is enough to differentiate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this is the tool to lint Java tests but does not explicitly mention when to use it versus tql.rules or tql.explain. It provides no exclusions or alternative routing, leaving the agent to infer based on context. A stronger definition would note that tql.rules lists rules and tql.explain explains them.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnlyHint=true and idempotentHint=true, so no side-effect warning is needed. The description adds the content shape of the returned catalogue, but since an output schema exists, this is somewhat redundant; no additional operational caveats such as pagination or rate limits are disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences: the first states what the tool returns, and the second routes to the alternative for deeper detail. There is no filler or repetition of structured fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless, read-only, idempotent list tool with an output schema, the description covers the purpose and the obvious follow-up action. Nothing needed for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there is no parameter semantics burden on the description. The baseline of 4 applies because no parameter documentation is needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Returns the catalogue of rules'), identifies the resource (rules the linter applies), and lists the included fields. It also distinguishes itself from tql.explain by noting that the sibling is for full documentation of a single rule, so an agent can select the right tool without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly tells the agent to use tql.explain or the toolkit://tql/rules/{id} resource when full documentation of one rule is needed, which implies this tool is for the complete catalogue. This is a clear alternative-routing statement, leaving little to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.