Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses that the tool executes pytest with pytest-cov, parses coverage.json (never the human-readable summary), and computes overall coverage, per-file coverage, and uncovered line counts. This is comprehensive for a coverage analysis tool, though it could mention any destructive effects or file modifications.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise at two sentences: the first line states the core purpose, and the second line details the execution and parsing behavior. Every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that an output schema exists (so return values are documented separately), the description adequately covers the tool's behavior, parameters, and execution context. It could mention prerequisites like pytest-cov installation, but for a tool with good schema coverage and output schema, this is mostly sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage, so the baseline is 3. The description adds value by explaining that test_path restricts which tests to run and defaults to the whole suite, and repo_path must be under allowed roots. However, it does not elaborate on the parameters further beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that this tool runs the test suite with coverage and reports real, per-file coverage numbers. It specifies the resource (test suite) and verb (analyze coverage), and the detail about parsing machine-readable coverage.json distinguishes it from any generic coverage tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use this tool (for running pytest with pytest-cov and getting machine-readable per-file coverage), but it does not explicitly say when not to use it or mention alternatives like detect_flaky_tests or record_test_run. It implies usage for coverage analysis among siblings focused on flaky detection, risk prediction, and history.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations were provided, so the description carries the full burden. It discloses that the tool runs the full suite multiple times (potentially expensive), uses stable pytest node IDs, controls test order and seeds, and reports flakiness rates. However, it does not mention if the tool modifies any files or has any side effects beyond reporting.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and structured effectively. The first sentence immediately states the core purpose, followed by a short paragraph explaining the mechanism and guarantees. Every sentence adds essential information without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that the tool has only 2 parameters with 100% schema coverage, an output schema exists, and no annotations are provided, the description sufficiently covers what the tool does, how it works, and what guarantees it provides (controlled order/seeds). There are no obvious gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds value by explaining that runs controls the number of executions and that higher values give more confidence but take proportionally longer. It also clarifies that repo_path must be a Python repository and resolves under allowed roots. The first sentence of the description contextualizes both parameters together.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose using a specific verb-resource pair: 'Run the test suite multiple times and flag tests with inconsistent outcomes.' It immediately distinguishes itself from siblings by focusing on flakiness detection rather than coverage analysis, risk prediction, or history retrieval.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implicitly suggests when to use this tool (to identify flaky tests) vs alternatives (e.g., analyze_coverage for coverage, get_test_history for historical results). It explains that test order and seeds are controlled to avoid false positives, but does not explicitly state when NOT to use it or provide direct comparisons to siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description correctly identifies the tool as read-only ('never triggers a new test run', 'reports on what's already been recorded'). This adequately covers the main behavioral trait. However, it could add more details like handling of empty results or performance implications for large histories.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the main purpose, and no wasted words. The description efficiently conveys the tool's role and its safety characteristic.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only query tool with an output schema, the description covers purpose, read-only nature, and relationship to record_test_run. It doesn't mention ordering (most recent first) but that is in the parameter description. Minor gap in not stating that history is returned sorted, but overall complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with both parameters fully described. The description adds no additional parameter semantics beyond the schema, so baseline 3 is appropriate. The context about reading history is helpful but does not directly enhance parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it queries previously recorded test run history, with the verb 'query' and resource 'test run history'. It distinguishes from siblings by noting it reads rows written by record_test_run and never triggers a new test run, differentiating it from record_test_run and other tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states this tool is for reading existing history and never triggers a new test run, guiding the agent to use it when querying past results. It indirectly points to record_test_run as the alternative for writing, providing clear usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the behavioral burden. It explains the internal steps (diffing, feature extraction, model inference) and names the model type (GradientBoostingClassifier), which aids transparency. However, it does not disclose side effects (e.g., does it write anything) or failure modes for missing refs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: the first is a concise summary, the second provides helpful internal detail. Each sentence adds value, though the second could be slightly tighter.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (2 params, 100% schema coverage, and an output schema exists), the description is largely sufficient. It explains how the tool works internally, which compensates for the lack of annotations, leaving no major gaps for typical use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents both parameters. The description calls out 'base_ref' implicitly by stating it diffs HEAD against base_ref, but does not add new semantics beyond the schema's default and description for base_ref.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool diffs the current branch against a base ref and ranks changed files by predicted risk. It uses specific verbs ('diff', 'rank') and resources ('changed files', 'predicted risk'), distinguishing it from siblings like 'analyze_coverage' or 'train_risk_model'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is used for assessing risk of changed files in a branch, which differs from siblings like 'detect_flaky_tests' or 'analyze_coverage'. It does not explicitly mention when not to use it or provide alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, description carries full burden. It discloses that the tool executes a test suite (potentially side-effecting) and persists results, and it emphasizes the explicit, distinct nature preventing implicit side effects. It does not mention failure behavior or data overwrite semantics, but it provides security context (repo_path must resolve under allowed roots).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, first is direct action, second adds valuable context on when to use. No redundant filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool executes arbitrary Python tests and persists data, description covers the core role and relation to siblings. Output schema exists, so return values are documented elsewhere. Missing some edge-case behavior but adequate for an AI agent to select and invoke.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers the only parameter fully with a description including absolute path and allowed root constraint. The tool description does not add further meaning, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Description starts with a specific action: 'Execute the test suite once and persist the results to the database.' It clearly distinguishes from siblings by explaining that this is the explicit write step while get_test_history and train_risk_model consume the data, and it never runs implicitly, so it's unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explains that this tool is the explicit step for writing history versus reading or analyzing, referencing sibling tools get_test_history and train_risk_model. It could be more explicit about when NOT to use it (e.g., for read-only queries, use get_test_history), but the contrast between 'written versus just read or analysed' provides clear context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description fully explains the tool's behavior: it trains a GradientBoostingClassifier, uses training history, reports metrics, and includes a caveat for small datasets. The disclosure about statistical meaningfulness is valuable. However, it doesn't mention how long training might take, whether it requires a running database, or if it's a destructive operation (overwrites the current model?), which would justify a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three well-structured paragraphs: first a one-liner summary, then technical detail, then returns section. Every sentence adds value without redundancy. It is concise (every sentence earns its place) and front-loaded with the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The output schema is present, so return values are documented elsewhere. The description covers the algorithm, training data source, metric reporting with caveats, and honesty about statistical significance. This is complete for a training-tool description given the schema availability and zero parameters.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters, and schema description coverage is 100% (schema is empty). The description doesn't need to explain parameters, but it adds value by detailing the training process and metrics returned. Baseline is 4 due to no parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Retrain the PR-risk classifier on everything currently in the database') and specifies the algorithm (GradientBoostingClassifier) and output metrics. It distinguishes the tool from siblings like 'predict_pr_risk' (which presumably uses the model, not trains it) and 'analyze_coverage' (which analyzes coverage, not risk).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for retraining the model using historical data, with no explicit when-to-use or when-not-to-use guidance. However, given that all sibling tools have distinct purposes (e.g., 'predict_pr_risk' for predictions, 'analyze_coverage' for coverage), the context is clear enough. No alternatives or exclusions are stated, so score 4.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.