Flaky CI Triage

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint, idempotentHint, and destructiveHint false. The description adds that it combines pipeline history, job oscillation, comparisons, and commit/MR context, but does not disclose potential rate limits, data size implications, or full behavior beyond what annotations suggest.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that effectively front-loads the core purpose. It is efficient but could be improved by structuring into bullet points or adding brief parameter guidance without losing conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (combining multiple analyses, 5 parameters, no output schema), the description is insufficient. It does not explain output format semantics, how min_samples or lookback_pipelines affect results, or typical use cases, leaving gaps for an agent to choose correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, meaning the schema provides only names and types without descriptions. The tool description does not compensate by explaining any parameter meaning, usage, or constraints beyond what the schema (e.g., default values, enums) implicitly shows.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Assess whether recent CI failures look flaky' using multiple data sources. This distinguishes it from sibling tools like gitlab_find_flaky_jobs or gitlab_compare_pipeline_runs, which focus on specific aspects.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not provide explicit guidance on when to use this tool versus alternatives. It mentions combining various sources but lacks when-not-to-use scenarios or prerequisites, leaving the agent to infer usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.