Identify flaky Playwright CI tests that failed then passed on retry within a chosen period, sorted by retry count.
Tests that were flaky (failed, then passed on retry) in the last N days, most flaky first, with counts.
| Name | Required | Description | Default |
|---|---|---|---|
| days | No |
Changes observed during successful MCP inspections.
v0.1.0Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations, so the description carries the full burden. It usefully defines the 'flaky' criterion, the sort order, and that counts are included, but says nothing about result limits, truncation, lookback default behavior at the 90-day cap, or permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with the resource first and the definition, ordering, and output shape packed in without waste. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter read tool with no output schema, the description covers what is returned (counts, ordering) and the semantics of flakiness. Only edge details like result caps or the interpretation of the maximum window are absent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but there is only one self-explanatory integer parameter ('days'). The phrase 'last N days' confirms it is a lookback window, matching the schema default of 14, but the description adds no detail beyond what the parameter name already conveys.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific resource (flaky tests) with an explicit definition of 'flaky' (failed then passed on retry), plus ordering and payload (most flaky first, with counts). This is clearly distinguishable from a generic test-history or recent-runs tool, though no sibling is named directly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance and no alternatives named. It is implied that this is for investigating flakiness rather than viewing runs, but an agent gets no explicit trigger or exclusion relative to turnsignal_test_history or turnsignal_recent_runs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.