Skip to main content
Glama

gluecron_list_flaky_tests

Read-only

List a repository's flaky tests. A test is flaky only with a forensic repro: when it failed, CI re-ran just that test in the same order, alone, with the clock shifted +137 ms and +1 s, in TZ=Pacific/Chatham and in a fixed random order, and a variable flipped its verdict N times in a row on the same git tree. Each entry has the test, file, state (active / expired / resolved, or unexplained = flipped with no repro, NOT flaky), label (clock-dependent, order-dependent, concurrency-dependent), the variable, the exact repro command, owner, expiry, issue number, flip rate and evidence runs. Information only — flaky tests are never skipped or quarantined; one targeted retry, and the merge gate excuses a failure only when that retry passed in the same run and the label is active.

Input Schema

NameRequiredDescriptionDefault
repoYes
limitNoMax tests (default 25, max 100)
ownerYes
window_daysNoFlip evidence from this many days back (default and max 30)

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, but the description adds real behavioral context beyond them: the tool is 'information only', flaky tests are never skipped or quarantined, one targeted retry is allowed, and the merge gate excuses a failure only when the retry passed in the same run and the label is active. That downstream policy disclosure is genuinely useful, though some text is definitional domain detail rather than tool behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first sentence, but the single dense paragraph carries excessive spec-level detail (clock shifted +137 ms, TZ=Pacific/Chatham) that is not needed for tool selection or invocation. The entry-field enumeration is useful, but the repro minutiae crowd out routing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description does the heavy lifting by enumerating entry fields (test, file, state, label, variable, repro command, owner, expiry, issue, flip rate, evidence runs), so an agent can anticipate the return shape. It is nearly complete, missing only pagination/limit and window semantics, which the schema partly covers.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 50%: 'limit' and 'window_days' are documented in the schema, while 'repo' and 'owner' are self-evident required identifiers. The description mentions 'owner' and an evidence window indirectly via 'flip rate and evidence runs', but adds no syntax or format meaning beyond what the schema already provides, so baseline 3 fits.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List a repository's flaky tests') and then defines 'flaky' with concrete repro criteria, so an agent knows exactly what the result set contains. No sibling tool overlaps this concept (nothing else in the list deals with flaky tests), so the lack of explicit sibling contrast is not a gap.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the definition of flakiness and the note that 'unexplained = flipped with no repro, NOT flaky', which tells the agent what is excluded. However, it never says when to reach for this tool versus failure-oriented siblings like gluecron_explain_failure or gluecron_get_workflow_logs, and names no alternatives or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources