# brightdata-gatekeeper

[![ci](https://github.com/rycerzes/brightdata-gatekeeper/actions/workflows/ci.yml/badge.svg)](https://github.com/rycerzes/brightdata-gatekeeper/actions/workflows/ci.yml)

**Evidence-based approval for Bright Data Scraper Studio self-healing.**

Bright Data repairs broken scrapers. It does not decide whether a repair is *correct*.
Gatekeeper does — it detects the break, writes the heal brief, then judges the proposed fix
against rows recorded while the scraper was healthy, and rejects it if they disagree.

| | |
| --- | --- |
| Dashboard | <https://gatekeeper.rycerz.es> — 29 collectors, 6,496 rows |
| Mutation target | <https://recalls.rycerz.es> |
| Walkthrough | [docs/DEMO.md](docs/DEMO.md) |
| Demo video | [video/out/gatekeeper-demo.mp4](video/out/gatekeeper-demo.mp4) — 83s, generated from HTML, not screen-recorded |

## The gap this fills

Bright Data's own CLI documentation states it plainly:

> **You are the detector.** The CLI never decides on its own that a scraper is broken —
> you inspect the run output and decide. […] Vague prompts produce vague heals.

`scraper heal` rewrites a collector's template and stops at an approval gate. `approve`
commits it. `--auto-approve` skips the question. Nobody checks the answer.

| Bright Data ships | Gatekeeper adds |
| --- | --- |
| `scraper run` | **Detector** — catches breakage that still returns HTTP 200 |
| `scraper heal "<prompt>"` | **Brief generator** — a typed field list carrying the diagnosis |
| the gate, `approve`, `--reject` | **Approver** — judges the preview against golden rows |

```
run → validate → diagnose → brief → heal → verify → approve or reject → re-brief
```

The cycle is also reachable from nowhere else. Bright Data's Python SDK covers Scraper
Studio's trigger/poll/fetch but not `heal` or `approve`; its 69-tool MCP server has no
Scraper Studio tools at all. Gatekeeper ships an MCP server, so an agent can drive it.

## The result that matters

Against a live collector on a deliberately broken page, Bright Data's heal recovered four
fields correctly and silently dropped the fifth to `null`. `units` is declared optional, so
every schema, type and range check passed — **zero broken findings**:

```
--auto-approve would : APPROVE   preview passes every field check
gatekeeper decided   : REJECT    2/2 golden rows disagree
                                 units: expected '118500', preview has 'null'
```

`--auto-approve` would have committed a scraper that quietly stopped collecting unit
counts. The rejection reason went into the next brief, the retry matched golden on every
compared row, and the fix was verified on a fresh live run.

## Quick start

```bash
uv sync --all-extras
npm install -g @brightdata/cli        # Node >= 20
cp .env.example .env                  # add BRIGHTDATA_API_KEY

gatekeeper doctor                     # check the setup
gatekeeper spec add specs/mutant-recalls.json
gatekeeper status                     # survey every collector
gatekeeper check   <collector_id>     # detect
gatekeeper golden  <collector_id>     # record known-good rows
gatekeeper heal    <collector_id>     # run the loop
gatekeeper history <collector_id>     # why a fix was committed or rejected
gatekeeper serve                      # dashboard on :8000
```

As an MCP server, so an agent can drive the loop:

```bash
claude mcp add gatekeeper -- gatekeeper mcp
```

`preview_heal_brief` shows the prompt that *would* be sent without spending an AI-Flow job.

## Rules the loop obeys

- **Never trust an exit code.** A run returning 200 with empty fields is the failure that matters.
- **Never heal a block.** `blocked` and `dead_page` alarm but are not layout changes.
- **Never approve without evidence.** No golden rows means abstain, not approve.
- **Verify after committing.** A preview is a promise; a live run is evidence.

## Collectors

Four managed collectors, three of them real sites, none in Bright Data's 1,743 pre-built
scrapers — which `gatekeeper coverage <url>` checks rather than asserts.

| Collector | Target | Rows | Notable |
| --- | --- | --- | --- |
| `mutant-recalls` | recalls.rycerz.es | 8 | Breaks on demand; five layouts, identical values |
| `arxiv-cs-lg` | arxiv.org | 950 | Stable identity, so golden rows survive churn |
| `hn-front-page` | news.ycombinator.com | 59 | Nested output, flattened via `row_path` |
| `lobsters` | lobste.rs | 125 | Arrived broken: 114 rows, every field absent |

Details, including the golden-bootstrap problem: [docs/SITES.md](docs/SITES.md).

Beyond those four, the loop was swept across **50 further sites** — one collector each, spec
inferred from its own output rather than hand-written. **20 diagnosed healthy across 5,354
records**, and the sweep found four real defects in Gatekeeper that single-site testing never
exposed, including rate-limited runs reading as healthy. Matrix and failure taxonomy:
[docs/SWEEP.md](docs/SWEEP.md).

## Layout

```
src/gatekeeper/
  detector/    rules, baselines, engine — decides a scraper broke
  brief/       diagnosis → typed heal prompt
  approver/    judges a preview against golden rows
  brightdata/  bdata CLI wrapper, envelopes, coverage check
  controller   the closed loop
  mcp          agent-callable tools
  api, static  the dashboard
mutant/        the mutation target, deployed as a Worker
```

## Tests

```console
$ pytest -q
161 passed
```

No network and no credits: the `bdata` CLI is stubbed at the subprocess boundary and the
loop runs against a scripted client, so the whole state machine — including
reject-and-reconverge — is verified offline.

## Notes

Deployment, and how the dashboard is published without credentials:
[docs/DEPLOY.md](docs/DEPLOY.md). Things the live API taught us that the docs did not:
[docs/FINDINGS.md](docs/FINDINGS.md).

MIT.