# Agent Accessibility Scorer

**Score how usable your website is for AI browser agents — and let Claude Code fix it in a loop.**

AI agents don't look at screenshots. Claude Code driving Playwright MCP, and most agent harnesses like it, read your page's **accessibility tree** as text and act on nodes by reference. Pixels never enter the loop.

So a site can look perfect and be unusable to an agent:

| Renders fine | What the agent receives |
|---|---|
| An icon button with a crisp SVG | `- button` — no name, nothing to target |
| `<div onclick="checkout()">Buy now</div>` | `- generic`, or nothing at all |
| Three pricing cards, each "Learn more" | three identical `- link "Learn more"` |
| A styled custom dropdown | no combobox role, no open/closed state |

axe and Lighthouse score for screen readers. They overlap, but they don't answer *can an agent complete a task here* — three links named "Learn more" pass axe cleanly and break agents constantly.

This does two things:

1. **Web app** — paste a URL, get a 0–100 score, prioritized fixes, and a side-by-side view of your page next to the text an agent actually gets.
2. **MCP server** — the same analysis as agent-callable tools, so a coding agent can run **the improve loop**.

---

## The improve loop

This is the part that makes it more than a linter. A coding agent connects over MCP, gets a report, edits your source, redeploys, and calls back to check whether it actually worked:

```mermaid
flowchart LR
    A["analyze_page(url)"] --> B["report <br/>score · findings · fixes <br/>+ runId"]
    B --> C["agent edits <br/>your source"]
    C --> D["redeploy"]
    D --> E["compare_runs <br/>(baselineRunId)"]
    E -->|"introduced > 0, or below target"| C
    E -->|"clean"| F["done"]

    style A fill:#1e3a5f,stroke:#5b9dff,color:#e6eaf0
    style E fill:#1e3a5f,stroke:#5b9dff,color:#e6eaf0
    style F fill:#14401f,stroke:#51cf66,color:#e6eaf0
```

**There is no loop code in this repo.** The agent *is* the loop — it uses its own editing and deploy tools. What we provide is the four things that make each iteration meaningful:

| | Where |
|---|---|
| A report specific enough to act on — exact selectors, not "add a label" | [`render/markdown.ts`](packages/analyzer/src/render/markdown.ts) |
| Persistence, so a baseline survives a redeploy | [`store.ts`](packages/analyzer/src/store.ts) |
| **Stable finding identity**, so the agent learns *which* defect it fixed | [`score.ts`](packages/analyzer/src/score.ts) |
| The diff: **fixed / persisting / introduced** | [`compare.ts`](packages/analyzer/src/compare.ts) |

`introduced` is the guardrail. A rising score can hide a brand-new critical failure — which is exactly how unsupervised optimization against a metric goes wrong. Findings are keyed by a hash of rule id + selector with `:nth-of-type(N)` stripped, so they survive unrelated markup churn instead of looking "fixed" every time you insert a sibling.

Here's the loop running for real, against the fixture pair — `bad.html` edited into `good.html`, same rendered UI, fixed semantics:

```text
analyze_page  → 34/100 (F)   runId=cc488273   20 findings
compare_runs  → 100/100 (+66)  fixed 20 · persisting 0 · introduced 0
                reachability 60→100 · nameability 24→100 · forms 4→100
```

---

## Quick start

```bash
pnpm install        # also fetches Chromium
pnpm build
pnpm dev            # api :8787, web :5173
```

Open http://localhost:5173 and paste a URL.

![The report: score, category breakdown, and prioritized findings](docs/images/report.png)

The panel that tends to land hardest is the side-by-side — your page on the left, the exact text an agent receives on the right, with unaddressable nodes highlighted. Most people have never seen the second one for their own site:

![Side by side: screenshot vs. the accessibility snapshot an agent reads](docs/images/side-by-side.png)

Production — the API serves the built frontend:

```bash
pnpm build && node apps/api/dist/server.js     # http://localhost:8787
```

## Use it from Claude Code

```bash
claude mcp add agent-a11y -- node /absolute/path/to/packages/mcp/dist/stdio.js
```

Then:

> Analyze https://staging.example.com with agent-a11y, fix the critical findings in this repo, redeploy, and compare against the baseline.

Four tools:

| Tool | What it does |
|---|---|
| `analyze_page` | Load, capture, score. Returns a markdown report with selectors and fixes. |
| `get_agent_snapshot` | Just the raw accessibility snapshot — what you'd be operating on. |
| `list_findings` | Structured JSON, filterable by category and severity. |
| `compare_runs` | Re-analyze and diff a baseline: **fixed / still present / newly introduced**. |

`compare_runs` is what closes the loop. It tracks findings by stable instance id, so it reports *which* problem you fixed — and what you broke.

Deployed remotely instead? The same server is mounted at `POST /mcp` (Streamable HTTP).

---

## What it measures

Six weighted categories, 0–100 each:

| Category | Weight | Question |
|---|---:|---|
| Snapshot reachability | 25 | Can the agent perceive it at all? |
| Element nameability | 25 | Can it unambiguously address what it sees? |
| Form comprehensibility | 20 | Can it understand and complete the inputs? |
| Structure & navigability | 15 | Can it orient itself in the page? |
| State & feedback | 10 | Can it tell what happened after acting? |
| Agent access hygiene | 5 | Is it blocked or slowed before it starts? |

33 rules. Each finding comes with the exact selectors, an explanation written for *agent* failure modes rather than generic a11y boilerplate, and a copy-pasteable fix.

A finding becomes points like this — categories are scored independently and never bleed into each other, so the report can tell you *your structure is fine, your forms are the problem*:

```mermaid
flowchart LR
    F["finding <br/>severity + instances"] --> W["severity weight <br/>× instanceFactor <br/>saturating"]
    W --> P["category penalty <br/>100 − Σ, floored at 0"]
    P --> O["overall <br/>Σ score × weight / 100"]
    O --> G["grade <br/>A ≥90 · B ≥80 · C ≥70 · D ≥60 · F"]

    style F fill:#3d1f1f,stroke:#ff6b6b,color:#e6eaf0
    style G fill:#14401f,stroke:#51cf66,color:#e6eaf0
```

The saturation matters: the first instance of a problem costs half the rule's ceiling, and further instances taper off. Without it an icon grid with sixty unnamed buttons would zero its category and drown out every other signal on the page — the score would stop being informative exactly when it's most needed.

## How it works

Three capture channels, correlated, so the analyzer can spot what's **missing** rather than only what's malformed. Detecting an absence requires knowing what *should* have been there:

```mermaid
flowchart TD
    P["page in headless Chromium"]

    P --> A["A · ariaSnapshot() <br/>the verbatim text <br/>an agent receives"]
    P --> B["B · CDP getFullAXTree <br/>computed role + name, <br/>name sources, <br/>ignoredReasons"]
    P --> C["C · DOM inventory <br/>the counterfactual: <br/>what a human can <br/>see and click"]

    B --> J{"join on <br/>backendNodeId"}
    C --> J
    J --> Q["visible and looks clickable — <br/>what does the agent get?"]
    Q --> R["33 rules"]
    R --> S["findings → score"]
    A --> D["shown verbatim in the UI <br/>and get_agent_snapshot"]

    style A fill:#1e3a5f,stroke:#5b9dff,color:#e6eaf0
    style B fill:#1e3a5f,stroke:#5b9dff,color:#e6eaf0
    style C fill:#1e3a5f,stroke:#5b9dff,color:#e6eaf0
    style Q fill:#4a3a12,stroke:#ffd43b,color:#e6eaf0
```

Four answers to that question, in descending severity: **absent from the tree** → **present but role `generic`** → **correct role, no name** → **correct role, ambiguous name**.

`ignoredReasons` is the most valuable field in the whole capture — when Chrome drops a node it tells you *why* (`notRendered`, `ariaHiddenSubtree`, `uninteresting`), which is a direct answer to "why can't the agent see this button" and isn't available any other way.

Full engineering reference: **[docs/DESIGN.md](docs/DESIGN.md)**.

## Architecture

```mermaid
flowchart LR
    U["person"] --> W["apps/web <br/>React + TS"]
    CC["coding agent <br/>(Claude Code)"] -.->|"MCP stdio"| M["packages/mcp <br/>4 tools"]
    CC -.->|"MCP over HTTP"| API

    W -->|"JSON"| API["apps/api <br/>Fastify <br/>REST + /mcp"]
    API --> M
    M --> AN
    API --> AN["packages/analyzer <br/>analyze() <br/>capture → rules → score"]
    AN --> CR["headless Chromium <br/>Playwright + CDP"]
    AN --> ST[("run store <br/>baselines for <br/>compare_runs")]

    style AN fill:#1e3a5f,stroke:#5b9dff,color:#e6eaf0
    style CC fill:#4a3a12,stroke:#ffd43b,color:#e6eaf0
```

```
packages/analyzer/   the engine — URL in, scored result out. No HTTP, no React.
packages/mcp/        MCP server (stdio + Streamable HTTP)
apps/api/            Fastify — REST + /mcp + static hosting
apps/web/            Vite + React + TypeScript
```

The web app, the REST API, and every MCP tool call one `analyze()` function. There is no second code path — if the web report and the MCP report ever disagree, that's a bug, not a config difference.

## Tests

```bash
pnpm test           # 65 tests
```

The headline is the **paired fixture test**: [`bad.html`](packages/analyzer/test/fixtures/bad.html) and [`good.html`](packages/analyzer/test/fixtures/good.html) render *identically* and differ only in semantics. Analyzed in a real browser, `bad` scores **34/100 (F)**, `good` scores **100/100 (A)**, and every finding in `bad` is absent from `good`.

59 of the 65 need no browser — captures serialize to JSON, so the whole rule engine tests against fixtures in milliseconds.

## Configuration

| Variable | Default | |
|---|---|---|
| `PORT` / `HOST` | `8787` / `0.0.0.0` | |
| `AGENT_A11Y_DATA` | `./data` | run storage |
| `AGENT_A11Y_CONCURRENCY` | `2` | parallel browsers |
| `RATE_LIMIT` | `30` | requests/min/IP |
| `AGENT_A11Y_ALLOW_LOCAL` | unset | permit private addresses (dev only) |

URLs are checked against private address ranges **after DNS resolution**, and again after redirects.

## License

MIT.