# Autopia

Autopia empirically tests HTTP APIs that ship an OpenAPI spec, discovers how they
**actually** behave, verifies every finding by reproducing it, and serves the result as a
versioned, queryable knowledge base that AI agents consult over MCP, and that a Claude Code
plugin injects into an agent's context at the moment it is about to call one of those APIs.

It is the MVP of the "Discrimen Compendium" described in `docs/compendium-spec.txt`
(the blueprint) and `docs/compendium-stack.txt` (stack choices). Read the spec's glossary
first; the code uses those exact terms. `HANDOFF.txt` holds the running state and decisions.

**Live:** the hosted server is at `https://autopia-mcp.onrender.com/mcp` (per-user keys, see
[Identity](#identity-per-user-keys)); the database is a managed Postgres on Neon; a GitHub
Actions schedule refreshes every target nightly (rules only) and weekly (with the model pass).
The catalogue currently holds 16 APIs, including GitHub REST, the npm registry, PyPI, the
Anthropic models endpoints, the Open-Meteo family, weather.gov, Binance, Frankfurter,
Open Library, PokeAPI and Nager.Date.

## What an agent gets

A **behavior sheet** per API: Markdown generated from verified claims only, ordered by value.
Where the live API contradicts its own spec comes first, then rules the spec never states,
then behavior common to every endpoint, then per endpoint: what responses look like,
error-body shapes, pagination, validation, auth. Every line ends with the trial ids that back
it. Examples of what the sheets record, all reproduced on the live APIs:

- GitHub silently applies `since=not-a-date` on commits as a filter that matches nothing
  (200, empty list), while issues rejects the same value with a 422 that spells out the format.
- GitHub accepts a missing token on public reads but serves the request from the 60/hour
  tier instead of 5000/hour; `per_page` is clamped to 100 (proved by the Link header).
- npm search accepts `size=2500` against a documented maximum of 250, and does not type-check
  its score-weight parameters.
- The Anthropic `/v1/models` cursors must name an existing model id; an unknown id is a 400
  as a cursor but a 404 as a path.

## The pipeline

```
discover -> exercise -> aggregate -> cross-ref -> verify -> write (+ diff)
 spec       TRIALS      rules +      spec +       evidence   CLAIMS
 fetch,     (no LLM)    shape rules  siblings +   check +    + DIFFS
 hash,                  + one LLM    self-        reproduce
 trim                   pass         consistency
```

- **Trials** are pinned requests generated by the **battery** (`src/worker/battery`,
  version 1.3.0), a deterministic catalogue of edge cases: omit required, wrong type, enum
  outside the documented set, null and empty values, numeric boundaries (zero, negative,
  max int, below the minimum, just above and far above the maximum, 2048-character strings),
  duplicate parameters, unknown fields, bad and missing credentials (scheme-preserving), page
  zero, a huge page size, a page beyond the last, four date formats for date-time parameters
  (date only, no timezone, epoch, garbage), wrong content type, trailing slash, HEAD. No
  randomness anywhere, so two runs produce identical trial inputs. Every stored response keeps
  the lengths of its arrays before truncation, so list semantics survive storage.
- **Claims** come from three sources: deterministic **rules** over the trials (free),
  **shape rules** over successful responses (content type, timestamp formats, null-valued
  fields, pagination fields, rate-limit and cache headers), and **one** batched LLM pass for
  open-ended discovery. The LLM output is validated with ajv against
  `src/schemas/claim.schema.json`; invalid objects are discarded. Rules classify what came
  back, not just the status: a 200 on a bad date value is "applied as an impossible filter"
  when the list is empty where the comparison request had rows, "apparently ignored" when the
  counts match, "interpreted" otherwise. A page-size cap is claimed only when more items
  provably exist (a Link `rel="next"`, a total field, or the documented maximum reached).
  Error-shape claims count only probes that sent a bad value, not auth, HEAD or content-type
  probes.
- **Verification** is the product: every claim cites trial ids and the status each showed.
  The evidence check is mechanical; reproduction re-runs the exact inputs and compares a
  normalized response **signature** (status, body kind, JSON shape, error message), never
  raw bytes, so live data can change without breaking reproduction. Non-reproducing claims
  are discarded. Stored claims are always `verified = true`.
- **Divergences** are claims that contradict a documented fact (required flag, type, enum,
  range). The money view. Silence in the spec is never a divergence: a rule the API enforces
  that the spec never states (parameters that must be sent together, implicit defaults) is
  stored under its own kind, `undocumented_constraint`.
- **Claim kinds:** `required`, `type`, `auth`, `pagination`, `error_behavior`,
  `response_shape`, `undocumented_constraint`, `undocumented`, `other`.
- **Sibling comparison.** Tools that share a `category` are compared probe by probe: the
  same rule on the same parameter name with the opposite verdict marks the claim
  `divergence_type = sibling` with a note naming the other tool.
- **Carry-forward.** Rules-only runs re-admit the previous run's LLM claims as candidates,
  remapped to tonight's trials by endpoint and probe, and send them through the same evidence
  check and reproduction. A carried claim is dropped when a fresh LLM claim on the same
  endpoint cites the same probes (the model restates findings in new words), and carried
  claims that duplicate each other keep the more confident one.
- **Diffs** compare claim sets across tool versions by stable `claim_key`. A tool gets a new
  version whenever its spec hash changes; large specs are stored trimmed to the exercised
  paths, with the hash still over the full document.

## The scenario battery: how an API behaves in use

The probe battery changes one parameter of one request and judges the answer against the spec.
That finds where the docs are wrong, but only along lines the docs already drew. The **scenario
battery** uses an API the way a program does, several requests with state flowing between them,
and judges the outcome against an **invariant** that must hold for any sane API, so it needs no
documentation to compare against and can learn what the docs never say.

Scenario 1, **pagination completeness**, runs on every endpoint whose baseline answered with a
list and that takes a page number. Two walks of a few small pages, one by page number and one
by following the `Link` header's `rel="next"`, plus one page read twice:

- each item appears exactly once across pages (or which ids repeat, and on which pages);
- the Link chain visits the same pages as counting up (or where it diverges, or that there is
  no Link header at all and paging must be driven by number);
- a page read twice a few seconds apart is identical (or the list is live-ordered and shifts);
- the requested page size is what comes back (or it is silently clamped);
- how the list ends: a short page, an empty page, no `rel="next"`.

Every request is a trial; every claim cites all of them and is reproduced by VERIFY like any
other. A finding such as duplicates is claimed only when **both** walks show it, so the second
walk reproduces the invariant itself. Budget: two walks of 4 pages of 5 (`scenarios.pagination:
{ pages, page_size }` in the target, or `false` to skip), ~8 requests per endpoint. The sheet
shows these under **How it behaves in use**; the brief sheet carries them as pagination watch-outs.

Scenarios 3 and 6, **conditional requests and rate-limit accounting**, run on every endpoint
with a working baseline, at most four requests each: a fresh GET, the same GET with
`If-None-Match` set to the ETag it returned, the same with a stale tag, and (when Last-Modified
was sent) one with `If-Modified-Since`. From that sequence:

- whether `If-None-Match` yields 304 with an empty body, and that a stale tag still yields 200
  (so the 304 was really conditional), or that the ETag is ignored, or that there are no
  validators at all and every poll is a full request;
- whether a 304 **costs a request** against the rate limit or is free;
- whether each full request decrements the remaining counter by exactly one with the reset
  unchanged (a fixed window), what format the reset value is in, and what the limit is.

Statements name no volatile numbers, so they stay stable across nightly runs. Disable with
`scenarios.conditional: false` in the target. On GitHub this found that `/rate_limit` is not
counted, that search has its own pool of 30 and no validators, and that every other endpoint
honors ETags with free 304s.

Scenario 4, **filter composition**, runs on list endpoints that have filter parameters with
known-good values in the target: one unfiltered page, one page per filter, one per pair. A
filter is "honored" when every returned item matches the field of the same name; a mismatch is
never read as "not honored" (GitHub's commits `sha` is a start ref, not an item predicate) but
as "no visible effect" or "does not select by that field"; combinations are checked for
containment against complete single results and for field agreement only on filters whose own
check passed. Scenario 2, **list vs detail**, finds the detail endpoint in the spec (list path
plus one template segment), reads two items' details and says which fields the detail adds,
which the list alone carries, and whether shared scalar values agree, so a program knows when a
second request is needed. Disable with `scenarios.filters: false` / `scenarios.list_detail: false`.

Scenario 5, **sort contract**, runs on list endpoints with a sort parameter: one unsorted page,
then one page per sort key (the target's value plus the spec's enum, at most three) per
direction, plus one repeat. It says whether each page is actually ordered by the field the key
names, in the stated direction; which sorted page the default order equals; whether the two
directions differ; and whether the same sort returns the same order twice. On GitHub issues it
found that `sort=comments&direction=asc` does not return the page in ascending order of the
items' `comments` field, while every other key and direction does. Disable with
`scenarios.sort: false`.

Next scenarios: encoding and boundaries. All read-only, none needing an account; the write
battery comes after.

## Health (for the maintainer)

`npm run autopia -- tool-delete <name> --yes` removes a dead tool and everything under it
(versions, runs, trials, claims, diffs) in one transaction; use it for an API that turned out to
be unreachable or was run once by hand and never scheduled.

`npm run autopia -- health` prints one line per tool that needs attention: **red** when a
scheduled tool's last claim-producing run is older than 3 days, when the latest run kept fewer
than half the claims of the best of the previous runs (or none at all), when the last three runs
all failed, or when a scheduled tool has had three runs and never stored a claim; **warn** when
only the latest run failed (one bad night is routine), when a tool has no target file and is
never refreshed, or when an unscheduled tool never stored a claim. The nightly `run-all` prints the same block at the end
and exits non-zero only on red, so a single transient failure no longer fails the workflow and
a quietly decaying tool does.

## Freshness

Every claim records when its evidence last reproduced. A full sheet says on which date its
claims were last re-executed against the live API; a claim not re-run in the last 3 nights says
so after its evidence ids; a sheet nobody has re-run for 3 nights carries a **Stale** banner; a
run that kept fewer than half the previous successful run's claims carries a **Coverage dropped**
banner, because missing is not disproved. The brief sheet mentions age only when something is
stale. `list_tools` rows carry `last_verified_at` / `verified_days_ago`; `search_claims` and
`get_tool_knowledge` records carry `verified_days_ago`. Ages are computed when a sheet is
rendered, so claim statements, which are diffed nightly, never change.

## Quick start

```bash
npm install
npm run autopia -- run targets/open-meteo-forecast.json     # full pipeline (LLM pass if a key is set)
npm run autopia -- docs open-meteo-forecast                 # the behavior sheet
npm run autopia -- claims open-meteo-forecast --divergences
npm run autopia -- search "is latitude required"
npm test
```

Configuration lives in a local `.env` file (gitignored), read on startup:

| Variable | Purpose |
|---|---|
| `ANTHROPIC_API_KEY` | enables the LLM pass; without it the pipeline runs rules-only and says so. Also the credential for the `anthropic-api` target |
| `AUTOPIA_MODEL` | model for the LLM pass, default `claude-opus-5-5` |
| `DATABASE_URL` | Postgres connection string; unset = embedded PGlite in `./data/pglite` |
| `GITHUB_TOKEN` | credential for the `github-rest` target (a fine-grained, public-read-only token is enough) |
| `AUTOPIA_MCP_TOKEN` | admin token for the HTTP MCP server (per-user keys work without it) |
| `AUTOPIA_MCP_OPEN` | `1` opens `/mcp` without any credential; tests only, never in a deployment |
| `AUTOPIA_MCP_PORT` | HTTP server port, default 4200 |
| `AUTOPIA_LOG` | `debug` for per-trial logs |
| `AUTOPIA_OFFLINE` | `1` makes the HTTP server answer 503 to everything but `/healthz` without opening the database (the kill switch) |
| `RENDER_API_KEY`, `NEON_API_KEY` | optional; let `npm run kill` / `npm run revive` suspend and resume the hosting from one command |

Target files reference credentials as `${ENV_VAR}` in headers, so a committed target never
contains a key.

**PGlite caveat:** the default database is an embedded Postgres (PGlite) under `data/pglite`
and only one process can open it at a time. Stop the worker before starting the MCP server
or dashboard. Set `DATABASE_URL` to a real Postgres to lift that limit (same schema, no code
change). `npx tsx scripts/whichdb.ts` says which one a command would use.

## Serving the compendium

```bash
npm run mcp          # MCP server over stdio (register in Claude Code / any MCP client)
npm run mcp:http     # the same server over Streamable HTTP at http://localhost:4200/mcp
npm run dashboard    # read-only dashboard at http://localhost:4100 (local only; not deployed)
```

Register the stdio server with a client as `npx tsx <repo>/src/server/mcp.ts`, or point an
HTTP-capable client at `/mcp` on the HTTP server (`GET /healthz` for liveness). The server
ships an **instructions block** telling an agent when to consult it: before its first request
in a session to a covered host, and after an API call fails or surprises it.

| MCP tool | Use |
|---|---|
| `get_tool_docs` | the behavior sheet; call it before the first request to a covered API. `brief: true` for divergences, undocumented rules, auth, pagination and errors only |
| `search_claims` | the error path: full-text search with host + symptom, e.g. "github per_page 500" |
| `get_divergences` | only the claims where the live API contradicts its spec |
| `get_tool_knowledge` | every verified claim as structured records, paginated, filter by kind |
| `list_tools` | the catalogue, each tool with the **hosts** its trials hit |
| `list_versions`, `get_version_diff` | spec versions, and what changed in behavior between two |
| `request_target`, `get_request_status` | ask for an API that is not covered yet (needs a key) |

The `tool` argument of every tool accepts a tool name, a host, or the URL you are about to
call (`api.github.com`, `https://registry.npmjs.org/-/v1/search`). The compendium is a
directory, not a dump: `get_tool_docs` with a URL or `path` returns **only the endpoint that
path matches** plus a one-line index of the tool's other endpoints, so an agent descends to
what it needs instead of receiving everything. Every sheet ends with a coverage section: which
endpoints of this API are probed, which other APIs exist (by host), what is not probed anywhere
yet, and the tools to fetch more, so a reader of one sheet knows what else it can ask for. Evidence is compact (trial
id, probe, status, request line) unless `verbose=true` asks for response previews. Stored
request inputs are redacted before storage, so no credential can be served as evidence;
`npx tsx scripts/scan-stored-secrets.ts` audits the database for anything that slipped through.

## Claude Code plugin: the sheet at the moment of use

An agent only reads a sheet if something tells it to. The plugin in `plugin/` makes that
deterministic: a **PreToolUse hook** on `Bash` and `WebFetch` matches the command or URL
against the covered hosts and injects the behavior sheet for that endpoint into the model's
context before the call, and a **PostToolUse hook** injects the verified claims that match a
4xx/5xx when a call to a covered host fails. Both fail open on any error. It also registers the
hosted MCP server for digging further and ships a skill carrying the same trigger text.

```bash
claude plugin install autopia --marketplace MaydayTrey/autopia
claude plugin configure autopia        # optional: a per-user key raises your quota; reads work without one
```

The hook fires only on URL-shaped mentions (`https://host/...` or `host/path`) and injects the
**brief** sheet **for the endpoint in that URL**: spec contradictions, undocumented rules, auth,
pagination and error behavior for that endpoint, tool-wide auth behavior, and a one-line index
of the other endpoints. Once per host and endpoint per session; well under 5KB.

**Project-level instructions.** A line in a repository's `CLAUDE.md` or `AGENTS.md` is
followed more reliably than any server instructions block. For a project that integrates with
a covered API, add:

```
Before the first call to api.github.com (or registry.npmjs.org, pypi.org, api.anthropic.com) in a
session, read its verified behavior sheet: Autopia MCP tool get_tool_docs {"tool":"<the URL you are about to call>"}.
On an unexpected 4xx/5xx or an empty result from one of those hosts, call search_claims with the host
and the symptom before retrying.
```

The plugin reaches Claude Code only. Cursor, Codex, Claude Desktop and any agent that reads
`AGENTS.md` get the same behavior from the hosted server plus a paste-in block; the blocks
are in [docs/distribution.md](docs/distribution.md#5-other-agent-hosts-cursor-codex-claude-desktop-anything-that-reads-agentsmd).

For a session-only trial from a checkout: `claude --plugin-dir ./plugin`. Until the key is
configured, the session shows the `autopia` MCP server as failed to connect (401) and the hook
injects a setup note instead of a sheet; both clear after `claude plugin configure autopia` and
a new session. Details, state locations and the privacy note are in `plugin/README.md`.

## Static pages: where agents already look

A web search for an error string should land on a page that cites reproduced trials, with no
plugin and no MCP connection. `npx tsx scripts/build-site.ts --out site --base <url>` renders
the compendium as a static site: a catalogue, one page per API, one per endpoint, an index of
every observed error message with what triggers it, `llms.txt` (the AI-crawler index), a sitemap
and robots.txt. Pages are the behavior sheets with the lightest HTML wrapping; every page has a
plain-text `.md` twin and names the MCP call that returns the same thing live.

`.github/workflows/pages.yml` builds it from the hosted database daily after the refresh (and
on demand) and publishes to GitHub Pages. One-time setup: repository Settings -> Pages ->
Source: GitHub Actions. The site is at `https://<owner>.github.io/<repo>/`.

## Data transfer (the metered resource)

Neon's free plan bills reads: 5 GB of data transfer a month, against 0.5 GB of storage. The
database itself is small (about 90 MB after ten days of nightly runs); what costs is reading
it. Three readers used to dominate and were cut on 2026-10-20: evidence trials are now read
without their response bodies unless a keyed caller asks for previews (the response column is
most of the trials table); the site build loads each tool's knowledge once instead of once per
page (about 50 MB a day down to 2 MB); the catalogue counts claims in SQL instead of loading
every claim row; and the server's response cache lives two hours instead of ten minutes.
Ad-hoc commands that render sheets or scan trials read the hosted database too, so point
`DATABASE_URL` at a local PGlite directory for experiments.

## Kill switch

If traffic ever gets out of hand, there are three levels, each reversible, none of which can
create a bill (Render and Neon have no card on file; the only billed account is the Anthropic
key used by the weekly LLM pass, which has its own spend limit in the Anthropic console):

1. **Stop serving, keep the process:** set `AUTOPIA_OFFLINE=1` in the Render service's
   environment. Every request except `/healthz` gets 503 with a retry-after and the database is
   never opened. The static site keeps serving the sheets. Remove the variable to resume.
2. **Stop the hosting:** `npm run kill` suspends the Render web service and the Neon compute
   endpoint through their APIs (needs `RENDER_API_KEY` and `NEON_API_KEY` in `.env`; the service
   and endpoint are discovered automatically). `npm run revive` brings both back;
   `npm run kill:status` shows the state. Without the keys the command prints the dashboard
   steps instead.
3. **Stop the spend:** disable the `run-all-llm` workflow under Actions. That is the only job
   that calls a paid API.

## Keeping it current (automation)

```bash
npm run run-all                        # every targets/*.json in sequence, one summary line per tool
npm run run-all -- --no-llm --prune-days 14
```

`run-all` first processes pending user requests, then re-fetches each spec, creates a new
tool version when the spec hash changes, re-exercises, re-verifies, and computes the diff
against the previous version. It exits non-zero if any target fails. Targets that are
unreachable are abandoned early rather than timing out trial by trial.

**Schedule.** `.github/workflows/run-all.yml` runs rules-only nightly (free);
`run-all-llm.yml` runs with the model weekly; both read `DATABASE_URL`, `ANTHROPIC_API_KEY`
and the built-in `GITHUB_TOKEN` from repository secrets and can be triggered by hand from the
Actions tab. The model pass is the only cost that scales with targets; carry-forward keeps its
claims alive between weekly runs. **Retention:** `--prune-days N` (or `autopia prune N`)
deletes runs older than N days except the latest finished run of each version and any run a
diff cites; `--dry-run` reports without deleting. Elsewhere, use whatever scheduler you have
(Task Scheduler, cron, a scheduled job on your host) with the same command.

## User-requested targets

Anyone with a key can ask the compendium to test an API it does not have yet, from inside
their agent, with the `request_target` MCP tool. The request is validated on the spot: the
spec is fetched and parsed, the battery is planned, and the user gets back how many endpoints
and trials it will take. Scope is enforced at submission, not by review:

- public http(s) hosts only, for the spec and for every server it names (no localhost,
  private ranges, IP literals, or `.internal`-style names: the worker must never become a
  proxy into wherever it runs)
- keyless: targets with headers are refused; credentials are the contributor tier's job
- politeness we control: concurrency 1, at least 300ms between calls, at most 300 trials
- a free tool name, and a quota of 3 requests per key per day

Pending requests are processed rules-only at the start of the next `run-all` pass, or by hand
with `npm run autopia -- requests process`. Once done, the stored target joins every later
pass like a file target. `get_request_status` reports pending / running / done / failed;
`npm run autopia -- requests list` shows them all.

## Keyed targets: two routes

**Server tier, credential in Actions secrets.** `github-rest` and `anthropic-api` are keyed
targets the server runs itself: the target file names the header as `${GITHUB_TOKEN}` or
`${ANTHROPIC_API_KEY}`, the workflow passes the secret, stored trial inputs are redacted by
header name, and reproduction restores the real value from the target in memory. The secret
is on the wire to the API and nowhere else. The battery's fake credential for bad-auth probes
is deliberately not redacted, so reproduction can tell it apart from the real key.

**Contributor tier, credential never leaves the contributor.** For APIs whose key cannot
live in this repository's secrets, the same open-source worker runs on the contributor's
machine with their API key, their model key and their own database, then uploads the result:

```bash
npm run autopia -- contribute my-keyed-api --to https://autopia-mcp.onrender.com --key ak_... --yes
```

What gets uploaded, by construction: claims, and evidence as status code plus response
signature (a type shape with no values in it). No request inputs, no headers, no response
bodies, no error-message text. Claim statements are scrubbed twice, locally and on the
server: anything that looks like an email, a token, a bearer credential, a long hex string or
a long identifier number drops that claim. Without `--yes` the command prints every statement
it would upload and stops. Contributor tools are a visible trust tier: their sheet opens with a
banner naming the contributor key prefix and the date the claims last reproduced, and states
that this server has not re-verified them since. A tool this server verifies itself can never
be overwritten by a contribution.

## Hosting: managed Postgres + the HTTP MCP server

The embedded database is for one laptop. A hosted compendium uses a managed Postgres so the
worker and the server can run at the same time, and runs the HTTP MCP server somewhere always on.
The code path is identical: `DATABASE_URL` switches drivers, migrations apply on connect, and
`tests/integration/postgres-path.test.ts` exercises it over the real Postgres wire protocol.

**1. Database (Neon or Supabase, both include pgvector).** Create a project, copy the
connection string, and move an existing compendium across:

```bash
npx tsx scripts/db-export.ts data/export.json
```

```bash
DATABASE_URL="postgres://..." npx tsx scripts/db-import.ts data/export.json
```

Import preserves ids, skips rows that already exist, and resets the sequences, so it is safe
to re-run. Put the same `DATABASE_URL` in `.env` and every command on the laptop now talks to
the hosted database.

**2a. Server on Render (free tier, no card; the current deployment).** `render.yaml` is a
Blueprint. In Render: New -> Blueprint -> select the repo. It builds the Dockerfile, asks for
`DATABASE_URL` and generates `AUTOPIA_MCP_TOKEN` (copy it from the service's Environment tab).
Free services sleep after ~15 idle minutes and wake on the first request, so expect a slow
first call; the plugin's hook tolerates that by failing open and retrying on the next call.
Render auto-deploys from `main`.

**2b. Server on Fly.io (paid account, always-on).** `fly.toml` and the `Dockerfile` are in the repo:

```bash
fly launch --copy-config --no-deploy
```

```bash
fly secrets set DATABASE_URL="postgres://..." AUTOPIA_MCP_TOKEN="$(openssl rand -hex 24)"
```

```bash
fly deploy
```

The server listens on `/mcp` with `GET /healthz` for the platform check. The image also
contains the worker, so `run-all` can run as a scheduled machine or by hand. Railway works the
same way from the same `Dockerfile`.

### Identity: anonymous reads, keys for writes

**Reads are open, within a budget.** The seven read tools answer without any credential, under
a per-IP limit (`AUTOPIA_ANON_PER_MINUTE`, default 30; `AUTOPIA_ANON_PER_DAY`, default 300) and a
global daily cap (`AUTOPIA_ANON_DAILY_CAP`, default 5,000). Anonymous callers never receive
response previews and their page sizes are capped. Read responses (sheets, catalogue, searches,
claims) are served from an in-memory cache for ten minutes (`AUTOPIA_CACHE_TTL_MS`), so repeated
anonymous calls do not reach the database. The defaults are sized for the free tiers underneath:
5,000 brief sheets a day is roughly 75 MB of egress and a few hundred database round trips, well
inside Render's and Neon's free allowances; raise the numbers when the hosting can take it. When
the cap is hit, anonymous callers get 429 with a retry-after; keyed callers are unaffected.
Public API behavior does not need per-user identity to read. `AUTOPIA_ANON_READS=0` restores
key-only access. `request_target` and `/contribute` always need a key.

**Keys** give a caller its own quota, a name in the usage log, and the write tools:

```bash
npm run autopia -- keys create alice --quota 2000
```

The key (`ak_...`) is printed once; only its hash is stored. Users send it as
`Authorization: Bearer ak_...`. `keys list` shows prefixes, quotas, and call counts;
`keys revoke <name-or-prefix>` disables one; a key over its daily quota gets 429. Every tool
call is logged with the key, the tool, the target asked about, success, and latency, and
`npm run autopia -- usage --days 7` summarizes it per key and per most-asked-about tool.

`AUTOPIA_MCP_TOKEN`, if set, works as an admin token (no quota; compared in constant time;
calls logged without a key). `AUTOPIA_MCP_OPEN=1` disables the anonymous limits entirely (tests
only). Keys are issued by the maintainer today; self-service issuance is on the roadmap.

## CLI

| Command | What it does |
|---|---|
| `run <target.json> [--no-llm] [--max-trials N] [--no-reproduce]` | full pipeline on one target |
| `run-all [dir] [--no-llm] [--prune-days N] [--dry-run]` | pending requests, then every target in `dir` (default `targets/`) |
| `plan <target.json>` | print the trial plan without sending anything |
| `tools`, `versions <tool>`, `runs <tool>` | catalogue |
| `docs <tool> [--version ID]` | the behavior sheet as Markdown |
| `claims <tool> [--divergences] [--version ID] [--json]` | verified claims with evidence |
| `search "<question>" [--tool NAME]` | text search over claims |
| `diff <tool> <fromVersionId> <toVersionId>` | changelog between versions |
| `trial <id>` | one trial in full |
| `compare-runs <tool>` | determinism check across the last two runs |
| `requests list`, `requests process [--llm]` | user-requested targets |
| `keys create <name> [--quota N]`, `keys list`, `keys revoke <name-or-prefix>` | per-user keys |
| `usage [--days N]` | calls per key and per tool |
| `contribute <tool> [--to URL --key ak_... --yes] [--out file]` | build (and upload) a contributor bundle |
| `prune [days] [--dry-run]` | retention |

`AUTOPIA_LOG=debug` prints every trial and every verification decision. Helper scripts under
`scripts/`: `run-progress.ts` (watch a run), `fail-stale-runs.ts` (mark interrupted runs),
`inspect-repro.ts` and `auth-probe-check.ts` (debug reproduction and auth probes),
`plan-dry-run.ts`, `db-smoke.ts`, `whichdb.ts`, `scan-stored-secrets.ts`.

## Adding a target

Create `targets/<name>.json` (the file name is the tool name):

```json
{
  "name": "my-api",
  "category": "weather",
  "spec_url": "https://example.com/openapi.yml",
  "include_paths": ["/v1/path"],
  "baseline": { "/v1/path": { "required_param": "known good value" } },
  "optional_values": { "/v1/path": { "optional_param": "known good value", "since": "2026-01-01T00:00:00Z" } },
  "headers": { "x-api-key": "${MY_API_KEY}" },
  "rate": { "concurrency": 1, "min_interval_ms": 500 },
  "rate_overrides": { "/v1/search": { "min_interval_ms": 2500 } },
  "server": "https://api.example.com",
  "max_trials": 300
}
```

- `include_paths` limits a large spec to the endpoints worth probing (GitHub's spec has 816
  paths; the target exercises 7). The stored spec is trimmed to those paths plus the components
  they reference, while the version hash covers the full document.
- `rate_overrides` slows down individual paths with their own quota (GitHub search: 30/min).
  The executor also treats a 403 with `x-ratelimit-remaining: 0` as rate limiting and sleeps
  until the reset.
- `server` supplies a base URL when the spec declares none; `override_server: true` replaces
  the declared one (a regional mirror).
- A target may point at a spec file in the repo instead of a URL. `specs/` holds hand-authored
  OpenAPI descriptions for APIs that publish none (npm registry, PyPI, the Anthropic models
  endpoints); divergences reported against those mean "the API disagrees with our description",
  and the file header says so.
- `targets/parked/` holds targets that are temporarily out (see its README for why).

Scope rule (v1): read-only GET endpoints of OpenAPI 3.x APIs, public or behind a single
credential header. Write endpoints and JSON request bodies are not probed yet. If a target
lacks a formal spec or has side effects, it is out of scope; `discover` stops with an error.

## Testing

`npm test` runs everything (130 tests); `npm run test:unit` and `npm run test:integration`
split them.

- Unit, one file per stage under `tests/unit`: spec parsing, battery, signature, HTTP executor
  (fake fetch), rules, shape rules, cross-ref, carry-forward, verify, diff, docs rendering,
  redaction, keys, contribution scrubbing, host resolution, the plugin hook's helpers.
- Integration, under `tests/integration`: the real pipeline against a **mock API with planted
  quirks** (`tests/mock-api/server.ts`) on a throwaway database; the MCP server over stdio and
  over HTTP (open, admin token, per-user keys); user requests; contributions; retention;
  unreachable targets; the Postgres wire path through PGlite; and the plugin hook run as a
  child process against a real HTTP server.
- `npm run mock-api` starts the mock API by hand for manual poking.

The Postgres-path test occasionally fails in its setup under the full parallel run and passes
alone; re-run before suspecting a change.

## Layout

```
docs/                  spec + stack docs
targets/               one JSON per API under study (parked/ for the ones on hold)
specs/                 hand-authored OpenAPI docs for APIs that publish none
drizzle/               generated SQL migrations (npm run db:generate after editing schema.ts)
src/db/                schema.ts (source of truth), client.ts (PGlite or Postgres), queries.ts,
                       docs.ts (behavior sheets), keys.ts, requests.ts, contributions.ts, retention.ts
src/worker/            pipeline stages: discover, exercise, aggregate/{rules,shape-rules,llm,carry},
                       crossref, verify, write, diff, pipeline.ts, cli.ts; battery/; signature,
                       http, redact, target, requests-runner, contribute
src/llm/               provider adapter (worker only, never the server)
src/schemas/           claim.schema.json
src/server/mcp.ts      MCP server (stdio and Streamable HTTP), instructions block, /contribute
src/dashboard/         Express + static page (local use)
plugin/                Claude Code plugin: PreToolUse hook, .mcp.json, skill, README
.claude-plugin/        marketplace.json (install source for the plugin)
.github/workflows/     test on push/PR; nightly rules-only and weekly LLM refresh
scripts/               export/import, audits, run helpers
tests/                 unit, integration, mock-api, helpers
```