# PAKT

PAKT reads contracts - most of them in Azerbaijani, as PDFs or scans - and tells you
where they hurt. It splits a contract into clauses and scores each one from our side of
the deal as high, medium or low risk, with who the clause favours and why. Then it
writes an overall verdict: the top risks and the asymmetries between the parties, which
you can also listen to. For an amendment it finds every conflict with the base contract:
what it contradicts, modifies, adds, or changes in our favour.

- **Ingestion:** PDF text extraction, and OCR for scanned pages and images, in
  Azerbaijani, Russian and English. The text is then split into numbered clauses.
- **Analysis:** LangChain chains over four LLM providers with automatic fallback. Every
  result records which provider, model and prompt version produced it.
- **Web app:** companies, contracts and amendments in Azerbaijani, English and Russian,
  in a day and a night theme. It has a public landing page at `/`, and the app is under
  `/app/`.
- **Claude Desktop:** an MCP server lets Claude list, read, analyse and compare your
  contracts.

Contract text, explanations and verdicts stay in the document's language; only the
interface is translated.

## Architecture

```mermaid
flowchart LR
    browser[Browser] -->|HTTPS| funnel[Tailscale Funnel<br>:443]
    claude[Claude Desktop<br>+ MCP server] -->|HTTPS| funnel
    funnel -->|127.0.0.1:8000| api
    subgraph server[This server - Docker Compose]
        api[FastAPI<br>API + web UI + background jobs]
        api --> db[(PostgreSQL)]
        api --> files[(uploads/)]
        n8n[n8n<br>internal only]
    end
    api -->|LCEL, with_fallbacks| llm[Groq → Gemini → Cerebras → NVIDIA]
    api -->|verdict audio| tts[fish.audio]
```

One FastAPI process serves the API and the static web UI, and runs the background jobs:

1. **Upload.** A document is stored under `uploads/<user>/` and the request returns 202.
2. **Extraction.** pypdf reads the text layer; pages without one go through pytesseract
   OCR (`aze+rus+eng`). The text is split into clauses, or an amendment into items with
   the clause numbers they name.
3. **Analysis.**
   - *Clauses:* scored in a batch through one LangChain LCEL chain per provider, joined
     with `with_fallbacks()`. The chains pace themselves under Groq's free-tier token
     limit and retry what failed everywhere.
   - *Verdict:* written from all the clauses, and streamed to the page over SSE.
   - *Amendments:* checked item by item against the base contract.
4. **Status.** The pages poll until each stage is done or failed. A restart fails
   whatever was left processing, so nothing hangs forever.

Every contract endpoint is scoped to the signed-in user (JWT). The MCP server is a
client of the same HTTP API, so it sees only what its account sees.

| Where | What |
|---|---|
| `app/routers/` | HTTP endpoints |
| `app/services/` | extraction, clause splitting, the pipeline, storage, security |
| `app/chains/`, `app/prompts/` | the LLM chains and their versioned prompt files |
| `app/static/` | the web UI (vanilla HTML, CSS and JS) and the landing page |
| `app/mcp_server.py` | the MCP server for Claude Desktop |
| `migrations/` | Alembic migrations, one per schema change |
| `eval/` | the evaluation script and its results |
| `scripts/` | the demo seed and the MCP token |
| `tests/` | pytest; fake LLM providers, never a real API |

## Running it locally

Requirements: Docker with the Compose plugin.

1. Create your env file and fill in the secrets:

   ```bash
   cp .env.example .env
   ```

   Set `POSTGRES_PASSWORD` (and the same password inside `DATABASE_URL`),
   `JWT_SECRET_KEY` and `N8N_ENCRYPTION_KEY`. None of them has a default:
   Docker Compose refuses to start while `POSTGRES_PASSWORD` or
   `N8N_ENCRYPTION_KEY` is empty, and the API refuses to start unless
   `JWT_SECRET_KEY` is at least 32 characters. Add at least one LLM provider key.
   If `id -u` / `id -g` aren't `1000`, set `HOST_UID` / `HOST_GID` too.

2. Start the development stack and create the tables:

   ```bash
   docker compose up -d --build
   docker compose exec api alembic upgrade head
   ```

3. Open:
   - The landing page: http://localhost:8000/ (the app is under `/app/`)
   - API docs: http://localhost:8000/docs
   - Health check: http://localhost:8000/health
   - n8n: http://localhost:5678

The development api container mounts the source and reloads on change. Host ports are
set in `.env` (`API_PORT`, `POSTGRES_PORT`, `N8N_PORT`), on 127.0.0.1 only.

### Everyday commands

```bash
docker compose logs -f api                  # follow API logs
docker compose exec api alembic revision --autogenerate -m "describe change"
docker compose exec api alembic upgrade head
docker compose exec api pytest              # run tests (uses a separate pakt_test database)
docker compose exec api python -m eval.run_eval --provider all groq gemini nvidia
                                            # score the LLM analysis against samples/expected.yaml
                                            # (real API calls; results go to eval/results/)
docker compose exec api python -m scripts.seed_demo --email you@example.com
                                            # demo companies and contracts, no LLM calls
docker compose down                         # stop (add -v to also wipe the database and n8n data)
```

To add a dependency, run `uv add <package>` on the host (install uv from
https://docs.astral.sh/uv/), then rebuild with `docker compose up -d --build api`.

## Deploying on this server (Tailscale Funnel)

Production runs `docker-compose.prod.yml`, which differs from the development setup:

- **Restarts:** every service restarts itself, after a crash and after a reboot.
- **API:** it runs from its image, with no source mount and no reload. Its entrypoint
  runs the migrations before the server starts.
- **Ports:** only the API is published, on `127.0.0.1:8000`. Postgres and n8n have no
  published port at all.
- **Exposure:** Tailscale Funnel puts that one port on the internet at
  `https://<machine>.<tailnet>.ts.net` - here `https://monoblock.tail1957ed.ts.net`.

Both compose files use the same project name, database and `uploads/`, one at a time.

### 1. Start the production stack

```bash
# in .env: the Funnel URL, the only origin CORS allows
PUBLIC_BASE_URL=https://monoblock.tail1957ed.ts.net

docker compose -f docker-compose.prod.yml up -d --build
docker compose -f docker-compose.prod.yml ps        # all three up; only api has a port
curl -s http://127.0.0.1:8000/health                 # {"status":"ok","database":"ok"}
```

### 2. In the Tailscale admin console (once)

At https://login.tailscale.com/admin:

1. **DNS:** keep **MagicDNS** on, and under **HTTPS Certificates** click **Enable
   HTTPS**. Funnel serves a certificate for the machine's `ts.net` name.
2. **Access controls:** allow Funnel. In the policy file, add a `nodeAttrs` entry
   granting the `funnel` attribute, and save:

   ```json
   "nodeAttrs": [
     { "target": ["autogroup:member"], "attr": ["funnel"] }
   ]
   ```

   To allow only this machine, target its tag or its address instead of every member.
   If this is missing, `tailscale funnel` prints a link to the page that adds it.

### 3. Turn on Funnel, on the server

```bash
sudo tailscale set --operator=$USER    # once: lets your user run serve and funnel
tailscale funnel --bg 8000             # https://monoblock.tail1957ed.ts.net → 127.0.0.1:8000
tailscale funnel status
```

`--bg` keeps the Funnel running in tailscaled, across reboots. It forwards port 443 to
the API only: Postgres and n8n aren't published, so nothing can forward to them. To
take the site off the internet:

```bash
tailscale funnel reset
```

Check from outside the tailnet, for example a phone on mobile data. The landing page
should open, and `curl -I https://monoblock.tail1957ed.ts.net/` should show the
security headers.

### After a reboot

- **Docker** is enabled at boot (`systemctl is-enabled docker`), and so is tailscaled,
  which brings the Funnel back.
- **The containers** restart themselves (`restart: unless-stopped`).
- **The API** migrates on start, and marks any document a previous run left processing
  as failed. Its page offers to upload it again or re-run the analysis.

### Operating it

```bash
docker compose -f docker-compose.prod.yml logs -f api      # logs (rotated, 5 × 10 MB per service)
git pull && docker compose -f docker-compose.prod.yml up -d --build   # update; migrations run on start
docker compose -f docker-compose.prod.yml exec postgres \
  sh -c 'pg_dump -U "$POSTGRES_USER" "$POSTGRES_DB"' > "pakt-$(date +%F).sql"   # back up the database
```

n8n has no published port. To open its editor for a while, forward it to the server's
localhost, and stop with Ctrl-C:

```bash
docker run --rm -p 127.0.0.1:5678:5678 --network pakt_default alpine/socat \
  tcp-listen:5678,fork,reuseaddr tcp-connect:n8n:5678
```

Then open http://localhost:5678 on the server, or `ssh -L 5678:127.0.0.1:5678 <server>`
from your computer. To go back to development, run `docker compose up -d --build`.

### What is in place before exposing it

- **Access:** every contract, company and amendment endpoint is scoped to the signed-in
  user.
- **Rate limits:** per client address, 10 sign-in attempts per 5 minutes and 5
  registrations per hour. Beyond them the API answers 429 with `Retry-After`. The
  limits are set in `.env`.
- **Upload size:** 20 MB per file, 21 MB per request. A larger body gets 413 before it
  is read.
- **CORS:** only `PUBLIC_BASE_URL` is allowed.
- **Headers:** every response carries a content security policy (the app itself plus
  the Google Fonts hosts), `nosniff`, `X-Frame-Options: DENY`, a referrer policy, a
  permissions policy and COOP. HSTS is added over HTTPS.
- **Network:** only the API is published, on 127.0.0.1. Postgres and n8n are reachable
  from the compose network only.
- **Secrets:** they live in `.env`, which is never committed. The logs carry counts,
  provider names and error types. A scan of the API's logs found none of the secret
  values from `.env`, and no tokens.

## Evaluation

`eval/run_eval.py` runs the real pipeline - extraction, the clause chain, the conflict
chain - on the samples in `samples/` and scores it against `samples/expected.yaml`:

- 30 clauses: 19 in the lease and 11 in the scanned IT services contract. 13 of them
  are expected red.
- The sample amendment, whose six expected conflicts are checked.

Measured on 2026-09-11, prompt set v1 (`eval/results/2026-09-11_comparison.md`):

| Run | Model | Risk accuracy | Red recall | False alarms | Favours accuracy | Conflicts found | Extra | Failed calls | Avg latency | Wall time |
|---|---|---|---|---|---|---|---|---|---|---|
| **Full chain** | Groq → Gemini → NVIDIA | **27/30 (90%)** | **11/13 (85%)** | **0** | **28/30 (93%)** | **5/6** | 1 | 0 | 1477 ms | 198 s |
| Groq | openai/gpt-oss-120b | 24/30 (80%) | 10/13 (77%) | 0 | 25/30 (83%) | 5/6 | 1 | 2 | 679 ms | 193 s |
| NVIDIA | deepseek-ai/deepseek-v4-flash-0731 | 24/30 (80%) | 11/13 (85%) | 0 | 23/30 (77%) | 4/6 | 0 | 5 | 2894 ms | 547 s |
| Gemini | gemini-3.5-flash | 0/30 | 0/13 | 0 | 0/30 | 0/6 | 0 | 36 | - | 249 s |
| Cerebras | not configured (the free tier had no model access) | | | | | | | | | |

**How the columns are counted:**

- **Risk accuracy:** a clause counts as right when its level matches, and where yellow
  is expected, red also counts.
- **Red recall:** expected-red clauses found red.
- **False alarms:** red given where green was expected.
- **Conflicts found:** expected conflicts matched by item, clause and type. **Extra** is
  conflicts beyond them.

The full chain beats every provider alone: a call that fails or returns malformed JSON
falls through to the next provider (Groq answered 23 clauses, NVIDIA 7). No clause was
left unscored. Every Gemini call failed during that run, so it contributed nothing.
Wall times are dominated by free-tier rate limits.

## Using PAKT from Claude Desktop (MCP)

`app/mcp_server.py` makes PAKT an MCP server with four tools:

- `list_contracts` - your contracts, filtered by company or status
- `get_contract` - one contract's overview: details, verdict, and every clause's risk
- `analyze_contract` - upload a document (new contract or existing one) and run the pipeline
- `compare_amendment` - an amendment's conflicts with its base contract, uploading it first if asked

It calls the PAKT API as your account, so it sees exactly what you see in the web app.

1. Write a token for your account into `.env`. It is never printed, and is valid for
   90 days unless you pass another `--days`:

   ```bash
   # production
   docker compose -f docker-compose.prod.yml run --rm -v "$PWD/.env:/app/.env" api \
     python -m scripts.mcp_token --email you@example.com --days 90
   # development
   docker compose exec api python -m scripts.mcp_token --email you@example.com --days 90
   ```

2. The computer running Claude Desktop needs [uv](https://docs.astral.sh/uv/) and a
   checkout of this repository whose `.env` has `PAKT_API_URL` and `PAKT_API_TOKEN`:
   - **On this server:** that is the `.env` from step 1, and `PAKT_API_URL` defaults to
     `http://127.0.0.1:8000`.
   - **On another machine:** set `PAKT_API_URL=https://monoblock.tail1957ed.ts.net`, and
     copy the `PAKT_API_TOKEN` line over from the server's `.env`.

3. In Claude Desktop open **Settings → Developer → Edit Config**, which opens
   `claude_desktop_config.json` (`~/Library/Application Support/Claude/` on macOS,
   `%APPDATA%\Claude\` on Windows), and add:

   ```json
   {
     "mcpServers": {
       "pakt": {
         "command": "uv",
         "args": ["run", "--directory", "/absolute/path/to/pakt", "python", "-m", "app.mcp_server"]
       }
     }
   }
   ```

   On Windows, write the path as `C:\\Users\\you\\pakt`. If Claude Desktop can't find
   `uv`, put its full path (`which uv`) in `command`. The first start takes a minute
   while uv installs the dependencies.

4. Restart Claude Desktop. PAKT's tools appear in the tools menu; ask, for example,
   "Which of my contracts have red clauses?" or "Compare the amendments of contract 14
   with the contract."

File paths you give `analyze_contract` and `compare_amendment` are paths on the
computer running Claude Desktop. The token acts as your account until it expires:
writing a new one doesn't withdraw the old one, but changing `JWT_SECRET_KEY`
withdraws every token at once.

## Articles of law

Under a clause's explanation, the text view shows the articles of law the clause relates
to ("Əlaqəli qanun maddəsi"): the article's number and title, the start of its official
text, and a link to the source. The links come from keyword rules in code
(`app/services/law_links.py`), with no LLM call:

- a penalty clause (dəbbə) links to the penalty articles
- a termination clause (xitam) links to the termination article
- lease, services and obligation clauses link to theirs

A clause may link to none. The links are stored when a contract's clauses are made, and
made again for every clause whenever the articles are loaded.

- **The articles** are in `app/data/law/civil_code.yaml`: number, title, topics, source
  address and text. All 10 articles have their official text. PAKT never writes article
  text: it is pasted into each article's `text` field by hand. An article whose text is
  empty stays linked to its clauses but isn't shown until its text is filled in.
- **Loading** them, and linking every clause again, runs on each start of the production
  API, or by hand:

  ```bash
  docker compose exec api python -m scripts.law_articles
  ```

- **A monthly sync** is written but not running: `n8n/workflows/law-articles-sync.json`
  (import it in n8n) would update the text of the articles in the table, in place.
  - **It ships without a source URL**, because it needs a permitted source. e-qanun.az
    serves its documents only to its own pages and answers 403 to direct requests.
  - **Permission first:** use it only with written permission from e-qanun.az (the
    Ministry of Justice) or with a licensed legal database. Never work around their
    check.

## Future work

- **An httpOnly cookie for the JWT.** The web UI keeps the token in `localStorage`,
  where any script running on the page could read it. An httpOnly, Secure, SameSite
  cookie set by the API would keep it out of scripts' reach, at the cost of CSRF
  protection.
- **Paid LLM tiers for speed.** On the free tiers, Groq's 8000 tokens a minute paces
  the clause analysis, so a 19-clause lease takes minutes. With paid limits the batch
  could run at full concurrency and finish in seconds.
- **Per-clause progress.** The processing panel shows the stages: upload, extraction,
  analysis. The pipeline already counts "clause 7 of 19" as results come in; streaming
  that to the page would show how far along it is.
- **Law database integration.** Clauses already link to articles of law by keyword
  rules. Next steps:
  - a permitted sync source to keep the articles' texts current
  - cover the laws on leases and services beyond the Civil Code
  - say in the explanation where a clause departs from the article it links to
- **A summary endpoint for the context column.** The wide-screen context column fetches
  the clauses of every analysed contract, one request each. A single endpoint with each
  contract's risk counts and riskiest clauses would keep that to one request as the
  number of contracts grows.