# chiron-mcp

An [MCP](https://modelcontextprotocol.io) server that exposes **Chiron**, a cohort-discovery
platform for human-subject research data, to an LLM client such as Claude.

Ask *"how many asthma patients are there, and what were their diagnosis dates?"* and the model
finds the right variable, builds the cohort, counts it, and pulls the rows, without anyone
writing SQL, and without bypassing Chiron's own access rules.

```
"How many female patients have asthma?"         -> 445        [Query · 445]
"How many patients do not have hypertension?"   -> 7,861      [Query · 7,861]
"Asthma patients earning over $50,000"          -> 702        [Query · 702]
"Break down asthma patients by gender and race" -> a 2x5 table and chart
"Save a type 2 diabetes cohort as a report"     -> 686        [Report]
"Which 5 conditions affect the most patients?"  -> a chart

A conversation, each question building on the last:
"How many patients have asthma?"                -> 841        [Query · 841]
"How many of them are women?"                   -> 445        [Query · 445]
"And of those, how many earn over $50,000?"     -> 370        [Query · 370]
"Save that as a report called 'Asthma women'"   ->            [Report]
"Rename it to 'Asthma, women, 50k+'"            ->            [Report]
"Delete that report"                            -> gone
```

Figures are from the 10,000-patient Synthea deployment and are checked against raw SQL by
`tests/ask_e2e.py`. Note the first two: the data spells values several ways ("Asthma",
"ASTHMA", "Asthm"; "F" and "f"), and "without hypertension" has a plausible wrong answer
(4,758) that a naive filter produces. Both are handled.

---

## Quickstart

Everything is bundled: Chiron itself, 300 synthetic patients, the MCP server, a chat
page, and the Chiron UI with an **Ask** tab. Nothing external is required beyond Docker,
Python 3.12+, Node 20+, and the [Claude Code CLI](https://claude.com/claude-code) logged in.

```bash
git clone https://github.com/ri-ds/chiron-mcp.git
cd chiron-mcp
./install.sh                                  # python venv + dependencies
docker compose up -d                          # Postgres warehouse
.venv/bin/python scripts/bootstrap_demo.py    # data dictionary + real ETL + demo users
./scripts/run_all.sh                          # starts all three servers
```

Then open **<http://localhost:5173>** and click **"Login as standard user with deidentified
data access"**. The demo login page is one click; there is no password to type.

The first `run_all.sh` installs the UI's npm dependencies, which takes a few minutes.
After that it starts in seconds.

### What you get

| URL | What |
|---|---|
| **http://localhost:5173** | **Chiron UI with the Ask tab**, start here |
| http://localhost:8001 | Chiron itself (server-rendered pages, admin) |
| http://localhost:8900 | The Ask chat page on its own |
| http://localhost:8765/mcp | The MCP tools over HTTP, for MCP clients on this machine |

In the UI, **Ask** sits in the header beside Aggregate. Type a question and you get an
answer with real figures, markdown tables, charts, and a button through to Chiron's
filter builder:

> What are the most common conditions?
> How many patients have asthma? *then* How many of them are women?
> Break down asthma patients by gender and race
> Show the top 10 medications as a chart
> Save an asthma cohort as a report *then* Rename it *then* Delete it
> Show asthma patients in my results *then* Add birthdate and city *then* Sort by birthdate, newest first

What the Ask tab does:

- **Counts and cohorts**, with spelling variants merged ("Asthma", "ASTHMA", "Asthm") and
  negation done right ("without hypertension" is 7,861, not the 4,758 a naive filter gives).
- **Follow-up questions.** The chat is a conversation: "them", "those patients", "now only
  women" refer to the previous answer's cohort. **New chat** starts another, and past
  conversations stay in a searchable sidebar. Under each answer, three suggested
  follow-ups; each new chat offers six fresh starters, one per feature.
- **Breakdowns** by one or two variables ("by gender", "by gender and race"): exact distinct
  patients per group and per cell, as a table and chart, with patients who have no recorded
  value called out so the table adds up.
- **Results tab**: put a cohort into Chiron's Results, add or remove columns, reorder
  them, sort (ask again to reverse), and change how each column is shown: stacked (one row
  per value), list, count, most frequent, has value ("whether they take albuterol", as
  true/false or a count), or sum, average, median, min, max for numbers and earliest or
  latest for dates. Each change is a new step in Chiron's undo history, and a **Results**
  button opens the updated table.
- **Reports**: save a cohort as a Chiron report, then rename it, change its filters or
  columns (the same column, sort and aggregation changes as Results), make it public or
  private, or delete it. Only the report's creator can change or delete it, and deletion
  needs its exact name.
- **Query · N** under any cohort answer: one click loads those filters into Chiron's query
  builder, and N is Chiron's own count for them.
- **Tables and charts** for rankings and comparisons. Two-variable breakdowns chart as
  grouped bars with a legend, so no bar is ever an unlabelled slice of the table.
- **A trace under every answer**, for finding out why one is wrong. **Trace** opens it.
  It shows a timeline of each step
  (start-up, every model call, every Chiron tool call, the server's checks), plus time,
  tokens and cost. Below that, at three levels of detail (Summary, Detailed, Debug), it
  shows each step: the model's summarised reasoning, the exact input and output of each
  tool call, where each figure in the answer came from, and the raw stream. Checks point
  at the step where something went wrong:
  - a figure that appears in no tool result, with the nearest count Chiron did return;
  - a cohort the model wrote by hand;
  - a field Chiron silently ignored;
  - a count rule Chiron does not read;
  - a spelling variant left out.
- **A call log** in the server's output: one line per model call and per Chiron call,
  with the user, dataset, model, tokens, cost, tool, status and time, so a conversation
  can be followed end to end with grep. See [The call log](#the-call-log).
- **Who is asking**: opened on its own, the Ask page's top bar shows the person's name
  and their access level on this dataset (full access, de-identified, aggregate only),
  resolved the way every tool resolves them, so the bar states what the answers obey.
  Inside the Chiron UI the bar stays empty: Chiron's own header and dataset page already
  show the name, the dataset and the level. `/whoami` returns the same as
  JSON and, for a staff account, the names of the request headers that reach the server
  and the values of those that may carry a sign-on identity, so a deployment behind a
  sign-on gateway can see what it is sent before trusting any of it.
- **A usage page** at `/usage`: how many answers, what they cost, how long they took and
  how many failed, for a period, per day, per person, per model and per dataset, with a
  CSV download. Everyone sees their own total at the foot of the Ask sidebar; a superuser
  gets the link, sees everyone, and gives and takes access on each dataset there. Fed by a
  ledger with one row per answer and no question text,
  kept for good. See [The usage page](#the-usage-page).

Questions take roughly 10 to 40 seconds, because the model makes several Chiron tool
calls. Each one shows as a chip while it works.

### Everything in Docker, at your own address

`deploy/` runs the same system as containers behind one HTTPS address and a password,
with Chiron's metadata in Postgres: `cd deploy && ./setup.sh --domain chiron.example.org`.
See [deploy/README.md](deploy/README.md), which also has a prompt to paste into Claude to
do the whole setup, and how to bring the 10,000-patient dataset along.

### Ask for a Chiron you already run

`askpatch/` adds Ask to an existing Chiron deployment without changing it: one container
built from that deployment's own Chiron, one route in its web server, one switch in its
UI, removable the same way. See [askpatch/README.md](askpatch/README.md).

### The demo data

Four datasets, the useful one being **300 synthetic patients** (Synthea) with conditions,
encounters, observations, medications and procedures:

| Dataset | Subjects | Content |
|---|---|---|
| `synthea-small` | 300 | Synthetic clinical records. The one to demo with |
| `dataset2_stored` | 6 | Chiron's own test fixture |
| `dataset1_stored` | 2 | Test fixture, the only one with event-date rules configured |
| `dataset3_stored` | 0 | Dictionary only, no rows |

Real figures out of the box: obesity 64, hypertension 57, depression 35, asthma 20.

Three demo accounts, each a button on the login page (password `demo1234` if you ever
need it): **`demouser`** (de-identified, what the chat is bound to), **`agguser`**
(aggregate only, refused row-level data), **`admin`** (PHI, staff).

### Pointing at a real deployment

Skip `bootstrap_demo.py` and set `CHIRON_MCP_METADATA_DB` and `CHIRON_MCP_WAREHOUSE_URL`
at your own Chiron. See [Configuration](#configuration).

## Table of contents

- [Quickstart](#quickstart)
- [What Chiron is](#what-chiron-is)
- [What this server does](#what-this-server-does)
- [Why it runs in-process](#why-it-runs-in-process-and-not-over-chirons-rest-api)
- [Install](#install)
- [Docker deployment](deploy/README.md)
- [Setup guide for an AI agent](#setup-guide-for-an-ai-agent)
- [Configuration](#configuration)
- [The tools](#the-tools)
- [How a query actually gets built](#how-a-query-actually-gets-built)
- [Access model and safety](#access-model-and-safety)
- [Known limits](#known-limits)
- [The call log](#the-call-log)
- [The usage page](#the-usage-page)
- [Troubleshooting](#troubleshooting)
- [Project layout](#project-layout)

---

## What Chiron is

Chiron (`is4r-chiron`) is a reusable Django application for exploring research and clinical
data, with a focus on human-subject and longitudinal datasets. Researchers use it to build
patient cohorts and pull reports without writing queries.

Its defining idea is that **the data dictionary is database rows, not a config file**:

| Layer | Holds | Reached via |
|---|---|---|
| **Metadata database** | The data dictionary (`Dataset` → `Collection` → `Concept`), plus users, access grants, saved reports and ETL logs | Django ORM |
| **Warehouse** | The actual subject data, one Postgres **schema per dataset** | SQLAlchemy only |

The warehouse's tables and columns are **generated at runtime from the dictionary rows**, not
migrated. The two databases cannot be joined, because there is no Django database router, so the ORM
describes the *shape* of the data while SQLAlchemy queries the data itself.

A researcher's work is expressed as two JSON documents:

- **`cohort_def`**: which subjects qualify
- **`table_def`**: which columns to show

Chiron compiles those into a single SELECT against the warehouse, injecting the user's
subject-visibility filter into the WHERE clause. A saved "report" is just a stored
`cohort_def` + `table_def` pair.

## What this server does

It exposes that machinery as **23 MCP tools** (19 for analysts, 4 for operators), so cohort
building can happen in conversation.

It is **stateless by design**: every tool takes a `cohort_def` and returns one, so the model
holds the working query and Chiron holds none of it.

It is **read-only by default**. Nothing here runs an ETL or alters a schema. Only with
`CHIRON_MCP_ALLOW_SAVE=1` can tools save or change the identity's own reports, or write its
query workspace (the Results tab), and workspace writes add a snapshot rather than erase
the undo history.

## Why it runs in-process, and not over Chiron's REST API

Three findings from reading the Chiron source, in order of how much they constrain the design:

**1. The query API cannot accept a cohort you give it.**
`QueryToolsViewSet._get_cohort_def` (`chiron/api/viewsets/query_tools.py:204-214`) discards any
supplied definition and returns the caller's *active server-side snapshot*, despite a docstring
promising otherwise. An HTTP wrapper would therefore have to overwrite a researcher's live
workspace on every turn, and `CohortDefSnapshot.save` deletes their redo stack
(`chiron/models/user_models.py:189-193`).

**2. Chiron has no token authentication.** No `rest_framework.authtoken`, no JWT, no OAuth, no
API key, no service-account concept, and no login *endpoint*. `rest_framework.urls` is commented
out. `REST_FRAMEWORK` is unset, so DRF's stock defaults (Session + Basic) apply by accident.

The result is an API split down the middle. HTTP Basic works on the metadata endpoints and is
rejected on **every endpoint that queries data**, because those viewsets set
`authentication_classes = [CsrfExemptSessionAuthentication]`, which *replaces* DRF's default list
rather than extending it:

| Basic auth works | Session cookie only |
|---|---|
| `/api/v2/auth/`, `/api/v2/dataset/` | `query_tools/` (count, preview, export) |
| `{ds}/concepts/`, `{ds}/collections/` | `cohort_def/`, `table_def/`, `analysis_def/` |
| `{ds}/concept_categories/` | `reports/`, `report_tools/`, `analysis_tools/` |

**3. The operator surface has no HTTP API at all.** ETL logs, dictionary validation and the
schema visualiser exist only as staff HTML pages and management commands.

So the server calls Chiron's own Python in the same interpreter, binding to one Django user and
resolving that user's real `ChironUser` per dataset. Chiron's permission logic does the filtering.

## Install

Requires Python 3.12+, a Chiron checkout, and network access to a Chiron metadata database and
warehouse.

```bash
git clone https://github.com/ri-ds/chiron-mcp.git
cd chiron-mcp
./install.sh
```

`install.sh` creates `.venv` and installs `mcp` plus Chiron's own base requirements. It finds
Chiron automatically when `is4r-chiron` sits beside this project or is pip-installed; otherwise
set `CHIRON_MCP_CHIRON_SRC`.

Then confirm the deployment resolves, before touching any MCP client:

```bash
CHIRON_MCP_USERNAME=<user> .venv/bin/python scripts/harness.py
```

It prints the Chiron source, both database locations, and every dataset that user can reach,
including the ones it refuses and why. **If this doesn't work, nothing else will.**

### Register with Claude

Copy the block from `claude_mcp_config.example.json` into your Claude config, filling in
absolute paths. On macOS that file is
`~/Library/Application Support/Claude/claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "chiron": {
      "command": "/abs/path/to/chiron-mcp/.venv/bin/chiron-mcp",
      "env": {
        "CHIRON_MCP_METADATA_DB": "/abs/path/to/chiron_metadata.sqlite3",
        "CHIRON_MCP_WAREHOUSE_URL": "postgresql://user:pass@localhost:5432/chiron",
        "CHIRON_MCP_USERNAME": "your-service-user",
        "CHIRON_MCP_MAX_ACCESS_LEVEL": "deid"
      }
    }
  }
}
```

Restart the client. The `chiron_*` tools appear (19, or 23 with `CHIRON_MCP_OPERATOR`).

### Grant the service user access

The server **never** creates access records. A Django user needs a `ChironUser` row per dataset:

```bash
.venv/bin/python scripts/grant_access.py --user <user> --datasets <dataset> --level deid
.venv/bin/python scripts/grant_access.py --list
.venv/bin/python scripts/grant_access.py --user <user> --datasets <dataset> --revoke
```

## Setup guide for an AI agent

Written for an AI assistant setting this up on someone's machine. Every step has a check.
Do not skip a check, and do not continue past a failed one.

> **Chiron and its UI are already vendored** at `vendor/is4r-chiron` and
> `vendor/is4r-chiron-ui`. Do not clone or `pip install` them. The `chiron` package on
> PyPI is an unrelated DNA basecaller, and the real repository is behind organisation SSO.

**Do this.** It is four commands, needs nothing external, and ships 300 synthetic
patients. Do not ask the user which setup they want, and do not go looking for an existing
Chiron deployment: everything is bundled.

The only exception is at the very bottom of this section, and it applies only when the
user has already told you, unprompted, that they run their own Chiron and want to query
their real data.

---

### Setup

```bash
./install.sh
docker compose up -d
.venv/bin/python scripts/bootstrap_demo.py
./scripts/run_all.sh
```

**Check:** `bootstrap_demo.py` ends with `Demo ready. 4 dataset(s) queryable.` and lists
`synthea-small: 300 subjects`. Then `run_all.sh` prints three URLs and stays running.

**Check:** `curl -s -o /dev/null -w '%{http_code}' http://localhost:5173/` returns 200.

Tell the user to open <http://localhost:5173>, click the **"deidentified data access"**
login button, and then **Ask**. **You are done.**

Requirements, which you should confirm first: Python 3.12+, Node 20+, Docker running, and
the Claude Code CLI installed and logged in (`claude --version`). The first `run_all.sh`
spends a few minutes on `npm install`.

---

### Only if the user already runs their own Chiron

Skip this unless the user has said so themselves. The bundled demo above is the right
answer for everyone else, and 300 patients is plenty to work with.

Same repo, but skip `bootstrap_demo.py` and aim the two database variables at their
deployment. Do **not** run `bootstrap_demo.py` in this case: it builds a separate demo
database and never touches theirs, but running it wastes several minutes.

#### 1. Gather four things from the user

1. The **metadata database**: usually a SQLite file holding datasets, users and access
   grants. In a Docker deployment it is often a bind mount; look in the compose file for a
   path mapped to `db.sqlite3`.
2. The **warehouse connection string**: `postgresql://user:pass@host:port/dbname`. Also in
   the compose file, as `CHIRON_SQL_ALCHEMY_CONNECTION_STRING`. If Chiron runs in Docker it
   may say `host.docker.internal`; from outside the container that is `localhost`.
3. The **Django username** the server should act as.
4. The **URL of their Chiron UI**, for hand-off links.

If you cannot get 1 and 2, stop and ask. Do not guess a path or a password.

#### 2. Verify before configuring anything

```bash
CHIRON_MCP_USERNAME=<user> \
CHIRON_MCP_METADATA_DB=/path/to/their_metadata.sqlite3 \
CHIRON_MCP_WAREHOUSE_URL=postgresql://user:pass@localhost:5432/chiron \
.venv/bin/python scripts/harness.py
```

**Check:** it ends with `N dataset(s) reachable by this identity`, N at least 1.

Lines marked `[refused]` are not failures; they mean that user has no access record for
that dataset, which is correct behaviour. If **every** dataset is refused, see step 3.

If this fails, nothing else will work. Fix it here.

#### 3. Grant access only if asked

```bash
.venv/bin/python scripts/grant_access.py --list
.venv/bin/python scripts/grant_access.py --user <user> --datasets <dataset> --level deid
```

This writes to **their** metadata database. Tell the user which account and which dataset
before doing it, and get their agreement. Never pick a superuser: the server refuses to
start as one in analyst mode, and that refusal is deliberate.

#### 4. Run against their deployment

```bash
export CHIRON_MCP_METADATA_DB=/path/to/their_metadata.sqlite3
export CHIRON_MCP_WAREHOUSE_URL=postgresql://user:pass@localhost:5432/chiron
export CHIRON_MCP_USERNAME=<user>
export CHIRON_MCP_UI_URL=http://their-chiron-ui        # for hand-off links
export CHIRON_MCP_ALLOW_SAVE=1                         # enables "open this in Chiron"

.venv/bin/python -m chiron_mcp.webapp                   # Ask chat at :8900
```

`run_all.sh` honours the same variables, but here do not start `serve_chiron.py`:
they already have a Chiron running, and a second one on `:8001` would confuse everyone.
Either run only the Ask server as above, or add the **Ask** tab to their own UI following
[docs/chiron-ui-integration.md](docs/chiron-ui-integration.md).

**Check:** ask the chat "how many patients are in <their dataset>?" and confirm with the
user that the number matches what Chiron shows.

#### 5. Also register with Claude Desktop, if they want it there too

See [Register with Claude](#register-with-claude). Use the same variables.

---

### If something goes wrong

| Symptom | What it means | Fix |
|---|---|---|
| `Could not find the Chiron source tree` | Discovery failed | Set `CHIRON_MCP_CHIRON_SRC` to the directory containing `chiron/` |
| `No Django project settings at ...` | Wrong host project | Set `CHIRON_MCP_PROJECT_DIR` to the directory with `manage.py`, and `CHIRON_MCP_BASE_SETTINGS` if its settings module is not `project.settings` |
| `has no ChironUser on dataset` | Working as designed | Grant it with `scripts/grant_access.py`, with the user's agreement |
| `is a superuser. Refusing to start` | Working as designed | Use a dedicated service account |
| `cannot see subject-level data` | The account is `agg` | Correct behaviour. Use `chiron_crosstab` or `chiron_concept_values` |
| `npm install` fails on peer deps | The UI pins eslint 9, a plugin wants 8 | `run_all.sh` already passes `--legacy-peer-deps`; use that flag if installing by hand |
| `You must set settings.ALLOWED_HOSTS` | Running Django with `DEBUG=False` | Already handled in `chiron_mcp/django_settings.py`; make sure you launch through `scripts/serve_chiron.py` |
| Tools missing after restart | Config not loaded | Check the JSON parses, paths are absolute, and you edited the right file |
| `Server disconnected` in the client | The client did not honour `cwd` | Set `command` to `.venv/bin/chiron-mcp` and drop `args` and `cwd` entirely |
| `pip install chiron` "worked" but nothing imports | Wrong package | Chiron is vendored here and is not on PyPI |

### What to tell the user when you are done

Which Django user the server is bound to, its access ceiling, and exactly which datasets
are reachable. If you granted any access during setup, say so
explicitly including the level: that is a change to their deployment, not just to this tool.

## Configuration

Every setting is an environment variable. Defaults are the conservative end of each choice.

| Variable | Default | Meaning |
|---|---|---|
| `CHIRON_MCP_USERNAME` | **required** | Django user the server acts as |
| `CHIRON_MCP_MAX_ACCESS_LEVEL` | `deid` | Ceiling. Clamps whatever the `ChironUser` actually has, so PHI is opt-in |
| `CHIRON_MCP_DATASETS` | *(all with a grant)* | Comma-separated allowlist |
| `CHIRON_MCP_OPERATOR` | off | Enables the operations tools (also requires `is_staff`) |
| `CHIRON_MCP_ALLOW_SAVE` | off | Lets `chiron_open_in_ui` save reports and load cohorts into Chiron, which powers the Query and Report buttons |
| `CHIRON_MCP_MAX_EXPORT_ROWS` | `50000` | Cap on `chiron_export_table` |
| `CHIRON_MCP_CHIRON_SRC` | auto-discovered | The `is4r-chiron` checkout |
| `CHIRON_MCP_PROJECT_DIR` | `<src>/test_project` | The Django host project |
| `CHIRON_MCP_BASE_SETTINGS` | `project.settings` | Host project's settings module |
| `CHIRON_MCP_METADATA_DB` | *(host project's)* | Metadata database: a SQLite file path, or a `postgresql://` URL |
| `CHIRON_MCP_WAREHOUSE_URL` | *(host project's)* | SQLAlchemy warehouse URL |
| `CHIRON_MCP_UI_URL` | `http://localhost:3000` | Where Query and Report links point |
| `CHIRON_MCP_TRANSPORT` | `stdio` | `stdio`, or `http` for a streamable-HTTP endpoint at `/mcp`. See [HTTP endpoint](#http-endpoint) |
| `CHIRON_MCP_HTTP_PORT` | `8765` | Port for the HTTP endpoint. The address is always `127.0.0.1` |
| `CHIRON_MCP_HTTP_AUTH` | `none` | `oauth` makes callers of the HTTP endpoint sign in as a Chiron user; each call then acts as that user. `CHIRON_MCP_USERNAME` is not needed in that mode |
| `CHIRON_MCP_OAUTH_SECRET` | *(generated)* | Key that signs OAuth tokens. Unset, a random one is kept in `CHIRON_MCP_STATE_DIR` (default `~/.cache/chiron-mcp/oauth.key`) |

### HTTP endpoint

By default an MCP client launches `chiron-mcp` itself and talks to it over stdio. Set
`CHIRON_MCP_TRANSPORT=http` to run it as a long-lived server instead, with the same tools
at `http://127.0.0.1:8765/mcp` (streamable HTTP):

```bash
CHIRON_MCP_TRANSPORT=http CHIRON_MCP_USERNAME=<user> .venv/bin/chiron-mcp
```

`scripts/run_all.sh` starts it alongside the other servers. Point a client at it by URL:

```bash
claude mcp add --transport http chiron http://localhost:8765/mcp
```

```json
{ "mcpServers": { "chiron": { "type": "http", "url": "http://localhost:8765/mcp" } } }
```

It is bound to `127.0.0.1` and there is no setting to change that. Requests whose
`Host` header is not local are refused, which stops a web page from reaching it through
DNS rebinding.

**By default it has no authentication**: every caller acts as `CHIRON_MCP_USERNAME`, with
whatever that user can see and, if `CHIRON_MCP_ALLOW_SAVE` is on, change. Check it with:

```bash
CHIRON_MCP_USERNAME=<user> .venv/bin/python -m tests.protocol_http
```

#### Signing in as a Chiron user (OAuth)

Set `CHIRON_MCP_HTTP_AUTH=oauth` and the endpoint stops being one shared identity. A
client must sign in, and every tool call then runs as the person who did, with their own
access level and permission groups on each dataset. `scripts/run_all.sh` turns this on.

```bash
CHIRON_MCP_TRANSPORT=http CHIRON_MCP_HTTP_AUTH=oauth .venv/bin/chiron-mcp
```

The server is its own small OAuth 2.1 authorization server (dynamic client registration,
authorization code with PKCE, refresh tokens), which MCP clients already speak. Nothing
changes in how you add it; the client notices the `401` and opens a browser. In Claude
Code that is `/mcp`, then the server, then **Authenticate**.

The browser lands on a sign-in page served by this server, which turns an identity into a
Chiron user in one of two ways:

- **Continue as *name***, if that browser is already logged in to Chiron. The page reads
  Chiron's own session cookie, so whatever Chiron uses to log people in (including SSO)
  is what authenticated them, and no password is typed here.
- **A Chiron username and password**, checked by Django's `authenticate()`, so the
  deployment's own authentication backends decide.

Either way the result is an existing, active Django user, and that username is the
token's subject. From there the rules are the ones in
[Access model and safety](#access-model-and-safety), per person:

- The user's own `ChironUser` on each dataset decides what they see. Nobody is
  provisioned: someone with no access record on a dataset is refused, as on stdio.
- `CHIRON_MCP_MAX_ACCESS_LEVEL` still clamps everyone, so PHI stays opt-in.
- An aggregate-only user is refused row-level tools; the operations tools need
  `CHIRON_MCP_OPERATOR` and a staff account; saves land in the signed-in user's own
  workspace and reports.
- A superuser may sign in as themselves, as in the Ask tab: the superuser refusal is
  about a *service* being configured as one.

What it keeps, and for how long:

- **One signing key and nothing else.** Tokens and client registrations are signed
  blobs the server verifies rather than stores, so they survive a restart. The key is
  not Django's `SECRET_KEY` (the bundled demo project ships a public one). Delete the key
  file to sign everyone out.
- Access tokens last an hour and refresh tokens thirty days. A token stops working at
  once if its user is deactivated or changes password.
- Signed tokens cannot be revoked one at a time, so there is no revocation endpoint.
- Clients may register loopback redirect URIs only, matching the loopback-only endpoint.

Check the whole flow, as two users with different access:

```bash
.venv/bin/python -m tests.protocol_oauth
```

The Ask server (`python -m chiron_mcp.webapp`) adds:

| Variable | Default | Meaning |
|---|---|---|
| `CHIRON_MCP_WEB_PORT` | `8900` | Port for the chat page |
| `CHIRON_MCP_ALLOW_SUPERUSER` | off | Lets a *logged-in* superuser use Ask as themselves. The fallback identity can never be a superuser |
| `CHIRON_MCP_THREAD_TTL_HOURS` | `12` | How long a conversation can be continued. After that its transcript is deleted |
| `CHIRON_MCP_TURN_SECONDS` | `240` | How long one answer may take before it is stopped and the page says to ask again |
| `CHIRON_MCP_STATE_DIR` | `~/.cache/chiron-ask` | Conversation index, answer traces (`traces/`) and the neutral working directory Claude runs in |
| `CHIRON_MCP_TRACE` | on | `0` stops recording answer traces (the **Trace** under each answer), and with them the call log |
| `CHIRON_MCP_CALL_LOG` | `stderr` | Where the call log goes (one line per model call and per Chiron call, see [The call log](#the-call-log)): the server's own output, a file path to append to, or `off` |
| `CHIRON_MCP_USAGE_DB` | `<state dir>/usage.sqlite` | The usage ledger, one row per answer, behind [the usage page](#the-usage-page): a file path, or `off` |
| `CHIRON_MCP_USAGE_RETENTION_DAYS` | *(kept)* | Ledger rows older than this many days are deleted when the server starts |
| `CHIRON_MCP_USAGE_GROUP` | *(none)* | A Django group whose members see everyone's usage and who has what access, as superusers do |
| `CHIRON_MCP_THINKING_DISPLAY` | `summarized` | Passed to Claude Code as `--thinking-display`, so traces show the model's reasoning. Empty leaves the flag off. Code that imports `ask_stream` rather than running the server leaves it off unless set |

See `.env.example`. A superuser is refused in analyst mode; point it at a dedicated service user.

## The tools

### Orient

| Tool | Purpose |
|---|---|
| `chiron_datasets` | Datasets this identity can reach, its access level on each, and why anything is refused |
| `chiron_find_concepts` | Search a dataset's variables by name or description |
| `chiron_describe_concept` | One variable in full, **including the exact field names required to filter on it** |
| `chiron_concept_values` | Distinct values with subject counts |

### Build a cohort

| Tool | Purpose |
|---|---|
| `chiron_edit_cohort` | Apply one transformation; returns the new `cohort_def` plus a plain-English description |
| `chiron_event_rule_options` | Which date/age restrictions apply to a criteria set |
| `chiron_count_cohort` | How many subjects match |

### Hand off to the Chiron UI

| Tool | Purpose |
|---|---|
| `chiron_open_in_ui` | Turn a cohort built in conversation into real Chiron filters and return a link. Requires `CHIRON_MCP_ALLOW_SAVE=1` |

### Get results

| Tool | Purpose |
|---|---|
| `chiron_run_table` | One page of rows; columns are a plain list, no `table_def` authoring |
| `chiron_export_table` | Complete result set, capped, with an acknowledged-count interlock |
| `chiron_breakdown` | Distinct patients per group of one or two Category variables, variants merged, as structured rows and cells. Small counts masked for `agg` |
| `chiron_crosstab` | Chiron's own analysis-view pivot (preformatted text); `chiron_breakdown` is usually better |
| `chiron_saved_reports` | Browse saved reports (`scope="mine"` for your own), or open one's stored definitions |
| `chiron_run_saved_report` | Run a saved report by id |
| `chiron_column_options` | How one variable can be shown as a column: its default and every aggregation, with the settings each takes |

### The Results tab

Both write tools require `CHIRON_MCP_ALLOW_SAVE=1`.

| Tool | Purpose |
|---|---|
| `chiron_results` | What the identity's Results tab holds now: filters, columns with their aggregation, sort, and a preview |
| `chiron_update_results` | Put a cohort into Results and add, remove, reorder, sort or re-aggregate columns. Previews before saving, and saves a new snapshot so Chiron's undo still works |

### Change saved reports

Both require `CHIRON_MCP_ALLOW_SAVE=1`, and both act only on reports the identity created.

| Tool | Purpose |
|---|---|
| `chiron_update_report` | Rename, re-describe, change filters, or make public/private; add, remove, reorder, sort or re-aggregate columns. Validates and previews before saving |
| `chiron_delete_report` | Permanently delete. `confirm_name` must equal the report's exact current name |

### Operations

Gated behind `CHIRON_MCP_OPERATOR` **and** a staff account.

| Tool | Purpose |
|---|---|
| `chiron_validate_dictionary` | Run Chiron's own validator. Non-empty means the next ETL will refuse to run |
| `chiron_schema_diagram` | The dataset's shape as a Mermaid ER diagram |
| `chiron_etl_history` | The ETL audit trail, which has no HTTP API |
| `chiron_explain_access` | Why this identity can or cannot use a variable, Chiron's verdicts verbatim |

## How a query actually gets built

The non-obvious part, and the reason `chiron_describe_concept` exists:

**Chiron publishes no machine-readable schema for cohort filter inputs.** A filter is submitted
as HTML *form field names*, and which names apply depends on the variable's processor. The only
source of truth is each processor's `validate_form()` body in `chiron/processors/cohort_def/`.
Those were read directly and transcribed into `chiron_mcp/filters.py`:

| Processor | Required input | Also accepts |
|---|---|---|
| `CohortDefCategory` | `selected_categories` (list) | `exclude_selected` |
| `CohortDefBoolean` | `selected_categories` (list) | |
| `CohortDefNumber` | `cd_numeric_min` / `cd_numeric_max` | `exclude_selected`, `include_null_and_missing` |
| `CohortDefNumberWithCategories` | `cd_numeric_min` / `cd_numeric_max` | `selected_categories`, and the above |
| `CohortDefDate` / `CohortDefDateDeid` | `query_type` | `cd_numeric_min`, `cd_numeric_max`, `days_ago` |
| `CohortDefDetailedAge` | `cd_age_min_year` / `cd_age_min_day` / `cd_age_max_year` / `cd_age_max_day` | |
| `CohortDefText` | `chiron_text_field_selection` (newline-separated) | `ignore_warnings` |
| `CohortDefOntology` | `chiron_ontology_field_selection` | `include_ontology_unknown`, `ignore_warnings` |

So the working sequence is always: **describe the concept, then filter it.**

```python
chiron_describe_concept(ds, "condition__description")   # -> CohortDefCategory
chiron_edit_cohort(ds, [], {
    "type": "add_entry",
    "concept_id": "condition__description",
    "selected_categories": ["Asthma"],
})                                                       # -> cohort_def
chiron_count_cohort(ds, cohort_def)                      # -> 838
```

Date and age restrictions ("diagnosed between 2015 and 2020") are a **second** step on a
criteria set, and take two transformations:

```python
chiron_event_rule_options(ds, cohort_def, entry_id)      # -> which options apply
chiron_edit_cohort(ds, cohort_def, {"type": "create_event_rule", "entry_id": eid})
chiron_edit_cohort(ds, cohort_def, {
    "type": "modify_event_rule", "entry_id": eid,
    "option_type": "date_range", "date_min": "2015", "date_max": "2020",
})
```

Note the key is `option_type`, not `option_id`. `chiron_event_rule_options` asks Chiron directly
rather than relying on a transcription, so it cannot drift from the code.

## Handing a cohort to the Chiron UI

The point of building a cohort in conversation is usually to keep working on it somewhere else.
`chiron_open_in_ui` writes the cohort into Chiron proper and returns a link.

It has two modes, because there are two genuinely different things a user means by "open it in
Chiron".

**`mode="report"` (default, non-destructive).** Saves the cohort and columns as a private Chiron
report and returns `/<dataset>/reports/<id>`. Nothing the researcher currently has open is
touched. They can then use Chiron's own "load as active" to pull it into the builder when ready.

**`mode="workspace"` (destructive).** Writes the cohort and table straight into the user's live
query workspace and returns `/<dataset>/query`, so the link opens the filter builder with the
filters already applied. This is the mode that makes a conversation become a UI page.

Be deliberate about the second one. It mirrors what Chiron's own `load_as_active` does
(`chiron/api_v2/viewsets/report_tools.py:108-133`): it calls `clear_history` on both snapshot
models, so the researcher's current cohort, current table **and their entire undo history on
that dataset** are replaced. Ask before using it.

### Why a link is enough

The React UI's query route fetches `GET /api/v2/<dataset>/cohort_def/`
(`src/store/cohortSlice.ts:319`), and that endpoint returns
`CohortDefSnapshot.get_active_cohort_def(request.chironuser)`. So writing an active snapshot for
that ChironUser *is* the hand-off; no URL parameters or deep-link encoding are involved, and the
UI needs no changes.

One consequence worth understanding: the hand-off lands in the workspace of the Django user the
server is bound to (`CHIRON_MCP_USERNAME`). If several people share one service account, they
share one workspace and will overwrite each other. Give each person their own account, or use
`mode="report"`, which has no such problem.

Set `CHIRON_MCP_UI_URL` to point the links at your deployment (default `http://localhost:3000`).

### Columns, aggregation and the Results tab

Chiron groups a table's rows by its **stacked** columns and combines every other column
within each group. With only gender stacked there is one row per gender; stack patient-level
columns such as birthdate and city to get about one row per patient. Each column is
`{"concept_id", "aggregation", "settings", "alias"}`, where `aggregation` is `"stack"` or one
of the variable's methods from `chiron_column_options` (Chiron's own default is stack on the
patient record and `list_distinct` elsewhere). Settings are checked against the method's
inputs, so `has_value` takes `{"values": [...], "return_value": "boolean" | "count"}`.

`chiron_update_results` and `chiron_update_report` take the same edits, applied through
Chiron's own table transformations in this order: `remove_columns`, `add_columns`,
`set_aggregation` (`[{column, aggregation, settings}]`), `order` (these columns first, the
rest after them), `sort` (`[{column, direction: "asc" | "desc"}]`, replacing the sort;
`[]` clears it). A column can be named by its entry id,
its concept id, or the label `chiron_results` gives ("medication description"); an ambiguous
or unknown name is refused with the columns that exist. Both preview the result before
saving, so a change Chiron cannot run is never stored.

Unlike `chiron_open_in_ui(mode="workspace")`, `chiron_update_results` saves new active
snapshots instead of clearing history, so Chiron's undo goes back to the previous table.

## Access model and safety

Access in Chiron is not a login check. It is a `ChironUser` record per (Django user, dataset),
carrying one of three levels plus `PermissionGroup` memberships.

| Level | Subject rows | Counts / crosstabs |
|---|---|---|
| `phi` | yes, identifiable | yes |
| `deid` | yes, with PHI variables automatically swapped for de-identified forms | yes |
| `agg` | **refused** | yes |

### Gates re-implemented here

Calling Chiron's Python directly skips three permission classes that exist **only** in the HTTP
layer. They are re-implemented in `chiron_mcp/identity.py`, and every tool calls them first:

| Gate | Mirrors |
|---|---|
| `require_subject_level` | `SubjectLevelAccess` (`chiron/api/permissions.py:64-79`) |
| `require_workspace` | `CanViewWorkspacePermission` (`chiron/api_v2/permissions.py:37-45`) |
| `require_analysis_view` | `AnalysisViewOn` (`chiron/api/permissions.py:55-62`) |

`Concept.user_can_view_concept_stats` is **not** a substitute for the first: it explicitly
*allows* agg users for non-PHI concepts (`data_definition_models.py:1114-1120`). The agg block is
the DRF class, and only the DRF class.

### Five hazards handled explicitly

1. **No silent provisioning.** `get_request_chironuser` (`chiron/authorization.py:61-72`) writes a
   `ChironUser` as a side effect of a *read*, granting at `Dataset.auto_access_level`, which is
   `phi` on some datasets. This server refuses instead and says what to create deliberately.
2. **Never a `None` chironuser.** `CohortDefProcessor.__init__:42` substitutes `SystemChironUser`
   (hardcoded PHI, every permission group, all datasets) when passed `None`.
   `identity.checked_chironuser()` makes that a hard error rather than a silent escalation.
3. **An errored cohort is never executed.** `clean_cohort_def` reduces an errored definition to an
   empty list, and an empty `cohort_def` matches **every subject**. Running one would silently
   return the whole dataset, so the server refuses.
4. **A failed filter returns no reusable cohort.** On failure `chiron_edit_cohort` omits the
   `cohort_def` key entirely and returns `unchanged_cohort_def` instead, so a failed transformation
   cannot be chained into an accidental whole-dataset query.
5. **A name is withheld on every path, not only as a column.** A synthetic dataset's dictionary
   marks no `has_phi`, so Chiron's own pages let a de-identified account without subject details
   add a first or last name column or filter on one. Here such a concept is refused wherever it
   could show or single out a person: as a column (listed, exported, in Results, in a report,
   added later), as the values of a field, in a description with statistics, as a breakdown or
   crosstab dimension, and as a filter (how many people named X have asthma is a question about
   a person), whether the filter arrives through `chiron_edit_cohort`, in a hand-written
   `cohort_def` or inside a saved report. A report or Results tab that already holds such a
   column runs with it withheld and says so. `tests/permissions_adversarial.py` tries each of
   these as that account.

### Verifying

```bash
CHIRON_MCP_USERNAME=<deid-user> .venv/bin/python tests/safety.py   # invariants
CHIRON_MCP_USERNAME=<agg-user>  .venv/bin/python tests/safety.py   # agg refusals
.venv/bin/python -m tests.protocol_http                            # HTTP endpoint, loopback only
.venv/bin/python -m tests.protocol_oauth                           # HTTP endpoint, OAuth login per user
.venv/bin/python tests/ask_e2e.py                                  # the Ask tab, end to end
.venv/bin/python tests/permissions.py                              # what each kind of account cannot get
.venv/bin/python tests/permissions_adversarial.py                  # every way past one's level: tools, questions, the server
.venv/bin/python -m tests.protocol                                 # real MCP stdio handshake
.venv/bin/python tests/usage.py; .venv/bin/python tests/header_login.py   # offline: the ledger, the header sign-on
```

`tests/ask_e2e.py` asks real questions through the Ask server exactly as the page does, and
checks everything a user would click against ground truth computed independently in raw SQL:
the stated figure, that the Query button loaded into Chiron makes Chiron's own count agree, that
a report opens with columns and the same subjects, that refusals leak nothing, and that no answer
contains an email address or file path. It is written against the 10,000-patient Synthea dataset.

The page's starter questions come from a pool of about 130 built from the dataset's own values
and tagged by feature; `tests/ask_e2e.py --suggestions` asks every one of them and fails any that
errors, is refused, shows no figure, or lacks the button its feature promises, so a starter cannot
ship broken.

`tests/permissions.py` is the permission suite: one question set (a count, a list with names,
a breakdown, the Results tab, a report with names, another dataset) asked through `/ask` as six
identities it makes for the run, a de-identified account with and without subject details, an
aggregate-only one, a full-access row under the ceiling, a login with no row on the dataset,
and `admin`. From each answer, its events and its trace (every tool result) it asserts what that
account must not get: no button, no patient-level tool and no unmasked small count for
aggregate; no identifying column in any tool result, the Results tab or a saved report for
de-identified without details; nothing at all for the login without a row; and for everyone
that another person's conversation cannot be continued, their trace cannot be read, request
headers show to staff only and the usage page shows one's own rows. It warns when a dataset's
auto-create rule would hand a newcomer more than aggregate access, and fails on it once header
sign-on is on.

It also runs whole conversations: three follow-ups that each narrow the last cohort (841, 445,
370, each checked in Chiron); "now show their medications", which must give the asthma
patients' figures (albuterol 587) and not the whole dataset's (696); a report saved, renamed,
refiltered and deleted, with Chiron's API checked after every step; a refused attempt to delete
someone else's report; and another user presenting someone's conversation id, who must get a
fresh conversation. Breakdowns are checked cell by cell, and a chart that shows one row of a
two-way table as if it were the whole fails. Reports the run creates are deleted at the end,
and so are its conversations and their transcripts.

Point it at a Chiron that shares file locking with the Ask server (`CHIRON_API`, default
`http://localhost:8000`). With the bundled demo that is `http://localhost:8001`.

`tests/permissions_adversarial.py` attacks instead of asking. As a de-identified account without
subject details, an aggregate-only one and a login with no row (plus a control that may see
names), it tries every way past the level: every tool called in-process with the arguments a
model could be talked into (the values of a name field, a breakdown or crosstab by last name, the
statistics of a name, a filter on a name, a hand-written `cohort_def` with one, a name column
added to Results or a report afterwards, a shared report that holds names, an export, another
dataset); adversarial questions through `/ask` (claimed authority, injected tool calls,
obfuscation in uppercase or base64, membership questions with real names, other datasets); and
the web server itself (no session, a forged one, forged identity headers, cross-site origins on
reads and on writes, another person's trace, conversation and usage rows, the CSV scope, path
tricks, the MCP route). It does not trust the answer: every real first and last name in the
dataset is read from the warehouse and searched for in the answer and in every tool result the
trace holds, and the control account proves the detector sees names where they are allowed. A
policy layer reports what the rules permit by design (birthdate, zip and gender per patient at
the de-identified level; what a masked breakdown still lets one derive; what Chiron's own pages
would show the same account) as observations. `--no-model` runs the tool and server layers
alone, in under a minute; `--layer` picks layers.

## Known limits

- **Crosstab needs a configured root collection.** `run_analysis` reads
  `dataset.root_collection.event_id_field` and dereferences it without a null check. Datasets that
  leave it unset get a clear refusal instead of an `AttributeError`.
- **Most collections are not event collections.** Where `event_date_field` is unset, event rules do
  not apply at all.
- **Regex text filters are broken upstream.** Chiron's own tests document `/slash-wrapped/` terms
  as returning wrong answers.
- **Terms absent from the data are rejected** with `Entry X not found`. Pass `ignore_warnings` to
  accept the filter anyway.
- **Most variables are multi-value**, living in their own lookup table joined one-to-many, so a
  filter matches a subject if *any* of their values match. That makes `exclude_selected` mean
  "has at least one record that is something else", not "has none": on the demo data it turns
  "patients without hypertension" into 4,758 instead of the right 7,861. The Ask server's prompt
  directs the model to Chiron's criteria-set count rule ("exactly 0") instead.
- **`chiron_crosstab` returns preformatted text**, not structured rows. That is what Chiron's
  analysis engine hands back, and it needs a root-collection setting the Synthea data lacks.
  `chiron_breakdown` does the same job with Chiron's count engine and returns structured rows.
- **Breakdowns group Category and Text variables only.** Numbers and dates have too many values
  to group; the model filters on ranges and counts each one instead. At most two variables, and
  the largest groups first (10 for one variable, 6 per axis for two, or `top`).
- **A report lists distinct rows, not patients, unless the dataset has a patient-ID
  variable.** Reports default to the variables the cohort filters on; with no ID column,
  Chiron collapses identical rows, so an asthma-in-women report on synthea-10k shows two rows
  for 445 subjects. The subject count is right; add an ID variable to the data dictionary to
  get one row per patient.
- **For `agg` accounts, masked cells can sometimes be inferred** by subtracting from a total.
  That is equally true of Chiron's own analysis view, whose rules the breakdown follows.
- **A SQLite metadata database must not be shared across the Docker boundary.** If Chiron runs
  in a container with its metadata SQLite bind-mounted from the host, and this server runs on
  the host against the same file, the two sides cannot see each other's file locks (Docker
  Desktop does not carry them across its VM). Concurrent writes then produce "database disk
  image is malformed" errors, and at worst real corruption. Run this server where Chiron runs:
  both on the host, as the bundled demo does, or both in containers on the same mount. Keep a
  backup (`sqlite3 db ".backup copy"`) either way.
- **Ask conversations are stored as Claude Code transcripts**, which contain patient data.
  They are deleted when you delete the conversation or after `CHIRON_MCP_THREAD_TTL_HOURS`; see
  `docs/chiron-ui-integration.md` for the details and a cleanup command for older ones.
- **Remote deployment is not supported.** Execution is in-process, so the server must run
  where it can reach both databases, and the [HTTP endpoint](#http-endpoint) listens on
  the loopback interface only. Its OAuth login identifies each caller as a Chiron user,
  which is the authentication a remote endpoint would need, but serving it beyond this
  machine also needs TLS and non-loopback redirect URIs, and neither is built.

## A Chiron bug this project found

`CohortDefProcessor._generate_cd_entry_template` (`chiron/processors/abstract/cohort_def_processor.py`)
has a mutable default argument, `additional_args={}`, and returns it after writing into it. Every
filter entry built without arguments in the same process is therefore **the same dict**, so
building a second filter silently rewrites the first: its values, its exclude flag and its
`entry_id`. In Chiron's web app each request serialises its entries quickly, which mostly hides
it. Anything that builds several filters in one process does not get that protection.

The bundled copy under `vendor/` is patched (`additional_args=None`, a fresh dict per call), and
`chiron_edit_cohort` also deep-copies its result, so this server is correct against an unpatched
Chiron too. The one-line fix belongs upstream.

A second patch, also for upstream: Chiron's relative-date rule between two event collections
("within 90 days after") read each event date straight off the collection table
(`LongitudinalCombo.generate_sql_alchemy_clauses_for_event`), which is where a single-value
concept lives. A dataset loaded with every concept multivalue, as the Synthea bundles are,
keeps its dates in lookup tables, so the rule failed with `AttributeError: <date concept>`
for Chiron's own query builder as much as for chiron_query's `within`. The where builder
now resolves an event date through the subquery's lookup alias (joined like the filters'
lookups) or, for a table without a subquery of its own, a correlated lookup
(`WhereBuilder.get_event_date_column`).

Against a Chiron that has neither fix (any deployment's own checkout, through `askpatch/`
or `CHIRON_MCP_CHIRON_SRC`), `chiron_mcp/chiron_patches.py` applies both in this server's
process right after Django starts, and says so on stderr. Chiron's files are not touched,
and Chiron's own web app, in its own process, keeps its behaviour until upstream merges
them. `askpatch/chiron-fixes.patch` is the same change as a patch for a Chiron checkout.

## The call log

The Ask server writes one line to its own output (stderr, which Docker collects) for
every call an answer makes, so a conversation, a user or a day can be followed through
the log with grep: the trace under an answer holds the detail for one answer, and this
is the operational copy for all of them.

```
2026-11-03T15:40:12.345Z ask request answer=2f1a… thread=6c3e… user=demouser dataset=synthea-10k turn=1 followup=0
2026-11-03T15:40:14.010Z ask model answer=2f1a… thread=6c3e… user=demouser dataset=synthea-10k call=1 model=us.anthropic.claude-haiku-5-5 in=4210 cache_read=3900 cache_write=0 out=88 stop=tool_use tools=query ms=1830
2026-11-03T15:40:14.420Z ask chiron answer=2f1a… thread=6c3e… user=demouser dataset=synthea-10k call=1 tool=query status=ok count=179 group=g1 rows=0 size=612 ms=410
2026-11-03T15:40:17.902Z ask model answer=2f1a… thread=6c3e… user=demouser dataset=synthea-10k call=2 model=us.anthropic.claude-haiku-5-5 in=8390 cache_read=7800 cache_write=0 out=152 stop=end_turn ms=3480
2026-11-03T15:40:20.114Z ask answer answer=2f1a… thread=6c3e… user=demouser dataset=synthea-10k turn=1 followup=0 status=ok model=us.anthropic.claude-haiku-5-5 model_calls=2 chiron_calls=1 chiron_errors=0 retries=0 in=12600 cache_read=11700 cache_write=0 out=240 cost_usd=0.00421 cost_total=0.00421 ms=7770 model_ms=5310 chiron_ms=410 rc=0
```

Every line is a UTC timestamp, the word `ask`, the kind, then `key=value` pairs. The kinds
are `request` (the question arrived), `model` (one model call: the tokens it used, why it
stopped, the tools it chose), `chiron` (one Chiron tool call: how it ended, the count or
the number of patient rows it returned, the size of the result), `retry` (the API was
busy), `reroute` (Bedrock refused the model and another answered, as a new conversation),
`restart` (the conversation had expired and the question was asked as a new one) and
`answer` (the whole answer: status, calls, tokens, this answer's cost and the
conversation's running total, time split between the model and Chiron). Each line of an
answer carries the same `answer`, `thread`, `user` and `dataset`, so:

```
docker compose logs --no-log-prefix ask | grep ' ask '                   # everything
docker compose logs --no-log-prefix ask | grep 'thread=6c3e'             # one conversation, end to end
docker compose logs --no-log-prefix ask | grep ' ask answer ' | grep 'user=demouser'   # one person's answers
docker compose logs --no-log-prefix ask | grep ' ask chiron ' | grep -c 'status=error' # failed Chiron calls
docker compose logs --no-log-prefix ask | grep ' ask answer ' | grep -o 'cost_usd=[0-9.]*' | cut -d= -f2 | paste -sd+ | bc   # spend
```

No question text, tool argument, filter value or result is written: a filter value can
be a patient's name, and the server log is read by people a trace is not shown to. An
error line carries a status, not the message; the trace has the message. The lines come
from the trace, so `CHIRON_MCP_TRACE=0` silences them too; `CHIRON_MCP_CALL_LOG` names a
file to append to instead of stderr, or `off`. In the deploy bundle and the add-on the
lines are in `docker compose logs ask`.

## The usage page

`/usage` on the Ask server (in the deploy bundle and the add-on, `<chiron address>/ask/usage`)
shows what Ask is being used for and what it costs: answers, spend, average answer time,
errors and tokens for a period (today, 7, 30 or 90 days, all, or a date range), spend per
day, and the same per model and per dataset, with the most recent answers listed one per
row and a CSV download of the period. Signed in as anyone, the page shows that person's
own answers. Signed in as a Django superuser, or as a member of the group
`CHIRON_MCP_USAGE_GROUP` names, it shows everyone's, a table per person (click a name to
see only theirs), and **Who has what**: every account with its access level on every
dataset as Ask applies it under the ceiling, the row's own level where that differs, the
two flags (identifying columns, workspace), the last sign-in, and a warning for any
dataset whose auto-create rule would give a newcomer access on their first visit to
Chiron's own pages, in red when that level is full access. A superuser changes access
right there: a level per dataset (no access, aggregate only, de-identified, full
access), whether a row may see identifying columns, and admin (superuser) for anyone
but themselves, so nobody can lock everyone out; full access and admin take a second
click. Every change goes into a change log under the table, with who made it, and
into the server's output.

The Ask page itself tells everyone their own total at the foot of its sidebar: questions
asked through Ask and what they cost, all time, refreshed as each answer finishes. The
link to the usage page sits beside it only for those who may see everyone's; the page's
back link leads to Chiron's dataset list (`CHIRON_MCP_UI_URL`).

Behind it is a ledger, `usage.sqlite` in the state directory (`CHIRON_MCP_USAGE_DB`
moves it or switches it off), with one row per answer: who, which dataset, at what
access level, when, which model, how many model and Chiron calls, the tokens, the cost
(this answer's share, and the conversation's running total), the time split between the
model and Chiron, and how it ended. No question text, no tool argument, no filter value,
no result, so it is kept for good (`CHIRON_MCP_USAGE_RETENTION_DAYS` prunes it) and is
safe to show an administrator; the trace under an answer has the details, for the person
who asked, while the conversation lives. The row is written when the answer finishes, at
the same moment as the call log's `answer` line, with the same figures.

Answers given before the ledger existed can be added from the call log, which carries
the same fields: `scripts/usage_backfill.py` reads call-log lines from stdin or files and
writes the answers the ledger does not have yet (rows it has are kept), so it is safe to
run again. In the deploy bundle:

```
docker compose logs --no-log-prefix ask | docker compose exec -T ask python /app/scripts/usage_backfill.py
```

`/usage.json?period=30d` is what the page reads, `/usage.csv?period=...` the download,
and `/usage/access.json` the who-has-what table with the change log; all three follow
the same rule about whose rows a person may see. `POST /usage/access` with
`{"username", "dataset", "level"}`, `{"username", "dataset", "subject_details"}` or
`{"username", "superuser"}` is the change, superusers only, same-site only.

## Troubleshooting

| Symptom | Cause |
|---|---|
| `Could not find the Chiron source tree` | Set `CHIRON_MCP_CHIRON_SRC` to the checkout containing `chiron/` |
| `No Django project settings at ...` | Set `CHIRON_MCP_PROJECT_DIR` (the directory with `manage.py`) and `CHIRON_MCP_BASE_SETTINGS` |
| `has no ChironUser on dataset` | Expected. Grant it with `scripts/grant_access.py` |
| `is a superuser. Refusing to start` | Use a dedicated service user, or set `CHIRON_MCP_OPERATOR=1` deliberately |
| `cannot see subject-level data` | The identity is `agg`. Use `chiron_crosstab` or `chiron_concept_values` |
| `Please select at least one value` | Wrong field name; call `chiron_describe_concept` and use its `filter_input` |
| Tools missing in the client | Restart it; check the paths in the config are absolute |
| `Server disconnected` / `No module named 'chiron_mcp'` | The client ignored `cwd`. Use the `.venv/bin/chiron-mcp` console script as `command`, with no `args` and no `cwd` |

## Project layout

```
chiron_mcp/
  config.py           environment configuration and path discovery
  bootstrap.py        starts Django in-process, keeping stdout clean for the protocol
  django_settings.py  inherits the host project's settings, overrides only the databases
  identity.py         identity resolution and the re-implemented permission gates
  oauth.py            OAuth login for the HTTP endpoint: sign in as a Chiron user
  filters.py          cohort filter input schemas, transcribed from validate_form()
  server.py           the 23 tools
  webapp.py           the Ask chat server embedded in the Chiron UI
  trace.py            the trace under each answer, and the hooks that feed the call log and the ledger
  calllog.py          the call log: one line per model call and per Chiron call
  usage.py            the usage ledger: one row per answer, kept for good, summarised for /usage
  header_login.py     sign-on by request header, for a Chiron behind a gateway (deploy bundle)
  chiron_patches.py   the two Chiron fixes, applied at run time when the host lacks them
  webui/index.html    the chat page (markdown, tables, charts, buttons)
  webui/usage.html    the usage page: spend, time, errors per person, model, dataset and day
scripts/
  harness.py          deployment check with no MCP involved
  grant_access.py     grant, update, revoke or list ChironUser rows
  usage_backfill.py   fill the usage ledger from call-log lines
  export_dataset.py   pack one dataset (dictionary + warehouse) into a file
deploy/               the whole stack in Docker: compose.yml, setup.sh, check.py
askpatch/             Ask as an add-on for an existing Chiron: compose.yml, proxy snippets, UI patch, check.py
tests/
  safety.py           permission and whole-dataset invariants
  smoke.py            end-to-end tool exercise
  protocol.py         real MCP stdio handshake
  protocol_http.py    the same over the HTTP endpoint, plus its loopback-only checks
  protocol_oauth.py   the HTTP endpoint's OAuth login, as two users with different access
  ask_e2e.py          the Ask tab end to end, checked against raw SQL and Chiron's API
  permissions.py      what each kind of account cannot get out of Ask, as six identities
  permissions_adversarial.py  every way past one's access level: tools in-process, questions, the web server
  usage.py            the usage ledger, its summaries and the backfill from the call log (offline)
  header_login.py     sign-on by request header: created on first sight, kept in step (offline)
  chiron_patches.py   the run-time fixes land on an unfixed Chiron and leave a fixed one alone
```

Built against Chiron 6.5.4.

## License

MIT