# mcp-compliance-router

An MCP server that screens text for PHI/PII, classifies it, redacts it, and
routes it to an approved model tier under a declarative policy, with an
evaluation harness that measures exactly how well the screening works.

This is a reference implementation of the enterprise AI gateway pattern
(screen, classify, redact, route, audit) at the smallest size that still
demonstrates the whole loop. It is not a certified compliance product; see
Design notes and limitations.

## Why this exists

Enterprises in regulated industries cannot send raw text to whichever model
is best for the task. Something must sit in the path that answers three
questions before any call is made: what is in this text, where is it allowed
to go, and what record proves the decision. In large healthcare environments
that job belongs to an AI gateway. This repo is that pattern, small enough to
read in one sitting and instrumented so its accuracy is measured, not assumed.

## Architecture

```mermaid
flowchart LR
    A[Incoming text] --> B[Screen: regex entity detectors]
    B --> C{Classify}
    C -- restricted --> D[Redact] --> E[Restricted tier: private models]
    C -- sensitive --> F[Redact] --> G[Standard tier]
    C -- clean --> G
    D --> H[(Audit log: entity types + SHA-256, never raw text)]
    F --> H
```

Three modules, one policy file:

- `core.py` detects entities (SSN, MRN, NAME, DOB, ADDRESS, EMAIL, PHONE),
  classifies the document (clean, sensitive, restricted), redacts spans, and
  writes containment-friendly audit records that store entity types and a
  SHA-256 of the text, never the text itself.
- `policy.py` loads `policy/routing_policy.yaml` and turns a classification
  into a routing decision: target tier, allowed models, action, audit flag.
  Unknown classifications fail closed to the restricted rule.
- `server.py` exposes the loop as three MCP tools: `screen_text`,
  `route_request`, and `get_audit_log`.

## Quickstart

```bash
pip install -e ".[mcp,dev]"
pytest                      # unit tests
python eval/run_eval.py     # regenerate eval/report.md
python -m compliance_router.server   # run the MCP server on stdio
```

## Use it from Claude

Claude Code:

```bash
claude mcp add compliance-router -- python -m compliance_router.server
```

Claude Desktop (`claude_desktop_config.json`):

```json
{
  "mcpServers": {
    "compliance-router": {
      "command": "python",
      "args": ["-m", "compliance_router.server"]
    }
  }
}
```

Then ask, for example: "Screen this support ticket with compliance-router and
tell me which tier it routes to and why."

## Evaluation

Screening quality is measured, not asserted. `eval/dataset.jsonl` holds 36
labeled synthetic examples (every identifier is fabricated), and
`eval/run_eval.py` produces `eval/report.md` with per-entity precision,
recall, and F1, document classification accuracy, routing tier accuracy, and
a failure analysis listing each miss and false alarm. The current report is
committed so the limits of the regex heuristics are visible in the repo.

## Design notes and limitations

- Detection is regex-based on purpose: it keeps the reference readable and
  makes the evaluation honest. It is not production-grade PHI detection.
  Bare person names without an honorific or patient-context cue are missed
  by design of the current NAME heuristic, and the report shows it.
- The detector sits behind a small interface, so swapping in a stronger
  backend (Presidio, a clinical NER model, or an LLM-based screen with
  conformal abstention) changes one module and rerunning the harness shows
  exactly what the upgrade bought.
- The audit log is in-memory for the reference; production would ship the
  same records to a log platform. The records are designed to be safe to
  ship: entity types and hashes, never content.
- Fail-closed policy: any classification the policy does not recognize is
  blocked and audited.

## Roadmap

- Pluggable detector backends (Presidio, clinical NER) behind the same
  interface, compared by the same harness
- Conformal abstention thresholds on the classifier, so uncertain documents
  escalate instead of guessing
- Policy lint: validate the YAML against tiers at startup
- Per-tenant policies keyed by MCP client identity

## License

MIT. All example data is synthetic. Developed with AI-assisted workflows
(Claude Code and similar agents) with human architecture and review.