CFR Compliance MCP

An agentic compliance analysis system that evaluates contract clauses against authoritative U.S. federal regulations using MCP-based retrieval, deterministic rules, LLM reasoning, evidence verification, durable memory, human review workflows, and production-ready persistence.

Why This Project

LLMs are powerful, but they should not be allowed to freely invent regulatory evidence or autonomously execute high-stakes decisions.

This system demonstrates a controlled AI architecture where:

• authoritative regulatory retrieval comes first — current CFR text is the only source of regulatory truth

• deterministic checks constrain the workflow — LLM-free rules provide fast, auditable verdicts before any model is consulted

• LLM reasoning is evidence-grounded — the model must cite only the provided regulation, and provenance is attached by the retrieval layer, never the model

• verification gates uncertain outputs — conflicts and ungrounded results resolve to NEEDS_REVIEW, never silent acceptance

• historical memory is advisory — prior outcomes are context, not law, and can never override the current regulation

• humans control unresolved decisions — an explicit review lifecycle preserves the original automated result and an immutable audit trail

• persistence and audit trails make every decision inspectable — via filesystem or optional PostgreSQL, with a review dashboard

Key Capabilities

• MCP-based retrieval of the official eCFR API (8 typed tools)

• Configurable LLM evaluation via an OpenAI-compatible endpoint

• Deterministic compliance rules (LLM-free, auditable)

• Evidence grounding + independent verification

• Prompt-injection defenses and security boundaries

• Advisory compliance memory with durable JSONL storage

• Immutable analysis snapshots

• Optional PostgreSQL backend with versioned SQL migrations

• Queryable analysis history

• Human-in-the-loop review with optimistic concurrency

• Immutable audit events

• Server-rendered Jinja2 dashboard

• Docker + Docker Compose deployment

• Liveness and readiness endpoints

• Comprehensive offline + PostgreSQL integration + live LLM test suites

Architecture

flowchart TD
    C[Contract / Clause] --> S[Security Boundary]
    S --> R[Authoritative eCFR Retrieval]
    R --> M[Memory Lookup<br/>Advisory only]
    M --> D[Deterministic Evaluation]
    D --> L[LLM Evaluation]
    L --> V[Verification]
    V -->|uncertain| NR[NEEDS_REVIEW]
    V -->|verified| OK[VERIFIED]
    NR --> HR[Human Review]
    OK --> P[Persistence]
    HR --> P
    P -->|File| F[FileRepository - JSON reports]
    P -->|Optional PostgreSQL| PG[PostgresRepository - history + review]

Compliance Memory is drawn below retrieval on purpose: it is advisory context that can influence an evaluation, but it is never authoritative regulatory evidence and can never reorder the hierarchy.

Core Engineering Decisions

Authoritative Retrieval Before Reasoning

The pipeline runs security → retrieval → deterministic rules → LLM → verification. Regulatory text is always fetched and made usable before any reasoning begins. If retrieval fails, the clause resolves to NEEDS_REVIEW — the system will not guess.

LLMs Do Not Control Evidence Provenance

The model is instructed to reference only the provided regulation text, and evidence provenance (source, retrieved_at, retrieval_method, version) is attached by the retrieval layer. The model never manufactures where evidence came from; ungrounded output is routed to review.

Compliance Memory Is Advisory

Verified historical outcomes are stored in an append-only JSONL store and can be surfaced as labeled HISTORICAL_CONTEXT (data, never instructions). Memory:

• never establishes or replaces a CFR requirement

• is injected only into the user prompt, never the system instructions

• cannot trigger a verdict reuse unless current authoritative retrieval is usable

• is not auto-indexed when it participated (prevents feedback loops)

Human Review Does Not Overwrite Automated Results

A reviewer decision transitions an explicit lifecycle and appends an immutable event; it never rewrites the original ComplianceResult, evidence, or audit trail.

PostgreSQL Is Optional

file is the default and fully self-contained. PostgreSQL is an explicit opt-in that adds queryable history and the review workflow. There is no silent fallback — misconfiguration fails clearly.

Fail-Open vs Fail-Fast Boundaries

• Evaluation path: optional persistence/memory failures are logged and never fail a successful compliance analysis (fail-open).

• Configuration path: an invalid backend or unreachable configured database fails fast rather than silently degrading (fail-fast).

Human-in-the-Loop Review

NEEDS_REVIEW
    ↓
UNDER_REVIEW
    ↓
APPROVED / REJECTED / ESCALATED

• Optimistic concurrency — a decision carries an expected_version; a stale write is rejected instead of overwriting another reviewer.

• Immutable decision events — every transition is appended to an audit log.

• Original results preserved — the automated result, evidence, and audit are never modified by review.

Dashboard

A server-rendered Jinja2 dashboard is served by the API at /dashboard:

• Overview — aggregate compliance metrics and recent analyses

• Analysis history — paginated, filterable list with click-through

• Analysis detail — clause results, authoritative CFR evidence, verification, audit, and memory-participation indicator

• Review queue — actionable NEEDS_REVIEW items with state filters

• Review detail — automated result, evidence, verification/audit, memory context, immutable decision history, and a decision form (valid transitions only)

Authoritative CFR evidence and advisory historical memory are visually distinct — memory is never presented as regulation. No screenshots are bundled; run the app and visit /dashboard.

Quick Start

Prerequisites: Python 3.12+, uv, and a working eCFR connection. An LLM key is only needed for the LLM-fallback evaluation step.

File persistence (default)

cp .env.example .env       # fill in credentials if desired
uv sync --extra dev
uv run uvicorn api:app --host 0.0.0.0 --port 8000
open http://localhost:8000/dashboard

Enable on-disk report persistence by setting CFR_REPORTS_DIR=reports in .env.

PostgreSQL (opt-in)

# 1. Configure the backend + DSN
CFR_PERSISTENCE_BACKEND=postgres
CFR_DATABASE_URL=postgresql://user:pass@host/db

# 2. Apply the versioned schema (idempotent, safe to re-run)
uv run python scripts/migrate.py --dsn "$CFR_DATABASE_URL"

# 3. Start the API (fails fast at startup if the DB is unreachable)
uv run uvicorn api:app --host 0.0.0.0 --port 8000

Docker Compose (reproducible local stack)

docker compose up -d --build
open http://localhost:8000/dashboard

The Compose stack runs the app + PostgreSQL 16 with a named volume, a one-shot migration service, and health checks. The app waits for migrations to complete.

Environment Variables

Never commit a real .env; it is git-ignored and excluded from Docker builds.

API Endpoints

Testing

# Offline suite (no external services required)
uv run pytest --deselect tests/test_live_llm_integration.py -q

# Optional PostgreSQL integration tests (clean-skip without a test DB)
CFR_TEST_DATABASE_URL=postgresql://user:pass@host/db uv run pytest tests/test_postgres_integration.py

# Live LLM tests (require an LLM key + network)
uv run pytest tests/test_live_llm_integration.py

# Lint
uv run ruff check .

Current validation: 268 offline tests, 9 PostgreSQL integration tests (clean-skip when unavailable), and 9 live LLM tests.

Project Structure

agent/
  compliance_pipeline.py   # orchestration + authority order
  compliance_agent.py      # LLM evaluation
  security.py              # prompt-injection + input defenses
  deterministic_rules.py   # LLM-free rules
  verification_agent.py    # evidence/consistency verification
  memory.py / memory_store.py  # advisory compliance memory
  reporting.py             # report persistence
  persistence/             # File / Postgres / in-memory repositories + review lifecycle
api.py                     # FastAPI service
dashboard.py               # server-rendered dashboard routes
templates/ static/         # dashboard UI
migrations/ scripts/       # SQL migrations + migration runner
src/cfr_compliance_mcp/    # FastMCP server + eCFR client + tools
tests/                     # offline, integration, live suites
benchmark/                 # deterministic benchmark harness
compose.yaml  Dockerfile  pyproject.toml  uv.lock

Tech Stack

AI / LLM — FastMCP, eCFR retrieval, deterministic rules, configurable OpenAI-compatible LLM, verification, advisory memory

Backend — Python 3.12+, FastAPI, Pydantic v2, httpx, structured logging

Persistence — filesystem JSON reports (default) + optional PostgreSQL (psycopg 3, versioned SQL migrations)

Infrastructure — Docker, Docker Compose, health/readiness endpoints

Testing — pytest, offline + optional PostgreSQL integration + live LLM

What This Project Demonstrates

• Agentic AI system design — a full retrieval → reason → verify → decide pipeline with explicit boundaries

• MCP tool integration — a typed server wrapping a real regulatory API

• LLM orchestration — deterministic + probabilistic control flow

• Retrieval and evidence grounding — provenance-first design

• AI security boundaries — prompt-injection defense and input sanitization

• Human-in-the-loop workflows — explicit review lifecycle with audit

• Transactional persistence — optimistic concurrency and immutable events

• API engineering — validated request/response models, structured errors

• Dockerized deployment — reproducible builds and a Compose stack

• Testing and integration validation — offline, PostgreSQL, and live suites

Current Limitations

• reviewer_identity is an unauthenticated placeholder (no auth system yet)

• PostgreSQL is single-instance (no distributed orchestration)

• Live LLM tests depend on external provider availability

• Compliance Memory is intentionally local and advisory

• The system does not automatically execute remediation

Roadmap

• Authenticated reviewer identities and role-based access control

• Production observability and richer audit reporting

• Background job processing for long-running evaluations

• A broader, versioned compliance benchmark suite

• Multi-node deployment support

AI-assisted compliance analysis is not a substitute for legal or regulatory professionals. Non-trivial findings are routed to human review; nothing in this system autonomously issues legal authorization.