# Agentic Ops Platform

A multi-agent back-office automation platform where **every state-changing
action is gated behind human approval by construction** — a supervisor agent
routes each request to an order, inventory, or billing specialist; role-based
middleware decides which tools that caller is even allowed to see; and any tool
that writes to the database pauses for a human before it runs. The same tools
are exposed a second way, through a real **MCP server**, so Claude Desktop or
any other MCP client can drive the same operations under the same approval
policy. Every turn is traced to MLflow, and a CI gate blocks the build if the
agents' routing, approval behavior, or safety invariants regress.

It is built on the current 2026 stack: LangChain 1.4 `create_agent` with the
agent middleware API, LangGraph checkpointing for interrupt/resume, and the
`mcp` 2.x Python SDK (`MCPServer` — what FastMCP was renamed to).

```
                    ┌─────────────────┐
   user message ──▶ │   SUPERVISOR    │  routes only; never acts
                    └────────┬────────┘
              ┌──────────────┼──────────────┐
              ▼              ▼              ▼
        ┌──────────┐  ┌────────────┐  ┌──────────┐
        │  ORDER   │  │ INVENTORY  │  │ BILLING  │   each: own tools,
        │  agent   │  │   agent    │  │  agent   │   own prompt per role,
        └────┬─────┘  └─────┬──────┘  └────┬─────┘   own checkpoint thread
             └──────────────┼──────────────┘
                            ▼
              ┌──────────────────────────────┐
              │  RISK REGISTRY (one source)  │
              │  read_only │ low │ high risk │
              └───────┬──────────────┬───────┘
       enforced for   │              │   enforced for
       LangChain      ▼              ▼   MCP clients
   HumanInTheLoopMiddleware   pending-action tickets
                            │
                            ▼
                   ┌─────────────────┐
                   │  OpsRepository  │  SQLite, all invariants
                   └─────────────────┘
```

## What makes it more than a demo

**One risk declaration, enforced on two surfaces.** `safety/risk.py` declares
each tool's risk tier once. The LangChain agents turn that into a
`HumanInTheLoopMiddleware` interrupt config; the MCP server turns the same
registry into a pending-approval ticket queue. A client cannot use MCP as a
back door around the agents' approval gate — `tests/test_mcp_server.py` asserts
exactly that. Registering a new tool without declaring its risk raises rather
than defaulting to "safe", so the system fails closed.

**Role gating is middleware, not prompt text.** `safety/middleware.py`
generalizes the authenticated/unauthenticated pattern into role → tools and
role → prompt maps. A guest never sees `create_order`; staff never see
`adjust_inventory`. The model is not asked nicely to refuse — the tool is
absent from the request, so refusing is the only thing it *can* do.

**Approval is proven, not asserted.** The tests check repository state, not
response wording: after a gated request, stock and orders must be *byte-for-byte
unchanged* until a human approves. Rejection must leave state untouched. A
credit note cannot be approved twice.

**The CI gate treats safety as absolute.** `scripts/gate_ci.py` gives routing
and gating accuracy zero tolerance, task success a 5% band — and treats
`safety_violations` as a hard invariant that fails the build no matter what the
baseline says. Proven by deliberately misclassifying `create_order` as
read-only: gating accuracy fell 1.0 → 0.92 and the gate failed the build, then
passed again once reverted.

## Layout

```
src/ops_platform/
  domain/        SQLite repository + models — every business invariant lives here
  tools/         core.py (framework-free) + langchain_tools.py (the LangChain wrapper)
  safety/        risk.py (the registry), middleware.py (HITL + role gating), auth.py
  agents/        order / inventory / billing specialists + supervisor router
  mcp_server/    MCPServer exposing the same tools, resources, and a prompt
  observability/ MLflow tracing for every turn
  platform.py    the orchestration entry point
eval/            13-step scripted session, metrics, locked baseline
scripts/         gate_ci.py
demo/            runnable walkthrough + Claude Desktop config example
tests/           88 tests
```

## Running it

No API key, no services, nothing to install beyond the requirements:

```bash
pip install -r requirements-dev.txt
python -m pytest tests/ -v                      # 88 tests
PYTHONPATH=src python demo/run_platform_demo.py  # full walkthrough
python scripts/gate_ci.py                        # eval + safety gate
```

Run the MCP server for Claude Desktop or any MCP client:

```bash
PYTHONPATH=src python -m ops_platform.mcp_server.server
```

See `demo/claude_desktop_config_example.json` for the client config.

## Configuration — where the API key goes

**By default there is no API key.** `LLM_PROVIDER=stub` (the default) runs a
deterministic, rule-based chat model that implements LangChain's tool-calling
interface, so the demo, all 88 tests, and CI run fully offline and reproducibly.

To run the agents on a real LLM, set these in `.env` (see `.env.example`):

| Variable | Needed when | Where to get it |
|---|---|---|
| `LLM_PROVIDER` | always (`stub` or `groq`) | — |
| `GROQ_API_KEY` | only when `LLM_PROVIDER=groq` | free at <https://console.groq.com> → API Keys |
| `LLM_MODEL` | optional | defaults to `llama-3.3-70b-versatile` |

Swapping providers changes nothing else: the agents, middleware, risk registry,
MCP server, and tests are all provider-agnostic.

## Honest limitations

- **The stub model does not understand language.** It matches keywords and
  reads the system prompt for a role-appropriate fallback. It is there to make
  control flow — routing, gating, approval, role visibility — deterministic and
  testable, not to fake comprehension. Real language understanding comes from
  the Groq path.
- **MCP approval uses ticket IDs, not native protocol elicitation.** The
  2026-07-28 MCP spec's Multi Round-Trip Requests could let a tool return
  `resultType: "input_required"` and have the client resume it directly. Client
  support for that is still new, so this uses a transport-agnostic pending-ticket
  queue instead — same policy, more portable.
- **MLflow uses the local file store**, which MLflow 3.x marks maintenance-mode
  (hence `MLFLOW_ALLOW_FILE_STORE`). Fine for a single machine; a real
  deployment would point `MLFLOW_TRACKING_URI` at Postgres or a hosted server.
- **The supervisor routes, then the platform dispatches** — specialists are not
  nested as subgraphs. That keeps each specialist's interrupt/resume cycle on its
  own checkpoint thread and independently testable, at the cost of not doing
  multi-hop agent-to-agent delegation.

## Eval baseline

```json
{
  "n_steps": 13,
  "routing_accuracy": 1.0,
  "gating_accuracy": 1.0,
  "task_success_rate": 1.0,
  "safety_violations": 0
}
```

The 13 steps walk one stateful session: read-only lookups, a guest denied an
order, staff placing one through approval, a follow-up status check against the
order ID the previous step created, a manager restock approved, another
rejected, a manager-only credit note, and an out-of-scope message that must be
routed nowhere.