Agentic Ops Platform

A multi-agent back-office automation platform where every state-changing action is gated behind human approval by construction — a supervisor agent routes each request to an order, inventory, or billing specialist; role-based middleware decides which tools that caller is even allowed to see; and any tool that writes to the database pauses for a human before it runs. The same tools are exposed a second way, through a real MCP server, so Claude Desktop or any other MCP client can drive the same operations under the same approval policy. Every turn is traced to MLflow, and a CI gate blocks the build if the agents' routing, approval behavior, or safety invariants regress.

It is built on the current 2026 stack: LangChain 1.4 create_agent with the agent middleware API, LangGraph checkpointing for interrupt/resume, and the mcp 2.x Python SDK (MCPServer — what FastMCP was renamed to).

                    ┌─────────────────┐
   user message ──▶ │   SUPERVISOR    │  routes only; never acts
                    └────────┬────────┘
              ┌──────────────┼──────────────┐
              ▼              ▼              ▼
        ┌──────────┐  ┌────────────┐  ┌──────────┐
        │  ORDER   │  │ INVENTORY  │  │ BILLING  │   each: own tools,
        │  agent   │  │   agent    │  │  agent   │   own prompt per role,
        └────┬─────┘  └─────┬──────┘  └────┬─────┘   own checkpoint thread
             └──────────────┼──────────────┘
                            ▼
              ┌──────────────────────────────┐
              │  RISK REGISTRY (one source)  │
              │  read_only │ low │ high risk │
              └───────┬──────────────┬───────┘
       enforced for   │              │   enforced for
       LangChain      ▼              ▼   MCP clients
   HumanInTheLoopMiddleware   pending-action tickets
                            │
                            ▼
                   ┌─────────────────┐
                   │  OpsRepository  │  SQLite, all invariants
                   └─────────────────┘

What makes it more than a demo

One risk declaration, enforced on two surfaces. safety/risk.py declares each tool's risk tier once. The LangChain agents turn that into a HumanInTheLoopMiddleware interrupt config; the MCP server turns the same registry into a pending-approval ticket queue. A client cannot use MCP as a back door around the agents' approval gate — tests/test_mcp_server.py asserts exactly that. Registering a new tool without declaring its risk raises rather than defaulting to "safe", so the system fails closed.

Role gating is middleware, not prompt text. safety/middleware.py generalizes the authenticated/unauthenticated pattern into role → tools and role → prompt maps. A guest never sees create_order; staff never see adjust_inventory. The model is not asked nicely to refuse — the tool is absent from the request, so refusing is the only thing it can do.

Approval is proven, not asserted. The tests check repository state, not response wording: after a gated request, stock and orders must be byte-for-byte unchanged until a human approves. Rejection must leave state untouched. A credit note cannot be approved twice.

The CI gate treats safety as absolute. scripts/gate_ci.py gives routing and gating accuracy zero tolerance, task success a 5% band — and treats safety_violations as a hard invariant that fails the build no matter what the baseline says. Proven by deliberately misclassifying create_order as read-only: gating accuracy fell 1.0 → 0.92 and the gate failed the build, then passed again once reverted.

Related MCP server: AgentsGate

Layout

src/ops_platform/
  domain/        SQLite repository + models — every business invariant lives here
  tools/         core.py (framework-free) + langchain_tools.py (the LangChain wrapper)
  safety/        risk.py (the registry), middleware.py (HITL + role gating), auth.py
  agents/        order / inventory / billing specialists + supervisor router
  mcp_server/    MCPServer exposing the same tools, resources, and a prompt
  observability/ MLflow tracing for every turn
  platform.py    the orchestration entry point
eval/            13-step scripted session, metrics, locked baseline
scripts/         gate_ci.py
demo/            runnable walkthrough + Claude Desktop config example
tests/           88 tests

Running it

No API key, no services, nothing to install beyond the requirements:

pip install -r requirements-dev.txt
python -m pytest tests/ -v                      # 88 tests
PYTHONPATH=src python demo/run_platform_demo.py  # full walkthrough
python scripts/gate_ci.py                        # eval + safety gate

Run the MCP server for Claude Desktop or any MCP client:

PYTHONPATH=src python -m ops_platform.mcp_server.server

See demo/claude_desktop_config_example.json for the client config.

Configuration — where the API key goes

By default there is no API key. LLM_PROVIDER=stub (the default) runs a deterministic, rule-based chat model that implements LangChain's tool-calling interface, so the demo, all 88 tests, and CI run fully offline and reproducibly.

To run the agents on a real LLM, set these in .env (see .env.example):

Swapping providers changes nothing else: the agents, middleware, risk registry, MCP server, and tests are all provider-agnostic.

Honest limitations

• The stub model does not understand language. It matches keywords and reads the system prompt for a role-appropriate fallback. It is there to make control flow — routing, gating, approval, role visibility — deterministic and testable, not to fake comprehension. Real language understanding comes from the Groq path.

• MCP approval uses ticket IDs, not native protocol elicitation. The 2026-07-28 MCP spec's Multi Round-Trip Requests could let a tool return resultType: "input_required" and have the client resume it directly. Client support for that is still new, so this uses a transport-agnostic pending-ticket queue instead — same policy, more portable.

• MLflow uses the local file store, which MLflow 3.x marks maintenance-mode (hence MLFLOW_ALLOW_FILE_STORE). Fine for a single machine; a real deployment would point MLFLOW_TRACKING_URI at Postgres or a hosted server.

• The supervisor routes, then the platform dispatches — specialists are not nested as subgraphs. That keeps each specialist's interrupt/resume cycle on its own checkpoint thread and independently testable, at the cost of not doing multi-hop agent-to-agent delegation.

Eval baseline

{
  "n_steps": 13,
  "routing_accuracy": 1.0,
  "gating_accuracy": 1.0,
  "task_success_rate": 1.0,
  "safety_violations": 0
}

The 13 steps walk one stateful session: read-only lookups, a guest denied an order, staff placing one through approval, a follow-up status check against the order ID the previous step created, a manager restock approved, another rejected, a manager-only credit note, and an out-of-scope message that must be routed nowhere.