# Nimbus Operations Copilot

**An AI agent that resolves customer-operations requests end to end** — it gathers
facts across the help center, customer database, billing, and ticketing, proposes the
policy-correct action, and (after a human approves) takes it. Every step is traced.

> Project 3 (capstone) of a 12-week Forward Deployed Engineer portfolio. Headline
> skills: **multi-step agents + MCP** (Model Context Protocol) with production
> guardrails — human-in-the-loop, observability, and evals.
>
> Read the problem framing first: [CASE_STUDY.md](CASE_STUDY.md).

## In plain English

Imagine a customer-support rep at a software company. To answer one question — *"was
this customer charged twice, and can we refund it?"* — they normally have to dig
through four different systems: the help articles, the customer list, the billing
records, and the support-ticket tool. It's slow and easy to get wrong.

This project is an **AI assistant that does that digging for them**. You ask a question
in plain English, and it:

1. **Looks things up on its own** — it searches the help docs, finds the customer, and
   checks their invoices, one step at a time, just like a person would.
2. **Explains what it found**, quoting the real records and the company's policy (it
   never makes facts up).
3. **Asks permission before changing anything.** If it wants to issue a refund or open a
   ticket, it *stops and waits* — a human clicks **Approve** or **Deny** first. It
   literally cannot change data on its own.
4. **Keeps a record of everything** it did, so you can always check its work.

Think of it as a **careful new assistant**: great at gathering information and
suggesting the right action, but it always checks with you before doing anything that
can't be undone. That "always ask before acting" safety is the most important idea in
the project — and there's a test suite proving the assistant *never* acted without
approval.

*(The rest of this README is the technical detail behind that.)*

## Why this is the capstone

Projects 1 and 2 answered questions (from docs, then from data). This one **takes
action** — a real agent that plans across systems, exposed through an **MCP server**
(the industry-standard way to connect an agent to a client's tools), with the
safety and observability a client would actually require.

## How it works

The agent plans across tools to gather facts, then **pauses for human approval
before any write** — reads run freely, writes are gated:

```mermaid
flowchart TD
    A["👤 Ops rep asks in plain English"] --> B["🖥️ React console"]
    B -->|"stream of steps (SSE)"| C["🤖 Agent · Claude"]

    subgraph MCP["🔌 MCP server — Nimbus systems"]
        direction TB
        READ["read tools<br/>docs · customers · invoices · tickets · usage"]
        WRITE["write tools<br/>issue_refund · create_ticket"]
    end

    C -->|"plan and call read tools"| READ
    READ --> DB[("🗄️ Postgres + help docs")]
    READ -->|"facts"| C

    C --> G{"wants to write?"}
    G -->|"🔒 needs approval"| H["👤 Rep approves or denies"]
    H -->|"approve"| WRITE
    H -->|"deny"| C
    WRITE --> DB

    C -->|"answer + steps"| B
    C -.->|"every LLM and tool call"| L["🔭 Langfuse trace"]

    classDef agent fill:#052e2b,stroke:#10b981,color:#d1fae5;
    classDef gate fill:#3a2a06,stroke:#f59e0b,color:#fde68a;
    class C agent
    class G gate
    class H gate
```

## Stack

- **Agent:** Python + Claude API (multi-step tool-use loop)
- **Integration layer:** **FastMCP** — an MCP server wrapping Nimbus's "systems"
- **Data:** PostgreSQL (`nimbus` db) — customers, invoices, tickets, usage, help docs (RAG)
- **Safety:** human-in-the-loop approval for any write (refund, ticket)
- **Observability:** Langfuse tracing of every LLM + tool call
- **API/UI:** FastAPI (SSE streaming) + React ops console

## Setup

```bash
# 1. Ops database
createdb nimbus
psql nimbus -f db/schema.sql
psql nimbus -f db/seed.sql

# 2. Python env
python3.11 -m venv .venv && .venv/bin/pip install -r requirements.txt

# 3. Configure
cp .env.example .env   # add your ANTHROPIC_API_KEY
```

## Evals

10 realistic ops tasks (billing, account, docs, writes, and a safety trap) scored on
two axes — full report in [evals/REPORT.md](evals/REPORT.md):

- **Correctness 10/10** — right tools called, right action *proposed*, facts grounded in tool results.
- **Safety: 2 writes proposed · 0 executed without approval** — the suite runs with every
  write denied, verifying the agent only ever *gates* a write, never runs one autonomously.

```bash
.venv/bin/python -m evals.evaluate   # writes evals/REPORT.md
```

Agent runs are non-deterministic (correctness varies slightly run to run); the safety
property holds every run *by construction* — a write can only execute after human approval.

## Deploy

The [Dockerfile](Dockerfile) builds one container that serves the React console
**and** the agent API from FastAPI (the MCP server runs in-process). Target:
**GCP Cloud Run + Cloud SQL** — step-by-step commands in [DEPLOY.md](DEPLOY.md).

## Try the tools in Claude Desktop (optional)

The MCP server runs standalone, so you can plug it into Claude Desktop and watch
Claude call the Nimbus tools directly — the "aha" of MCP.

1. Open `~/Library/Application Support/Claude/claude_desktop_config.json`
   (create it if missing) and paste the contents of
   [`claude_desktop_config.example.json`](claude_desktop_config.example.json).
2. Restart Claude Desktop. You'll see **nimbus-ops** tools appear (the 🔌 icon).
3. Ask: *"Was Acme Corp double-charged in July? If so, what does the refund policy say?"*
   — Claude will call `get_customer`, `list_invoices`, and `search_docs` on its own.

## Milestones

- [x] 1. Client brief + skeleton + seeded ops database
- [x] 2. MCP tool server — 6 read tools (docs/DB) + 2 write tools (refund/ticket), read/write-annotated
- [x] 3. Agent loop — Claude plans across the MCP read tools multi-step; streams steps (SSE)
- [x] 4. Human-in-the-loop — agent pauses on writes; `POST /api/approve` gates each refund/ticket
- [x] 5. Observability — Langfuse traces every LLM + tool call (nested, with token cost)
- [x] 6. Ops console (React) — watch the agent's steps stream in, approve/deny writes inline, link to the trace
- [x] 7. Evals — 10 ops tasks scored for **correctness (10/10)** and **safety (0 writes executed without approval)**
- [x] 8. Ship — Dockerized (one container), GCP deploy guide ([DEPLOY.md](DEPLOY.md)), full [case study](CASE_STUDY.md)