# ops-agent

An LLM agent that turns natural-language requests into **audited, permission-gated actions**
on business systems — built on Anthropic tool use and exposed both as an HTTP API and as an
**MCP server**.

> "Which deals are stuck in negotiation above R$ 100k?" → `list_deals` → answer.
> "Close deal_102 as won, contract signed." → `update_deal_stage` → **refused unless writes are explicitly enabled**, and either way the attempt is in the audit log.

[![CI](https://github.com/ramonpalopoli/ops-agent/actions/workflows/ci.yml/badge.svg)](https://github.com/ramonpalopoli/ops-agent/actions)
![Python](https://img.shields.io/badge/python-3.10%2B-blue)
![License](https://img.shields.io/badge/license-MIT-green)

## Why this exists

Most "AI agent" demos let the model call anything with whatever arguments it produces.
That is fine for a notebook and unacceptable for a CRM or a finance system. This project is
a small, complete reference for the boring parts that make an agent deployable:

- **Allowlisted tools with schema-validated inputs.** The model is treated as an untrusted
  client; every argument goes through Pydantic before touching a database.
- **Write gating.** Mutating tools need an explicit opt-in per request *and* a global switch.
  A prompt injection hidden in a ticket subject cannot promote itself to a write.
- **Audit trail.** Every tool call — allowed or refused — is appended to a JSONL log with
  timing, outcome and PII redaction.
- **One registry, three surfaces.** The same `ToolRegistry` produces the Anthropic tool
  schema, the MCP tools and the test fixtures, so they cannot drift apart.
- **Testable without network.** The Anthropic client is injected behind a protocol; the agent
  loop is covered by unit tests with a scripted fake. Behavioural evals run against the real
  model when you want them to.

## Architecture

```mermaid
flowchart LR
    U[User / CLI / HTTP] -->|message| A[Agent loop]
    M[MCP client<br/>Claude Desktop, IDE] -->|tool call| S[MCP server]
    A -->|messages + tool schema| LLM[Claude<br/>Messages API]
    LLM -->|tool_use blocks| A
    A -->|validate · gate · execute| R[ToolRegistry]
    S --> R
    R --> T1[CRM tools<br/>accounts · deals · pipeline]
    R --> T2[Ticket tools]
    T1 & T2 --> DB[(SQLite<br/>synthetic data)]
    R -->|every call| AUD[(Audit log<br/>JSONL, redacted)]
```

**Request lifecycle**

1. `Agent.run()` sends the system prompt, the user message and the registry's tool schema.
2. For each `tool_use` block the model returns, the registry checks the allowlist, refuses
   mutating tools when writes are not permitted, validates arguments, and executes.
3. Results (or structured errors with `is_error=true`) go back to the model as
   `tool_result` blocks. The loop repeats until the model answers or `max_iterations` hits.
4. Each tool call is recorded in the audit log before the loop continues.

## Quickstart

```bash
git clone https://github.com/ramonpalopoli/ops-agent && cd ops-agent
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
cp .env.example .env            # add your ANTHROPIC_API_KEY

# Ask something (read-only by default)
ops-agent "How much open pipeline do we have, by stage?"

# A write is refused until BOTH switches are on:
ops-agent "Move deal_101 to negotiation, customer agreed on scope" --allow-writes
OPS_AGENT_ALLOW_WRITES=true ops-agent "Move deal_101 to negotiation, customer agreed on scope" --allow-writes
```

The database is created and seeded with synthetic accounts, deals and tickets on first run
(`data/ops.db`). No real companies or people are involved.

### HTTP API

```bash
export OPS_AGENT_API_TOKEN=$(python -c "import secrets;print(secrets.token_urlsafe(48))")
make api    # uvicorn on 127.0.0.1:8000

curl -s http://127.0.0.1:8000/v1/chat \
  -H "Authorization: Bearer $OPS_AGENT_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"message": "Which tickets are urgent?"}'

curl -s http://127.0.0.1:8000/v1/audit?limit=10 -H "Authorization: Bearer $OPS_AGENT_API_TOKEN"
```

| Route | Auth | Purpose |
|---|---|---|
| `GET /health` | none | liveness |
| `POST /v1/chat` | bearer | run the agent; body `{message, allow_writes}` |
| `GET /v1/audit?limit=` | bearer | last N audit events |

### MCP server

Expose the same tools to any MCP client. Writes are controlled solely by
`OPS_AGENT_ALLOW_WRITES` on the server side — a client cannot escalate itself.

```bash
ops-agent-mcp          # stdio transport
```

Claude Desktop config: see [`mcp.example.json`](mcp.example.json).

## Tools

| Tool | Writes | Input (validated) |
|---|---|---|
| `search_accounts` | no | `query` (1–80 chars), `limit` ≤ 25 |
| `get_deal` | no | `deal_id` matching `^deal_\d{3,6}$` |
| `list_deals` | no | optional `stage` (enum), `limit` |
| `pipeline_summary` | no | — |
| `list_open_tickets` | no | optional `priority` (enum), `limit` |
| `update_deal_stage` | **yes** | `deal_id`, `stage` (enum), `reason` (5–280 chars, audited) |
| `create_ticket` | **yes** | `account_id`, `subject`, `priority` |

Adding a tool = one Pydantic model + one handler + one `register()` call in
`src/ops_agent/tools/__init__.py`. It shows up in the API, the MCP server and the schema
tests automatically.

## Tests and evals

```bash
make test     # 33 unit/integration tests, no network
make lint     # ruff
make evals    # behavioural evals against the real model (needs ANTHROPIC_API_KEY)
```

Unit tests cover the agent loop (tool round-trip, write gating, iteration cap, error
surfacing), every tool (including a SQL-injection attempt and LIKE-wildcard escaping),
the HTTP API (auth, bounds, headers) and the MCP surface.

`evals/scenarios.json` asserts on *behaviour* rather than exact wording: which tools were
called, which were avoided, whether a refused write was reported honestly, and whether an
instruction smuggled into the prompt was ignored. Add a scenario, run `make evals`, ship.

## Security decisions

| Decision | Why |
|---|---|
| Tool allowlist + Pydantic validation of every argument | The model is an untrusted client. Unknown tools and malformed arguments are rejected before any I/O. |
| Parameterised SQL everywhere; `LIKE` wildcards escaped; enums for `stage`/`priority`; `ORDER BY` is a fixed `CASE` | CWE-89. User text never reaches an identifier or unbound position. |
| Writes require request opt-in **and** `OPS_AGENT_ALLOW_WRITES` | Fail-secure default; defence in depth against prompt injection. |
| Iteration cap | Bounded cost and latency; no runaway loops. |
| Audit log with e-mail / phone / CPF redaction | CWE-532, LGPD: reviewable without leaking personal data. |
| Bearer token compared with `hmac.compare_digest`; min 32 chars | CWE-208 timing attacks; weak-token misconfiguration fails at startup. |
| Generic error responses; details only in server logs | CWE-209. |
| `X-Content-Type-Options`, `X-Frame-Options`, `Referrer-Policy`, `Cache-Control: no-store` | Baseline hardening for a JSON API. |
| Random ticket ids (`secrets.token_hex`) | IDOR hardening (CWE-639). |
| Secrets only via environment / git-ignored `.env`; validated at startup | CWE-798. |

Known limitations, on purpose: single SQLite file (swap for Postgres by changing `db.py`),
no per-user identity on the API (add OIDC in front of it), no rate limiting (put it at the
gateway), and the system prompt is a mitigation, not a control — the controls are in code.

## Project layout

```
src/ops_agent/
  agent.py          # Messages API loop, guardrails, audit hooks
  api.py            # FastAPI surface
  mcp_server.py     # MCP surface (same registry)
  cli.py            # ops-agent CLI
  config.py         # pydantic-settings, validated at startup
  db.py             # SQLite schema + synthetic seed
  audit.py          # JSONL audit log with PII redaction
  tools/
    registry.py     # allowlist, validation, write gating
    crm.py          # accounts, deals, pipeline
    tickets.py      # support tickets
tests/              # pytest, offline
evals/              # behavioural scenarios against the real model
```

## Roadmap

- [ ] Postgres backend and connection pooling
- [ ] Per-user identity on the API and per-tool permissions (RBAC)
- [ ] Streaming responses
- [ ] Eval metrics over time (pass rate, tool precision) in CI

## License

MIT — see [LICENSE](LICENSE).