# Control Inventory Agent

A small project that puts an LLM agent in front of a security control inventory. The
data is exposed as callable tools through the **Model Context Protocol (MCP)**; you ask
a question in plain English, and the agent decides which tools to call, chains them, and
answers.

It's a learning project, built to feel the difference between a *workflow* (code decides
the steps) and an *agent* (the model decides the steps), and to learn MCP hands-on.

![Control Inventory MCP](docs/control-inventory-mcp.png)

```
control-inventory-agent/
├── controls.json      # sample control inventory (synthetic)
├── server.py          # MCP server: search_controls, get_control, list_gaps
├── agent.py           # the agent loop that drives the server (via AWS Bedrock)
├── requirements.txt   # mcp, anthropic (+ boto3 for Bedrock)
├── .gitignore
└── README.md
```

## How it works (current state)

Three pieces:

**`controls.json`**, the data. A small, synthetic inventory of ~16 security controls
across CIS v8 / NIST 800-53 / SOC 2, each with an id, title, description, implementation
status (`implemented` / `partial` / `not_implemented`), owner, and cross-framework
mappings. This is sample data, not a real inventory.

**`server.py`**, an MCP server (FastMCP, from the official `mcp` SDK) that exposes three
tools over that data:

- `search_controls(query)`, keyword/substring match across the controls
- `get_control(control_id)`, full detail for a single control
- `list_gaps()`, controls that are `partial` or `not_implemented`

The tools are deliberately simple: `search_controls` is a keyword match, `get_control` is
a lookup, `list_gaps` is a filter. **There is no LLM inside the server**, it's plain data
access.

**`agent.py`**, the agent loop. It connects to the server over stdio, hands Claude (via
AWS Bedrock) the server's tools, and runs the loop:

1. send the question to Claude with the available tools,
2. if Claude responds with a tool call, execute it and feed the result back,
3. repeat until Claude stops asking for tools, then return its answer.

The model chooses which tools to call and in what order. Nothing about that sequence is
hardcoded, that loop (about 20 lines in `agent.py`) is the entire "agent."

### Example

Ask: *"We got flagged for missing S3 access logging. Which controls cover that, are they
implemented, and where's the gap?"*

The agent, on its own:

1. calls `search_controls("logging")` → finds the audit/logging controls,
2. calls `get_control(...)` on the matches to check their status,
3. answers: the relevant controls (CIS-8.2, NIST-AU-2/3, SOC2-CC7.2, …), flagging that
   CIS-8.2 is `not_implemented`, the real gap, while the others are `partial`.

You didn't tell it to search-then-fetch-then-assess. It decided that.

### Running it

```bash
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt boto3        # boto3 is needed for the Bedrock client

export AWS_REGION=us-east-1                   # a region with Bedrock access to Claude Haiku
aws sts get-caller-identity                   # confirm your AWS credentials are valid

python agent.py "Which logging controls aren't implemented?"
# or run with no argument for an interactive prompt
```

The agent uses `AnthropicBedrock()`, which reads your AWS credentials the same way `boto3`
does. To use a direct Anthropic API key instead, see the comment at the bottom of
`agent.py`.

## When is this worth it? MCP + agent vs. keyword search vs. SQL

This is the question that matters, because the agent's tools *are* keyword search and
lookups. So when does wrapping them in an LLM agent actually earn its cost, and when is
plain search or a SQL query the better answer?

| You want to… | Reach for | Why |
|---|---|---|
| Get exact, specifiable results ("all NIST controls", "count by status", "gaps owned by SecOps") | **SQL** | deterministic, fast, free, auditable |
| Match text on a known keyword | **keyword search / grep** | trivial, instant, no model needed |
| Answer a fuzzy question without knowing the schema ("where's our logging gap?") | **agent** | the LLM maps intent to the right lookups |
| Chain steps where each depends on the last, and the shape varies by question | **agent** | the LLM sequences calls dynamically |
| Turn several results into an explanation, not just rows | **agent** | synthesis and reasoning over the data |
| Reach many different systems through one interface | **MCP** (with or without an agent) | one standard, reused by any client |

### The honest version

**Keyword search and SQL win when the question is precise and you can express it
directly.** "List every control where `status != 'implemented'` and `owner = 'SecOps'`" is
a one-line SQL query: instant, free, exact, and reproducible. Putting an LLM in front of
that adds latency, token cost, and the chance of a different answer next run, all
downside, no upside. For a *system of record*, especially in compliance where
auditability matters, deterministic queries are usually the right call.

**The agent earns its place when the question is fuzzy, multi-step, or needs synthesis.**
"We got flagged for missing S3 access logging, where's our gap?" is not a query anyone
wrote field names for. The user doesn't know the data is keyed on "audit logging" rather
than "S3 access logging," doesn't know which framework, and doesn't know the steps. The
LLM's contribution is the translation: from a vague human sentence to "search for logging
controls, look up each one's status, and explain the gap." It decides the sequence, adapts
it when the next question is shaped differently, and writes an answer instead of returning
rows. That's the part SQL can't do.

A useful way to hold it: **the agent doesn't replace search or SQL, it sits on top and
decides when and how to call them.** It's a natural-language front door over your existing
lookups, not a new system of record.

### Where MCP specifically fits

MCP is a separate question from "agent vs. SQL." MCP is the *interface standard*, the way
a tool or data source advertises what it can do so that any compatible client can use it.
Its payoff isn't intelligence; it's **reuse and portability**. The same `server.py` here
works unchanged with Claude Desktop, with this custom agent, with Cursor, or with any other
MCP-aware host, you expose the data and tools once, and every client speaks the same
protocol. Without MCP, each of those integrations would be bespoke glue code.

So the two decisions are independent:

- *Do I need an agent?*, only if the questions are fuzzy/multi-step/synthesis-heavy.
- *Should I use MCP?*, worth it whenever the same tools will be reached by more than one
  client, or you want to plug into the growing MCP ecosystem, regardless of whether an
  agent is involved.

For this project, MCP is somewhat over-spec for a single JSON file, a direct function call
would do. It's here because the *point* was to learn the protocol and the agent loop, and
because the realistic version (pointing the server at a live control system, reached by
several clients) is exactly where MCP stops being over-spec.

## Note on the data

`controls.json` is **synthetic sample data**. If you swap in a real control inventory, do
**not** commit it to a public repo: a real inventory lists which controls are *not*
implemented, which is effectively a public map of your security gaps (plus internal owners
and systems). Keep the sample file in the repo and your real data local and git-ignored.