# iac-security-copilot

Ask natural-language questions about a Terraform plan and get cited, evaluated answers grounded in the **CIS AWS Foundations Benchmark** and **OWASP Top 10 CI/CD Security Risks**. Run it as a CLI, hit it over HTTP, or expose it to Claude Code / Cursor as an MCP server — same retriever, same scan agent, same citations.

What's in the box:

- **RAG over a security corpus** — hybrid retrieval (BM25 + dense + RRF + cross-encoder rerank) over Postgres + pgvector.
- **LangGraph scan agent** — parses a Terraform plan JSON, fans out one branch per resource, retrieves matching guidance, asks an LLM (Anthropic / OpenAI / Ollama / a deterministic stub) for a structured finding with citations.
- **MCP stdio server** — `search_corpus`, `scan_terraform_plan`, `list_sources` exposed as tools so editor agents can ground their answers in the same corpus.
- **FastAPI service** — `POST /v1/search`, `POST /v1/scan`, `GET /v1/sources`; same business logic as the CLI and MCP server.
- **Eval harness** — 22 hand-curated golden cases (17 Terraform plan JSON + 5 CloudFormation templates), scored with custom resource-centric matching + severity tolerance + keyword grounding + citation correctness. Committed regression reports for every supported LLM.
- **Local Kubernetes demo** — `./k8s/up.sh` brings up a `kind` cluster with Postgres + the FastAPI app behind ingress-nginx + a real HPA in one command.

## Latest eval reports

Two reports live side-by-side, each answering a different question:

- **[`evals/qwen3-baseline.md`](evals/qwen3-baseline.md)** — real-quality measurement against `ollama:qwen3:latest` (8B). The headline number when you want to know what the product actually does.
- **[`evals/stub-baseline.md`](evals/stub-baseline.md)** — deterministic regression check using the stub LLM. F1=1.00 by construction — this baseline gates the *framework*, not the *product*. Runs in <1s.

Top-line against the 17 Terraform cases of the 22-case golden set (CFN cases are excluded from the stub baseline because the stub's regex patterns target Terraform conventions — see the `--exclude-tags` flag in the CLI reference). Corpus is OWASP CI/CD Top 10 + CIS AWS Foundations Benchmark; citations accepted from either framework.

| Provider | Precision | Recall | F1 | TP | FN | FP | Wrong severity | Notes |
|---|---|---|---|---|---|---|---|---|
| **`ollama:qwen3:latest`** | **1.00** | **0.90** | **0.95** | 9 | 1 | 0 | 3 | Real LLM. Surfaced 1 recall miss (DB deletion protection), 1 severity inflation, 2 scorer-pedantry keyword misses. ~15 min sequential run. |
| `stub` | 1.00 | 1.00 | 1.00 | 13 | 0 | 0 | 0 | Pattern-match heuristics aligned to the golden cases by construction. Use for CI regression, not quality measurement. |

The qwen3 F1=0.95 is the honest signal. Two of the three `wrong_severity` entries are word-form issues the scorer is over-strict on (e.g. expected `"encrypted"` didn't substring-match qwen3's `"encryption"`); the third is real severity inflation on the versioning case (qwen3 marked it CRITICAL where expected is MEDIUM). The one false negative is qwen3 missing the deletion-protection medium in the multi-issue plan. See `evals/qwen3-baseline.md` for the per-case detail.

Switching providers is one env var: `LLM_PROVIDER=anthropic ANTHROPIC_API_KEY=... iac-copilot eval --output evals/anthropic-baseline.md` produces a comparable report for any future model.

## Architecture

```mermaid
flowchart LR
    %% --- Inputs / external ---
    USER["Engineer / AI client"]
    OWASP[("OWASP CI/CD Top 10<br/>raw.githubusercontent.com")]
    CIS[("CIS AWS Foundations<br/>corpus/cis-aws/ (committed)")]
    WAR[("AWS Well-Architected<br/>(planned)")]
    PLAN["Terraform plan JSON"]

    %% --- CLI entry point ---
    CLI["iac-copilot CLI<br/>(migrate · ingest · query · plan · scan · stats)"]

    %% --- Ingestion path (done) ---
    subgraph Sources["ingestion/sources"]
        OWASP_SRC["OwaspCicdSource"]
        CIS_SRC["CisAwsSource"]
        WAR_SRC["AwsWarSource (planned)"]
    end

    PIPE["pipeline.py<br/>orchestrator"]
    CHUNK["chunker.py<br/>heading-aware Markdown<br/>+ frontmatter strip"]
    EMB["embedder.py<br/>BGE-small / OpenAI factory"]

    %% --- Storage ---
    DB[("Postgres 16 + pgvector<br/>documents · chunks<br/>HNSW + GIN indices")]

    %% --- Retrieval ---
    subgraph Retrieval["retrieval/"]
        BM25["bm25.py<br/>ts_rank_cd (OR-rewritten)"]
        DENSE["dense.py<br/>pgvector cosine"]
        HYBRID["hybrid.py<br/>RRF fusion (k=60)"]
        RERANK["reranker.py<br/>fastembed cross-encoder"]
    end

    %% --- Scan agent ---
    subgraph Scan["plan/ + agent/"]
        PARSER["terraform.py<br/>parse_terraform_plan<br/>→ ResourceGraph"]
        GRAPH_NODE["agent/graph.py<br/>LangGraph<br/>(Send fan-out)"]
        LLM["agent/llm/<br/>Anthropic · OpenAI · Ollama · Stub"]
    end

    REPORT["ScanReport<br/>(findings + citations)"]

    %% --- Additional protocol surfaces ---
    FASTAPI["FastAPI service<br/>/v1/search · /v1/scan · /v1/sources"]
    MCP["MCP stdio server<br/>search_corpus · scan_terraform_plan · list_sources"]

    %% --- Eval harness ---
    GOLD["golden/<br/>22 hand-curated cases<br/>(17 TF plan + 5 CFN)"]
    RAGAS["eval/<br/>scorers + reporter<br/>(Markdown regression report)"]

    %% --- Ingestion edges (solid: done) ---
    USER --> CLI
    OWASP --> OWASP_SRC
    OWASP_SRC --> PIPE
    CIS --> CIS_SRC
    CIS_SRC --> PIPE
    WAR -.-> WAR_SRC
    WAR_SRC -.-> PIPE
    CLI --> PIPE
    PIPE --> CHUNK
    CHUNK --> PIPE
    PIPE --> EMB
    EMB --> PIPE
    PIPE --> DB

    %% --- CLI query path (solid: done) ---
    CLI --> HYBRID
    DB --> BM25
    DB --> DENSE
    BM25 --> HYBRID
    DENSE --> HYBRID
    HYBRID --> RERANK
    RERANK --> USER

    %% --- CLI scan path (solid: done) ---
    PLAN --> CLI
    CLI --> PARSER
    PARSER --> GRAPH_NODE
    GRAPH_NODE --> HYBRID
    GRAPH_NODE --> LLM
    LLM --> GRAPH_NODE
    GRAPH_NODE --> REPORT
    REPORT --> USER

    %% --- FastAPI + MCP entry points ---
    USER --> FASTAPI
    USER --> MCP
    PLAN --> FASTAPI
    FASTAPI --> PARSER
    FASTAPI --> HYBRID
    MCP --> HYBRID
    MCP --> PARSER

    %% --- Eval edges ---
    GOLD --> RAGAS
    GRAPH_NODE --> RAGAS

    %% --- Styling ---
    classDef done fill:#e8f5e9,stroke:#2e7d32,color:#000
    classDef planned fill:#fff3e0,stroke:#ef6c00,color:#000,stroke-dasharray: 4 3
    classDef storage fill:#e3f2fd,stroke:#1565c0,color:#000
    classDef external fill:#fafafa,stroke:#616161,color:#000

    class CLI,OWASP_SRC,CIS_SRC,PIPE,CHUNK,EMB,BM25,DENSE,HYBRID,RERANK,PARSER,GRAPH_NODE,LLM,REPORT,GOLD,RAGAS,FASTAPI,MCP done
    class WAR_SRC planned
    class DB storage
    class USER,OWASP,CIS,WAR,PLAN external
```

**Legend** — green nodes and solid edges are implemented. Orange dashed nodes are planned (AWS Well-Architected as an additional source). The blue node is the storage layer; grey nodes are external actors / sources.

Six paths through the system:

1. **Ingestion** (`iac-copilot ingest`): sources → pipeline → chunker + embedder → Postgres.
2. **Query** (`iac-copilot query`): natural-language question → BM25 + Dense retrievers → RRF fusion → cross-encoder rerank → top chunks.
3. **Scan** (`iac-copilot scan`): Terraform plan JSON → parser → LangGraph agent (per-resource parallel branches) → retrieves guidance + reasons with the configured LLM (Anthropic / OpenAI / Ollama / Stub) → structured `ScanReport` with citations.
4. **Eval** (`iac-copilot eval`): runs the scan agent against every hand-curated `golden/` case, scores each finding for resource match + severity bucket + keyword grounding + citation correctness, and emits a Markdown regression report.
5. **HTTP API** (`POST /v1/search`, `POST /v1/scan`, `GET /v1/sources`): same retriever and scan agent reachable over HTTP, suitable for a deployed service.
6. **MCP server** (`iac-copilot mcp`): same retriever and scan agent reachable as stdio MCP tools, suitable for Claude Code / Cursor.

The retrieval subgraph is the seam — the CLI's `query` command calls into it directly; the scan agent's per-resource branches call into the same retrievers; the FastAPI and MCP entry points wrap the same `service.py` factories so behaviour stays identical across protocols.

### How a scan composes

The Mermaid diagram above shows the system at one altitude. Zooming into the `iac-copilot scan` path specifically:

```
Terraform plan JSON
    ↓  (parser — generic walker over the plan JSON, no per-resource code)
ResourceGraph
    ↓  (LangGraph agent — Send-based parallel fan-out)
    │
    │  for each create/update/replace resource:
    │      ↓  retrieve guidance         (hybrid retriever:
    │      │                             BM25 + Dense → RRF → cross-encoder rerank)
    │      ↓  analyze with LLM          (Anthropic via tool-use /
    │      │                             OpenAI via response_format strict /
    │      │                             Ollama via format=json-schema /
    │      │                             Stub via deterministic heuristics)
    │      ↓  structured Finding        (Pydantic-validated — severity, issue,
    │                                    why-it-matters, fix, citations, confidence)
    │
    ↓  (merge via Annotated[list, operator.add] reducers — N branches → 1 state)
ScanReport (JSON via --output / --json, or human-readable with severity colours)
```

Each box has a single owner module: the parser doesn't know about retrieval, the retriever doesn't know about LLMs, the agent doesn't know about Postgres. Adding a new input format (CloudFormation, Pulumi) means writing a new parser that produces the same `ResourceGraph` shape — nothing else changes. Swapping LLM providers is one env var.

## Repository structure

```
iac-security-copilot/
├── Dockerfile                            # multi-stage build (uv + slim runtime, non-root, Cloud Run-ready)
├── docker-compose.yml                    # local Postgres 16 + pgvector
├── pyproject.toml                        # project metadata, deps, ruff config, CLI entry point
├── uv.lock                               # locked dependency graph (committed; used by --frozen builds)
├── .env.example                          # template for runtime config (copy to .env)
├── .gitignore                            # excludes .venv, .env, corpus caches, fastembed_cache
├── .dockerignore                         # keeps build context lean
├── .pre-commit-config.yaml               # hooks: ruff, detect-secrets, hadolint, hygiene checks
├── .secrets.baseline                     # detect-secrets allowlist (empty for now)
├── db/
│   └── init/
│       └── 01-extensions.sql             # CREATE EXTENSION vector; runs on first Postgres start
├── src/
│   └── iac_security_copilot/
│       ├── __init__.py                   # version constant
│       ├── main.py                       # FastAPI app — /health + /v1/search + /v1/scan + /v1/sources
│       ├── service.py                    # shared retriever + scan agent factory used by FastAPI and the MCP server
│       ├── mcp_server.py                 # MCP (stdio) server exposing search_corpus, scan_terraform_plan, list_sources
│       ├── cli.py                        # `iac-copilot` command surface (migrate/ingest/query/plan/scan/eval/mcp/stats/reset)
│       ├── config.py                     # pydantic-settings: env vars → typed Settings object
│       ├── db/
│       │   ├── client.py                 # context-managed psycopg connection with pgvector adapter registered
│       │   └── migrations.py             # schema (documents, chunks) parameterised by embedding dimension; generated tsvector column
│       ├── ingestion/
│       │   ├── embedder.py               # Embedder protocol + BGE-small + OpenAI implementations + factory
│       │   ├── chunker.py                # heading-aware Markdown chunker + Jekyll frontmatter stripper
│       │   ├── pipeline.py               # orchestration: fetch → chunk → embed → persist
│       │   └── sources/
│       │       ├── base.py               # Source protocol + FetchedDocument dataclass
│       │       ├── owasp_cicd.py         # OWASP Top 10 CI/CD Security Risks fetcher (GitHub raw)
│       │       └── cis_aws.py            # CIS AWS Foundations Benchmark — reads from corpus/cis-aws/ with provenance URLs
│       ├── retrieval/
│       │   ├── base.py                   # Retriever protocol + RetrievedChunk dataclass
│       │   ├── dense.py                  # pgvector cosine similarity via HNSW
│       │   ├── bm25.py                   # Postgres FTS (ts_rank_cd) with OR-rewritten plainto_tsquery
│       │   ├── hybrid.py                 # Reciprocal Rank Fusion across dense + BM25
│       │   └── reranker.py               # fastembed cross-encoder (Xenova/ms-marco-MiniLM-L-6-v2)
│       ├── plan/
│       │   ├── base.py                   # Resource, ResourceGraph, Action, ResourceMode — provider-agnostic
│       │   └── terraform.py              # parse_terraform_plan — generic walker over plan JSON, no per-resource code
│       ├── agent/
│       │   ├── schemas.py                # Finding, Citation, ResourceAnalysis, ScanReport, AgentState (TypedDict + reducer)
│       │   ├── prompts.py                # SYSTEM_PROMPT + build_user_prompt (redacts sensitive attributes)
│       │   ├── graph.py                  # LangGraph topology: load_plan → Send-based fan-out → analyze_one
│       │   └── llm/
│       │       ├── base.py               # LLMClient Protocol
│       │       ├── factory.py            # get_llm_client(settings)
│       │       ├── anthropic_client.py   # Claude via tool-use (structured output enforcement)
│       │       ├── openai_client.py      # OpenAI via response_format=json_schema strict
│       │       ├── ollama_client.py      # Local model via Ollama; format=schema for structured output
│       │       └── stub.py               # Deterministic heuristic provider (no API key needed)
│       └── eval/
│           ├── schemas.py                # GoldenCase, ExpectedFinding, FindingMatch, CaseResult, EvalReport
│           ├── golden.py                 # load_case + load_golden_set (reads golden/<id>/{plan,expected}.json)
│           ├── scorers.py                # score_case — resource-centric matching with severity tolerance + keyword grounding
│           ├── harness.py                # run_eval — orchestrates per-case scans with error isolation
│           └── reporter.py               # render_report — Markdown with aggregate + severity breakdown + per-case + diagnostic
├── corpus/
│   └── cis-aws/                           # 10 CIS AWS Foundations Benchmark control summaries (markdown)
│       ├── CIS-1.16-iam-admin-policy.md   # IAM: no full administrative privileges
│       ├── CIS-1.22-iam-wildcard-trust.md # IAM: no wildcard principals in trust policies
│       ├── CIS-2.1.1-s3-encryption.md     # S3: server-side encryption at rest
│       ├── CIS-2.1.2-s3-versioning.md     # S3: versioning + MFA delete
│       ├── CIS-2.1.5-s3-public-access.md  # S3: Block Public Access settings
│       ├── CIS-2.3.1-rds-encryption.md    # RDS: storage encryption
│       ├── CIS-2.3.2-rds-deletion-protection.md  # RDS: deletion protection
│       ├── CIS-2.3.3-rds-public-access.md # RDS: publicly_accessible = false
│       ├── CIS-4.1-sg-ssh-from-internet.md   # SG: no 0.0.0.0/0 on port 22
│       └── CIS-4.2-sg-rdp-from-internet.md   # SG: no 0.0.0.0/0 on port 3389
├── docs/
│   └── mcp.md                            # editor integration recipe — Claude Code + Cursor `.mcp.json` snippets
├── k8s/                                  # local Kubernetes demo via kind
│   ├── README.md                         # one-command bring-up + manifest walkthrough
│   ├── up.sh / down.sh                   # idempotent bring-up + tear-down scripts
│   ├── kind-cluster.yaml                 # kind config — single node + ingress port mapping
│   ├── 00-namespace.yaml                 # iac-copilot namespace
│   ├── 10-configmap.yaml                 # non-secret env: DATABASE_URL, LLM_PROVIDER, retrieval knobs
│   ├── 11-secret.example.yaml            # template; real Secret is created by up.sh from host env vars
│   ├── 20-postgres.yaml                  # pgvector/pgvector:pg16 Deployment + Service (emptyDir)
│   ├── 30-app-deployment.yaml            # FastAPI Deployment with startup/readiness/liveness probes
│   ├── 31-app-service.yaml               # ClusterIP service on 8080
│   ├── 32-app-hpa.yaml                   # HPA: 1–5 replicas at 70% CPU
│   ├── 33-app-ingress.yaml               # ingress-nginx rule routing host:80 → app
│   └── 90-ingest-job.yaml                # one-shot Job: migrate + ingest --all into in-cluster Postgres
├── examples/
│   └── terraform-plan-example.json       # 9-resource plan with intentional misconfigurations for demos + tests
├── golden/                                # 22 hand-curated eval cases — each dir has either plan.json (Terraform) or template.yaml (CloudFormation) + expected.json
│   ├── 01-s3-public-acl/                  # single-issue: public ACL bucket
│   ├── 02-s3-no-encryption/               # single-issue: missing SSE config
│   ├── 03-s3-no-versioning/               # single-issue: versioning + MFA delete disabled
│   ├── 04-s3-clean/                       # clean baseline — no findings expected
│   ├── 05-iam-admin-policy/               # single-issue: AdministratorAccess attached
│   ├── 06-iam-wildcard-principal/         # single-issue: Principal:* in assume_role_policy
│   ├── 07-iam-clean/                      # clean baseline
│   ├── 08-db-public-access/               # single-issue: publicly_accessible=true
│   ├── 09-db-unencrypted/                 # single-issue: storage_encrypted=false
│   ├── 10-db-no-deletion-protection/      # single-issue: deletion_protection=false
│   ├── 11-sg-open-ssh/                    # single-issue: 0.0.0.0/0 ingress on port 22
│   ├── 12-sg-clean/                       # clean baseline
│   ├── 13-multi-s3-iam/                   # multi-issue: public S3 + admin IAM
│   ├── 14-multi-mixed/                    # multi-severity: critical + medium + explicitly-clean
│   ├── 15-empty-plan/                     # edge case: no resource_changes
│   ├── 16-only-deletions/                 # edge case: agent skips delete actions
│   ├── 17-only-data-source/               # edge case: agent skips read actions
│   ├── cfn-01-s3-public-acl/              # CFN: S3 bucket with PublicRead access control
│   ├── cfn-02-iam-admin-policy/           # CFN: IAM role attached to AdministratorAccess managed policy
│   ├── cfn-03-sg-open-ssh/                # CFN: security group with SSH ingress from 0.0.0.0/0
│   ├── cfn-04-rds-unencrypted/            # CFN: RDS instance with StorageEncrypted=false
│   └── cfn-05-clean/                      # CFN: clean baseline — encryption + versioning + Block Public Access
├── evals/
│   ├── stub-baseline.md                  # deterministic regression report against the stub LLM
│   └── qwen3-baseline.md                 # real-quality report against ollama:qwen3:latest
└── tests/
    ├── test_chunker.py                   # 8 tests covering chunker + frontmatter strip
    ├── test_embedder.py                  # 3 tests covering embedder protocol + factory
    ├── test_retrieval.py                 # 5 tests covering RRF fusion semantics
    ├── test_plan.py                      # 12 tests covering parser + graph queries
    ├── test_agent.py                     # 14 tests covering schemas, prompts, stub (incl. guidance-scoping regression), graph topology
    ├── test_ollama_client.py             # 5 tests with httpx.MockTransport
    ├── test_eval.py                      # 17 tests covering scorers, golden-set loading, harness, reporter
    ├── test_cis_aws_source.py            # 6 tests covering the local CIS source: file discovery, provenance URLs, license metadata, error paths
    ├── test_http_api.py                  # 6 tests covering /v1/search, /v1/scan, /v1/sources via FastAPI TestClient (service layer monkeypatched)
    └── test_mcp_server.py                # 5 tests covering tool registration, tool plumbing, and JSON-serializable schemas
```

### File-by-file explanation

**Top-level**

- `Dockerfile` — Multi-stage build. Builder stage uses `python:3.12-slim-bookworm` with `uv` to install dependencies into a venv; runtime stage copies just the venv + source into a fresh slim image, drops to a non-root `app` user, and binds to `$PORT` (defaulting to 8080 for local). Final image ~189MB. Cloud Run-compatible without modification.
- `docker-compose.yml` — Single service: `pgvector/pgvector:pg16` on port 5433 with a named volume. Mounts `./db/init/` as `/docker-entrypoint-initdb.d/` so the `vector` extension is created on first boot.
- `pyproject.toml` — Project definition. Declares Python ≥3.12, runtime dependencies (FastAPI, psycopg, pgvector, fastembed, openai, tiktoken, typer, pydantic-settings, mistune, httpx), dev dependencies (ruff, pytest, pre-commit, detect-secrets), the ruff lint config (E, F, W, I, B, UP, S, RUF rule families), the hatchling build backend pointing at `src/iac_security_copilot/`, and the `iac-copilot` console script entry point.
- `uv.lock` — Fully resolved dependency tree pinned to specific versions. Committed so Docker builds can use `uv sync --frozen` for reproducibility.
- `.env.example` — Documents every environment variable the app reads. Copy to `.env` and adjust for local dev; `.env` is gitignored.
- `.pre-commit-config.yaml` — Configures four hook repos: general hygiene (trailing-whitespace, EOF-fixer, YAML/TOML/JSON validation, merge conflict markers, large-file guard), ruff (lint with `--fix` plus format), Yelp's detect-secrets, and hadolint for the Dockerfile.
- `.secrets.baseline` — Detect-secrets baseline file. Empty today; tracks known-safe secret-like strings if any are intentionally committed in the future.

**Database**

- `db/init/01-extensions.sql` — Single statement: `CREATE EXTENSION IF NOT EXISTS vector;`. Runs once on the first container start because Postgres executes any `.sql` file in `/docker-entrypoint-initdb.d/` during init. This avoids a chicken-and-egg problem where the Python client's `register_vector` call would fail against a fresh DB without the extension.

**Source — application core**

- `src/iac_security_copilot/__init__.py` — Just `__version__ = "0.1.0"`. Imported by `main.py` and surfaced via the FastAPI `version` field.
- `src/iac_security_copilot/main.py` — FastAPI app exposing two endpoints: `/health` (returns `{"status": "ok"}`) and `/` (returns name + version). Imported by uvicorn at runtime. Will grow to host the retrieval and agent endpoints in later weeks.
- `src/iac_security_copilot/cli.py` — Typer-based CLI exposing eight commands: `migrate`, `ingest`, `query`, `plan`, `scan`, `eval`, `stats`, `reset`. Registered as the `iac-copilot` console script in `pyproject.toml`.
- `src/iac_security_copilot/config.py` — A single `Settings` class using pydantic-settings to load environment variables (with `.env` file support) into a strongly-typed config object. Defines three enums: `EmbeddingProvider` (`bge-small`, `openai-small`, `openai-large`), `RetrievalMethod` (`dense`, `bm25`, `hybrid`), and `LLMProvider` (`stub`, `anthropic`, `openai`). All other modules import `get_settings()` rather than reading env vars directly.

**Source — database layer**

- `src/iac_security_copilot/db/client.py` — Provides a `connection(database_url)` context manager that opens a psycopg connection and registers the pgvector type adapter. Every caller in the codebase gets vector columns working without thinking about it; this is also the natural seam for adding pooling, retries, or instrumentation later.
- `src/iac_security_copilot/db/migrations.py` — Holds the schema as a Python string template. The `chunks.embedding` column is sized by injecting the active embedder's dimension count (384 for BGE-small, 1536 for OpenAI small, 3072 for OpenAI large) at migration time. Switching embedding providers therefore requires re-running `reset` + `migrate` + `ingest`. Creates two indices: HNSW on `embedding` (fast cosine retrieval) and GIN on a generated `content_tsv tsvector` column (fast BM25 lookups via Postgres FTS).

**Source — ingestion**

- `src/iac_security_copilot/ingestion/embedder.py` — Defines the `Embedder` protocol (`name: str`, `dimensions: int`, `embed(texts) -> list[list[float]]`) and three implementations: `BGESmallEmbedder` (fastembed, ONNX, ~30MB, CPU-friendly, default), `OpenAIEmbedder` (text-embedding-3-small / -large, requires `OPENAI_API_KEY`), and a `get_embedder()` factory that reads `EMBEDDING_PROVIDER` from settings. Swapping providers is a one-env-var change at call sites.
- `src/iac_security_copilot/ingestion/chunker.py` — Heading-aware Markdown chunker. Strips Jekyll/YAML frontmatter (`---\\n…\\n---`) before processing — without this, every OWASP doc started with a `layout: col-sidebar` chunk that dominated BM25 with boilerplate. Then walks the document line by line, tracking the current heading path (h1 → h2 → h3). Each section between headings becomes a chunk if it fits in `max_tokens`; oversized sections slide with `overlap_tokens` of overlap. Each chunk carries its heading path so retrievers can surface where an answer came from. Token counting uses `tiktoken`'s `cl100k_base` encoding for OpenAI/Claude-compatible token math.
- `src/iac_security_copilot/ingestion/pipeline.py` — Orchestrates the full ingestion flow: fetches a source's documents, upserts them into the `documents` table (clearing any existing chunks for re-runs), chunks each document, embeds the chunks in one batch per document, and inserts the chunk rows with their embeddings. Returns an `IngestStats` dataclass with document and chunk counts.
- `src/iac_security_copilot/ingestion/sources/base.py` — Defines the `Source` protocol (just `name: str` and `fetch_all() -> list[FetchedDocument]`) and the `FetchedDocument` dataclass that sources produce. Sources stay narrow — they fetch and identify content; chunking + embedding + persistence is the pipeline's job.
- `src/iac_security_copilot/ingestion/sources/owasp_cicd.py` — Concrete source for OWASP Top 10 CI/CD Security Risks. Holds the list of 10 known markdown files in the OWASP GitHub repository and fetches each via raw `githubusercontent.com` URLs. Used today as the substitute for the (non-existent) "OWASP IaC Top 10" project; CICD-SEC-6/7/8 cover IaC security concerns directly.
- `src/iac_security_copilot/ingestion/sources/cis_aws.py` — CIS AWS Foundations Benchmark source. Reads ten hand-curated control summaries from `corpus/cis-aws/`. Each summary is written for this project (not verbatim CIS material — the official benchmark is licensed CC BY-NC-SA 4.0) and carries the canonical CIS landing URL as the document `url` for citation provenance. Controls picked to overlap with the golden set so eval immediately gets multi-source signal: IAM (1.16, 1.22), S3 (2.1.1, 2.1.2, 2.1.5), RDS (2.3.1, 2.3.2, 2.3.3), security groups (4.1, 4.2).

**Source — retrieval**

- `src/iac_security_copilot/retrieval/base.py` — Defines the `Retriever` protocol (`name: str`, `search(query, top_k) -> list[RetrievedChunk]`) and the `RetrievedChunk` dataclass. Every retriever returns the same shape: chunk content + heading path + document title/source/URL for citations + native score + rank + the method that produced it. The protocol is structural — anything that exposes `name` and `search` satisfies it.
- `src/iac_security_copilot/retrieval/dense.py` — `DenseRetriever`. Embeds the query with the configured `Embedder`, then orders chunks by pgvector cosine distance (`embedding <=> query`). Returns the cosine *similarity* (1 − distance) as the score so 1.0 = identical, 0.0 = orthogonal. Uses the HNSW index from migrations.
- `src/iac_security_copilot/retrieval/bm25.py` — `BM25Retriever`. Uses Postgres' `ts_rank_cd` cover-density ranker against the generated `content_tsv` column. Rewrites `plainto_tsquery` output from `&` to `|` so multi-word natural-language queries don't AND themselves into zero results — recall is the priority at the candidate-pool level; precision is the reranker's job downstream.
- `src/iac_security_copilot/retrieval/hybrid.py` — `HybridRetriever` plus the standalone `reciprocal_rank_fusion()` function. RRF combines dense and BM25 result lists using only rank position (not native score, since cosine and BM25 are on incomparable scales). Standard formula: `score = sum(1 / (k + rank+1))` with `k=60`. Pulls `candidates_per_retriever` from each retriever (default 30), then fuses.
- `src/iac_security_copilot/retrieval/reranker.py` — `CrossEncoderReranker`. Wraps fastembed's `TextCrossEncoder` (`Xenova/ms-marco-MiniLM-L-6-v2` by default — ~80MB ONNX model). Re-scores a candidate set jointly over `(query, passage)` pairs; significantly better top-k precision than bi-encoder retrieval because the model sees both pieces of context together. Lazy-imported in the CLI so users running `--no-rerank` don't pay the model-download cost.

**Source — plan parser**

- `src/iac_security_copilot/plan/base.py` — Provider-agnostic data model: `Resource` (frozen + hashable, suitable for use as a graph node or set member), `ResourceGraph` (with `by_address`, `by_type`, `with_action`, `managed`, `data_sources` query helpers), `Action` enum (`CREATE`, `UPDATE`, `DELETE`, `REPLACE`, `READ`, `NO_OP`), and `ResourceMode` enum (`MANAGED`, `DATA`). The model is intentionally semantic-free — it captures *what resources exist with what attributes*, not what those attributes mean. The same shape is the target output for the Terraform parser today and for planned CloudFormation / Pulumi parsers later.
- `src/iac_security_copilot/plan/terraform.py` — `parse_terraform_plan(plan_json) -> ResourceGraph`. Generic walker over `resource_changes` (the single source of truth — catches deletions, which `planned_values` misses). No per-resource code anywhere; new resource types are picked up automatically. Module path is extracted from the address regex rather than recursing into `planned_values.root_module.child_modules`. Dependency edges deliberately not extracted yet — added once the agent needs them.

**Source — scan agent**

- `src/iac_security_copilot/agent/schemas.py` — `Finding` (resource address + severity + issue + why-it-matters + suggested fix + citations + confidence), `Citation` (document title + URL + heading path + verbatim excerpt), `ResourceAnalysis` (wrapper returned by the LLM per resource), `ScanReport` (aggregate across the plan), and `AgentState` (LangGraph `TypedDict` with `Annotated[list, operator.add]` reducers so parallel branches merge cleanly).
- `src/iac_security_copilot/agent/prompts.py` — `SYSTEM_PROMPT` (fixes the analyst role + guardrails: only flag what the guidance supports, cite verbatim, be conservative on severity) and `build_user_prompt(resource, retrieved_chunks)` (renders the per-resource payload + numbered citation blocks; redacts attributes marked sensitive before they reach the LLM).
- `src/iac_security_copilot/agent/graph.py` — LangGraph topology with three nodes: `load_plan` (parses + filters to actionable resources), `_fan_out` (emits one `Send` per resource for parallel analysis), `analyze_one` (retrieves guidance + calls the LLM + normalises the address). The compiled graph is reusable — invoke it many times with different `plan_json` payloads. `AgentDeps` injects the retriever, LLM client, and `retrieval_top_k`; tests swap these for fakes.
- `src/iac_security_copilot/agent/llm/base.py` — `LLMClient` Protocol (`name`, `analyze(system, user) -> ResourceAnalysis`). Structural typing — anything that exposes `name` and `analyze` satisfies it.
- `src/iac_security_copilot/agent/llm/factory.py` — `get_llm_client(settings)` reads `LLM_PROVIDER` and instantiates Anthropic / OpenAI / Stub. Swapping providers is a one-env-var change.
- `src/iac_security_copilot/agent/llm/anthropic_client.py` — Claude backend. Uses tool-use to enforce the structured `ResourceAnalysis` schema: defines a synthetic `record_resource_analysis` tool whose input schema is the Pydantic JSON schema, then forces the model to call it. More reliable than asking for free-form JSON.
- `src/iac_security_copilot/agent/llm/openai_client.py` — OpenAI backend. Uses `response_format=json_schema` with `strict: true` — same intent as Anthropic's tool-use, different mechanism. Walks the schema once at construction time to add `additionalProperties: false` everywhere strict mode requires it.
- `src/iac_security_copilot/agent/llm/ollama_client.py` — Local model via [Ollama](https://ollama.com). Uses Ollama's `/api/chat` with `format` set to the Pydantic JSON schema (Ollama 0.5+ constrains generation to match) — same structured-output guarantee as Anthropic / OpenAI, running locally with no API key. Default model `qwen3:latest` (~5GB, 8B parameter Qwen 3). CLI auto-serialises the per-resource fan-out for Ollama because local models serve one request at a time per model — parallel branches just queue at Ollama and blow the per-request timeout.
- `src/iac_security_copilot/agent/llm/stub.py` — Deterministic pattern-matching provider. Produces synthetic findings for well-known IaC misconfigurations (public ACL, unencrypted storage, open security groups, wildcard IAM, etc.) by regex-matching the user prompt. Used for local dev without an API key and for the agent's own test suite.

**Source — eval framework**

- `src/iac_security_copilot/eval/schemas.py` — Pydantic models for the eval framework: `ExpectedFinding` (resource address + severity + keyword + citation document expectations), `GoldenCase` (plan + expectations + clean-resource list), `FindingMatch` (one of five outcomes: `true_positive`, `false_negative`, `false_positive`, `wrong_severity`, `missing_citation`), `CaseResult` (per-case aggregate with precision/recall/F1 properties), `EvalReport` (top-level aggregate across the full set).
- `src/iac_security_copilot/eval/golden.py` — `load_case` (reads one `golden/<id>/{plan.json,expected.json}` pair) and `load_golden_set` (loads every case under `golden/`, sorted by id). The id defaults to the directory name; explicit `id` in expected.json wins.
- `src/iac_security_copilot/eval/scorers.py` — `score_case` does resource-centric matching: for each expected finding on a given resource address, greedily match the first actual finding on the same address and judge by severity bucket (with off-by-one tolerance), keyword presence in `issue`/`suggested_fix`, and citation document filter. Leftover actual findings on unexpected addresses are false positives. Severity calibration drift and citation-grounding failures are surfaced as sub-grade outcomes so the report distinguishes "missed the finding entirely" from "found it but mis-classified the severity" from "called it correctly but didn't cite the right document."
- `src/iac_security_copilot/eval/harness.py` — `run_eval` orchestrates the per-case scans, wraps each in try/except so one failure doesn't abort the run, aggregates per-case results into the `EvalReport`, and accepts an optional `on_case_complete` progress callback (the CLI uses it for the per-case OK / F1=N.NN line).
- `src/iac_security_copilot/eval/reporter.py` — `render_report` produces the Markdown that gets committed at `evals/report.md`. Two audiences: top-of-file aggregate + severity breakdown + per-case table (for skimmers ), and a diagnostic section for imperfect cases only (for the project author fixing regressions).

**Tests**

- `tests/test_chunker.py` — Eight tests covering empty input, single-section under budget, heading-path nesting, oversized-section splitting with overlap, token counting, frontmatter stripping (presence + absence), and that the chunker actually strips frontmatter before emitting chunks.
- `tests/test_embedder.py` — Three tests covering the OpenAI embedder's API-key requirement, rejection of unknown OpenAI model names, and protocol shape via a fake embedder.
- `tests/test_retrieval.py` — Five tests covering the `reciprocal_rank_fusion` helper: empty-input handling, that chunks appearing in multiple lists outrank single-list chunks, that fusion ignores native scores in favour of rank position, that `top_k` is respected, and that returned scores are monotonically decreasing.
- `tests/test_plan.py` — Twelve tests covering the Terraform plan parser: empty input, single resource extraction, action collapsing (create+delete → REPLACE), delete uses `before` attributes, data-source classification, module path extraction (including instance-keyed modules), sensitive-attribute flagging, graph query helpers, malformed-entry resilience, fallback for unknown action combinations, and the pure-function helpers.
- `tests/test_agent.py` — Thirteen tests covering schemas (Pydantic round-trip), prompt construction (resource details, citations, redacted sensitive attributes), the stub LLM (public bucket, clean resource, critical open security group), end-to-end graph runs against the example plan, graceful behaviour on empty plans, the agent's address-normalisation of LLM output, action filtering (only CREATE/UPDATE/REPLACE are analysed), and resilience to LLM exceptions.
- `tests/test_ollama_client.py` — Five tests using `httpx.MockTransport` to exercise the Ollama client without needing a running Ollama: schema-conformant response parsing, empty-content error, non-JSON content error, HTTP 5xx error, and the `name` property.
- `tests/test_eval.py` — Seventeen tests covering the eval framework: scorer outcomes (true_positive, false_negative, false_positive, wrong_severity, missing_citation, off-by-one tolerance, multi-finding-same-resource greedy matching, citation grounding), golden-set loading (real cases + missing-file errors + round-trip), harness behaviour (aggregation + per-case error isolation), and reporter output (aggregate table + diagnostic detail section).

## Quickstart

```bash
# 1. Install dependencies
uv sync

# 2. Copy env template (defaults work for local dev)
cp .env.example .env

# 3. Start Postgres + pgvector
docker compose up -d

# 4. Apply schema (dimensions inferred from EMBEDDING_PROVIDER)
uv run iac-copilot migrate

# 5. Ingest every seed source (OWASP CI/CD Top 10 + CIS AWS Foundations).
#    Single-source: `iac-copilot ingest owasp-cicd-top-10` or `cis-aws-foundations`.
uv run iac-copilot ingest --all

# 6. Try a query (downloads the cross-encoder reranker model on first use)
uv run iac-copilot query "how do I prevent credential leaks in CI/CD"

# 7. Scan the example Terraform plan (uses the deterministic stub LLM by default,
#    no API key required). Switch to real Claude / OpenAI / Ollama via LLM_PROVIDER in .env.
uv run iac-copilot scan examples/terraform-plan-example.json

# 8. Run the eval harness against the golden set. Produces evals/report.md.
uv run iac-copilot eval
```

On the first ingest run, fastembed downloads the BGE-small ONNX model (~30MB) and caches it under `fastembed_cache/`. The first `query` similarly downloads the cross-encoder (~80MB) before serving the first result; subsequent calls hit the cache. With both models warm, a typical end-to-end query runs in ~1.5 seconds, a scan of the 9-resource example plan completes in ~2 seconds with the stub LLM, and the 17-case Terraform eval runs in under a second with the stub (the 5 CFN cases are excluded by default — pass `--include-tags cfn` to run them against a real LLM provider).

## Configuration

All runtime config is environment variables (loaded from `.env` if present). See [`.env.example`](.env.example) for the full list.

### Switching embedding providers

The `chunks.embedding` column is sized to match the active embedder's vector dimension. To switch:

```bash
# Set EMBEDDING_PROVIDER in .env, then:
uv run iac-copilot reset      # drops chunks + clears documents
uv run iac-copilot migrate    # recreates chunks with new dimension count
uv run iac-copilot ingest     # re-fetches and re-embeds with the new provider
```

## Managing the local Postgres

The Postgres container declared in `docker-compose.yml` is a long-lived local dev service. Data persists in the named Docker volume `iac-security-copilot_postgres_data` — survives container restarts, Docker Desktop restarts, and machine reboots. It does NOT survive `docker compose down -v` (the `-v` removes named volumes).

### Day-to-day commands

```bash
# Start (or resume) the Postgres container in the background
docker compose up -d

# Check status (running, exited, etc.)
docker compose ps

# Stop without removing — data stays
docker compose stop

# Restart after a stop
docker compose start

# Tear down container + network (keeps the named volume)
docker compose down

# Tear down EVERYTHING including the data volume — only when you want a clean slate
docker compose down -v
```

### After a machine restart

By default the container stays stopped after a Docker Desktop / machine restart. You'll see `Connection refused` against `127.0.0.1:5433` until you run:

```bash
docker compose up -d
```

If you'd rather have it auto-resume, add a `restart` policy to the Postgres service in `docker-compose.yml`:

```yaml
services:
  postgres:
    image: pgvector/pgvector:pg16
    restart: unless-stopped   # auto-restarts on Docker startup; respects explicit `docker compose stop`
    # ...
```

Three policies to choose from:

| Policy | When the container auto-starts | When it stays down |
|---|---|---|
| (none) | Never — manual `up`/`start` only | Anytime it's not running |
| `unless-stopped` | After Docker / machine restart **if** it was running when Docker last shut down | After an explicit `docker compose stop` or `docker stop` |
| `always` | After any restart, regardless of how it stopped | Only after `docker compose down` |

`unless-stopped` is the right default for a local dev DB — quiet during normal use, doesn't fight you when you intentionally stop it.

### Health check

The compose file already declares a healthcheck (`pg_isready`). To see it:

```bash
docker compose ps        # shows "healthy" / "starting" / "unhealthy" in the STATUS column
docker inspect iac-copilot-postgres --format '{{.State.Health.Status}}'
```

If the container is up but the app can't connect, check `docker compose logs postgres` for startup errors.

## CLI reference

The `iac-copilot` command is installed by `uv sync` as a console script. Available subcommands:

```bash
uv run iac-copilot migrate
# Applies the schema using the active embedder's dimension count.
# Idempotent — safe to run repeatedly.

uv run iac-copilot ingest [SOURCE] [--all]
# Fetches all documents from the named source(s), chunks, embeds, and persists.
# Known sources: `owasp-cicd-top-10`, `cis-aws-foundations`.
# `--all` ingests every known source sequentially.
# Re-running an ingest replaces existing chunks for matching
# (source, source_id) pairs, so it's safe to run repeatedly.
# Examples:
#   iac-copilot ingest                          # owasp-cicd-top-10 (default)
#   iac-copilot ingest cis-aws-foundations      # just CIS
#   iac-copilot ingest --all                    # both sources

uv run iac-copilot query "<question>" [--method dense|bm25|hybrid] [--top-k N] \
                                       [--candidates N] [--rerank|--no-rerank]
# Search the corpus and print the top matching chunks with provenance.
# Examples:
#   iac-copilot query "what is poisoned pipeline execution"
#   iac-copilot query "credential hygiene" --method bm25 --top-k 3
#   iac-copilot query "artifact integrity" --method hybrid --no-rerank
# Defaults: --method hybrid, --top-k 5, --candidates 30, --rerank.
# Hybrid pulls `candidates` chunks from each of BM25 and dense, fuses via RRF,
# then optionally reranks with a cross-encoder for top-k precision.

uv run iac-copilot plan <PATH> [--show-attributes]
# Parse an IaC configuration and print a summary of the resource graph —
# resource counts by action and type, plus a per-resource listing.
# Auto-detects the input shape from the path / file extension / content:
#   - *.tf file or directory of *.tf → Terraform HCL source
#   - *.yaml / *.yml / *.template    → CloudFormation YAML template
#   - *.json file with `resource_changes` key  → Terraform plan JSON
#   - *.json file with a `Resources` mapping   → CloudFormation JSON template
# The plan-JSON path sees fully-resolved attributes (post variable +
# module expansion); HCL and CloudFormation source paths see what was
# written — variables, intrinsic functions (`var.x`, `!Ref`, `!Sub`)
# survive as literal strings or their dict equivalents. For full
# resolution, prefer plan JSON.
# Examples:
#   iac-copilot plan examples/terraform-plan-example.json
#   iac-copilot plan path/to/main.tf
#   iac-copilot plan path/to/terraform-directory/
#   iac-copilot plan path/to/stack.yaml
#   iac-copilot plan path/to/cloudformation-template.json

uv run iac-copilot scan <PATH> [--json] [--output FILE]
# Scan an IaC configuration for security issues. Same input shapes as
# `plan` above — Terraform plan JSON, .tf files / directories, or
# CloudFormation YAML / JSON templates. Combines the hybrid retriever
# (BM25 + dense + RRF + rerank) with the parser to drive the LangGraph
# agent. The deterministic stub LLM's regex patterns target Terraform
# conventions; for CloudFormation templates, use a real LLM provider
# (Anthropic / OpenAI / Ollama) — the stub will miss findings the
# real LLMs catch. LLM provider is configured by LLM_PROVIDER:
#   - stub      (default) — deterministic heuristics, no API key needed
#   - anthropic — requires ANTHROPIC_API_KEY (uses tool-use for structure)
#   - openai    — requires OPENAI_API_KEY (uses response_format strict mode)
#   - ollama    — requires Ollama running locally + a model pulled
#                 (e.g. `ollama pull qwen3:latest`). No API key.
# Output is a structured ScanReport with findings, severities, suggested
# fixes, and citations into the corpus. --json prints raw JSON to stdout;
# --output writes it to a file.
# Examples:
#   iac-copilot scan examples/terraform-plan-example.json
#   iac-copilot scan path/to/main.tf
#   iac-copilot scan path/to/terraform-directory/ --output report.json
#   LLM_PROVIDER=ollama iac-copilot scan examples/terraform-plan-example.json

uv run iac-copilot eval [--output FILE] [--json-output FILE] [--golden-root DIR]
# Run the LangGraph scan agent against every case under `golden/` and emit
# a Markdown regression report. Uses the active LLM_PROVIDER, so the same
# command produces comparable reports across stub / Ollama / Anthropic / OpenAI.
# Defaults: --output evals/report.md. Golden cases live under golden/<id>/
# as (plan.json, expected.json) pairs.
# Examples:
#   iac-copilot eval                                    # stub baseline, < 1s
#   LLM_PROVIDER=anthropic iac-copilot eval             # real-quality run
#   iac-copilot eval --json-output evals/report.json    # also emit raw JSON

uv run iac-copilot stats
# Prints document and chunk counts, grouped by embedding provider.

uv run iac-copilot reset
# Drops the chunks table and clears documents. Prompts for confirmation.
# Required when changing embedding provider — see "Switching embedding
# providers" above.
```

## Running the FastAPI service

For local development:

```bash
uv run uvicorn iac_security_copilot.main:app --reload --port 8080
curl http://localhost:8080/health     # → {"status":"ok"}
```

The service exposes three real endpoints on top of `/health`:

| Route | Body | Purpose |
| --- | --- | --- |
| `POST /v1/search` | `{query, top_k}` | Hybrid retrieval over the corpus — same engine the CLI's `query` command uses |
| `POST /v1/scan` | `{plan_json}` | Run the LangGraph scan agent against a Terraform plan JSON |
| `GET /v1/sources` | — | Document + chunk counts grouped by source |

Example:

```bash
curl -s -X POST http://localhost:8080/v1/search \
  -H 'Content-Type: application/json' \
  -d '{"query":"s3 encryption","top_k":3}' | jq

curl -s -X POST http://localhost:8080/v1/scan \
  -H 'Content-Type: application/json' \
  -d "$(jq '{plan_json: .}' examples/terraform-plan-example.json)" | jq '.findings'
```

Via Docker:

```bash
docker build -t iac-security-copilot:0.1.0 .
docker run --rm -p 8080:8080 iac-security-copilot:0.1.0
```

The Docker image is Cloud Run-ready: listens on `$PORT` (defaults to 8080), binds to `0.0.0.0`, runs as a non-root `app` user, and starts in under one second.

## Editor integration (MCP)

The same retriever and scan agent are exposed as an MCP (Model Context Protocol) server, so editor agents like Claude Code and Cursor can call them as tools while reviewing IaC diffs in-editor.

```bash
uv run iac-copilot mcp      # boots a stdio MCP server
```

Three tools are registered: `search_corpus`, `scan_terraform_plan`, `list_sources`. The full editor config recipe (Claude Code `.mcp.json` snippet, Cursor `~/.cursor/mcp.json` snippet, JSON-RPC smoke test) is in [docs/mcp.md](docs/mcp.md).

## Local Kubernetes deploy (kind)

The `k8s/` directory ships a one-command demo that brings the full stack up in a local `kind` cluster — Postgres + pgvector, the FastAPI app behind ingress-nginx, an HPA wired to a real metrics-server, and a one-shot ingest Job that loads the corpus:

```bash
./k8s/up.sh                                           # build + deploy + smoke test
curl http://localhost/v1/sources | jq                 # through the ingress
kubectl -n iac-copilot get pods,svc,hpa,ingress       # full surface
./k8s/down.sh
```

Manifest-by-manifest walkthrough lives in [k8s/README.md](k8s/README.md). The same set of YAML (with a PVC instead of emptyDir and a real LoadBalancer) is what would ship to a managed cluster like GKE Autopilot.

## Cloud deployment

Two end-to-end recipes for running this on Google Cloud:

- [docs/deploy-cloud-run.md](docs/deploy-cloud-run.md) — serverless HTTPS endpoint via Cloud Run + Cloud SQL. Scales to zero, no Kubernetes, one image push and one `gcloud run deploy`. Right tool when you want a cheap always-on demo URL.
- [docs/deploy-gke.md](docs/deploy-gke.md) — full Kubernetes deploy on GKE Autopilot + Cloud SQL using the same manifests as the local `kind` cluster (with Workload Identity for Cloud SQL auth, a `cloud-sql-proxy` sidecar, and a managed LoadBalancer). Right tool when you actually want K8s semantics.

Each recipe has a `gcloud` walkthrough plus a fully-equivalent Terraform module under [`terraform/`](terraform/) — `terraform apply` instead of the shell calls when you'd rather have the infra in code. Both modules pin every upstream image version, generate the DB password with `random_password`, and put the connection string in Secret Manager.

The `Dockerfile`, the agent code, the eval harness, and the MCP server are unchanged in both — only the surrounding infrastructure differs.

## Running the tests

All tests live under `tests/` and use plain pytest.

```bash
# Full suite
uv run pytest

# Verbose, per-test output
uv run pytest -v

# Single file
uv run pytest tests/test_chunker.py

# Single test
uv run pytest tests/test_chunker.py::test_heading_path_tracks_nesting
```

The current suite is 118 tests, runs in under two seconds end-to-end. No external dependencies (no DB, no API calls) — the embedder tests use a fake implementation rather than spinning up BGE, the retrieval tests exercise the RRF fusion logic in isolation, the plan tests use synthetic JSON / HCL / CloudFormation fixtures inline (the HCL parser has 15 tests covering attribute shapes + module skipping; the CFN parser has 19 covering YAML intrinsics + multi-file merge), the agent tests inject a `_StaticRetriever` + the deterministic stub LLM rather than requiring Postgres or any LLM API, the Ollama client tests use `httpx.MockTransport`, the eval tests construct golden cases inline, the CIS source tests run against the committed `corpus/cis-aws/` directory directly, and the HTTP API + MCP server tests monkeypatch the service-layer entrypoints so the route plumbing and tool plumbing can be tested independently of Postgres.

## Development workflow

```bash
# Lint and auto-fix Python
uv run ruff check --fix

# Format Python
uv run ruff format

# Run every pre-commit hook over the whole tree
uv run pre-commit run --all-files
```

### Pre-commit hook installation

The repo is set up to install pre-commit's hook into `.git/hooks/pre-commit`. If your machine has `core.hooksPath` set globally (common when using a shared dotfiles repo), you'll need a per-repo override:

```bash
git config --local core.hooksPath .git/hooks
uv run pre-commit install   # or `cp` the hook into .git/hooks/ manually
```

Subsequent `git commit` will run all hooks: hygiene checks, ruff lint + format, detect-secrets, hadolint on the Dockerfile.

## What's not in the repo yet

- **Hosted demo URL** — the FastAPI service is Cloud Run-ready, but no always-on instance is deployed.
- **CloudFormation / Pulumi / Bicep parsers** — the `ResourceGraph` shape is provider-agnostic so adding one is a parser-only change, but only Terraform plans are wired up.
- **AWS Well-Architected as a source** — listed under planned in the architecture diagram. The OWASP CI/CD and CIS AWS sources already cover the same ground for IaC-specific findings.
- **GitHub Action wrapper** — auto-PR mode that comments findings on PRs touching `*.tf` files. Easy to layer on top of the CLI.

## Sources

- [OWASP Top 10 CI/CD Security Risks](https://github.com/OWASP/www-project-top-10-ci-cd-security-risks) — primary ingestion source for CI/CD-shaped IaC concerns. Substitutes for the (non-existent) "OWASP IaC Top 10".
- [CIS AWS Foundations Benchmark](https://www.cisecurity.org/benchmark/amazon_web_services) — ten hand-curated control summaries under `corpus/cis-aws/`, attributed to the canonical CIS reference (the benchmark itself is licensed CC BY-NC-SA 4.0 so verbatim reproduction is avoided).
- [AWS Well-Architected Framework](https://docs.aws.amazon.com/wellarchitected/latest/framework/welcome.html) — planned as an additional source.
- [pgvector](https://github.com/pgvector/pgvector) — Postgres extension for vector similarity search.
- [fastembed](https://github.com/qdrant/fastembed) — lightweight ONNX-based embedding library (BGE backend + cross-encoder reranker).
- [LangGraph](https://github.com/langchain-ai/langgraph) — the agent topology library used for the scan agent's Send-based parallel fan-out.
- [Model Context Protocol](https://modelcontextprotocol.io/) — the open spec the MCP stdio server implements.

## License

Apache 2.0