Skip to main content
Glama

iac-security-copilot

Ask natural-language questions about a Terraform plan and get cited, evaluated answers grounded in the CIS AWS Foundations Benchmark and OWASP Top 10 CI/CD Security Risks. Run it as a CLI, hit it over HTTP, or expose it to Claude Code / Cursor as an MCP server — same retriever, same scan agent, same citations.

What's in the box:

  • RAG over a security corpus — hybrid retrieval (BM25 + dense + RRF + cross-encoder rerank) over Postgres + pgvector.

  • LangGraph scan agent — parses a Terraform plan JSON, fans out one branch per resource, retrieves matching guidance, asks an LLM (Anthropic / OpenAI / Ollama / a deterministic stub) for a structured finding with citations.

  • MCP stdio serversearch_corpus, scan_terraform_plan, list_sources exposed as tools so editor agents can ground their answers in the same corpus.

  • FastAPI servicePOST /v1/search, POST /v1/scan, GET /v1/sources; same business logic as the CLI and MCP server.

  • Eval harness — 22 hand-curated golden cases (17 Terraform plan JSON + 5 CloudFormation templates), scored with custom resource-centric matching + severity tolerance + keyword grounding + citation correctness. Committed regression reports for every supported LLM.

  • Local Kubernetes demo./k8s/up.sh brings up a kind cluster with Postgres + the FastAPI app behind ingress-nginx + a real HPA in one command.

Latest eval reports

Two reports live side-by-side, each answering a different question:

  • evals/qwen3-baseline.md — real-quality measurement against ollama:qwen3:latest (8B). The headline number when you want to know what the product actually does.

  • evals/stub-baseline.md — deterministic regression check using the stub LLM. F1=1.00 by construction — this baseline gates the framework, not the product. Runs in <1s.

Top-line against the 17 Terraform cases of the 22-case golden set (CFN cases are excluded from the stub baseline because the stub's regex patterns target Terraform conventions — see the --exclude-tags flag in the CLI reference). Corpus is OWASP CI/CD Top 10 + CIS AWS Foundations Benchmark; citations accepted from either framework.

Provider

Precision

Recall

F1

TP

FN

FP

Wrong severity

Notes

ollama:qwen3:latest

1.00

0.90

0.95

9

1

0

3

Real LLM. Surfaced 1 recall miss (DB deletion protection), 1 severity inflation, 2 scorer-pedantry keyword misses. ~15 min sequential run.

stub

1.00

1.00

1.00

13

0

0

0

Pattern-match heuristics aligned to the golden cases by construction. Use for CI regression, not quality measurement.

The qwen3 F1=0.95 is the honest signal. Two of the three wrong_severity entries are word-form issues the scorer is over-strict on (e.g. expected "encrypted" didn't substring-match qwen3's "encryption"); the third is real severity inflation on the versioning case (qwen3 marked it CRITICAL where expected is MEDIUM). The one false negative is qwen3 missing the deletion-protection medium in the multi-issue plan. See evals/qwen3-baseline.md for the per-case detail.

Switching providers is one env var: LLM_PROVIDER=anthropic ANTHROPIC_API_KEY=... iac-copilot eval --output evals/anthropic-baseline.md produces a comparable report for any future model.

Related MCP server: depguard

Architecture

flowchart LR
    %% --- Inputs / external ---
    USER["Engineer / AI client"]
    OWASP[("OWASP CI/CD Top 10<br/>raw.githubusercontent.com")]
    CIS[("CIS AWS Foundations<br/>corpus/cis-aws/ (committed)")]
    WAR[("AWS Well-Architected<br/>(planned)")]
    PLAN["Terraform plan JSON"]

    %% --- CLI entry point ---
    CLI["iac-copilot CLI<br/>(migrate · ingest · query · plan · scan · stats)"]

    %% --- Ingestion path (done) ---
    subgraph Sources["ingestion/sources"]
        OWASP_SRC["OwaspCicdSource"]
        CIS_SRC["CisAwsSource"]
        WAR_SRC["AwsWarSource (planned)"]
    end

    PIPE["pipeline.py<br/>orchestrator"]
    CHUNK["chunker.py<br/>heading-aware Markdown<br/>+ frontmatter strip"]
    EMB["embedder.py<br/>BGE-small / OpenAI factory"]

    %% --- Storage ---
    DB[("Postgres 16 + pgvector<br/>documents · chunks<br/>HNSW + GIN indices")]

    %% --- Retrieval ---
    subgraph Retrieval["retrieval/"]
        BM25["bm25.py<br/>ts_rank_cd (OR-rewritten)"]
        DENSE["dense.py<br/>pgvector cosine"]
        HYBRID["hybrid.py<br/>RRF fusion (k=60)"]
        RERANK["reranker.py<br/>fastembed cross-encoder"]
    end

    %% --- Scan agent ---
    subgraph Scan["plan/ + agent/"]
        PARSER["terraform.py<br/>parse_terraform_plan<br/>→ ResourceGraph"]
        GRAPH_NODE["agent/graph.py<br/>LangGraph<br/>(Send fan-out)"]
        LLM["agent/llm/<br/>Anthropic · OpenAI · Ollama · Stub"]
    end

    REPORT["ScanReport<br/>(findings + citations)"]

    %% --- Additional protocol surfaces ---
    FASTAPI["FastAPI service<br/>/v1/search · /v1/scan · /v1/sources"]
    MCP["MCP stdio server<br/>search_corpus · scan_terraform_plan · list_sources"]

    %% --- Eval harness ---
    GOLD["golden/<br/>22 hand-curated cases<br/>(17 TF plan + 5 CFN)"]
    RAGAS["eval/<br/>scorers + reporter<br/>(Markdown regression report)"]

    %% --- Ingestion edges (solid: done) ---
    USER --> CLI
    OWASP --> OWASP_SRC
    OWASP_SRC --> PIPE
    CIS --> CIS_SRC
    CIS_SRC --> PIPE
    WAR -.-> WAR_SRC
    WAR_SRC -.-> PIPE
    CLI --> PIPE
    PIPE --> CHUNK
    CHUNK --> PIPE
    PIPE --> EMB
    EMB --> PIPE
    PIPE --> DB

    %% --- CLI query path (solid: done) ---
    CLI --> HYBRID
    DB --> BM25
    DB --> DENSE
    BM25 --> HYBRID
    DENSE --> HYBRID
    HYBRID --> RERANK
    RERANK --> USER

    %% --- CLI scan path (solid: done) ---
    PLAN --> CLI
    CLI --> PARSER
    PARSER --> GRAPH_NODE
    GRAPH_NODE --> HYBRID
    GRAPH_NODE --> LLM
    LLM --> GRAPH_NODE
    GRAPH_NODE --> REPORT
    REPORT --> USER

    %% --- FastAPI + MCP entry points ---
    USER --> FASTAPI
    USER --> MCP
    PLAN --> FASTAPI
    FASTAPI --> PARSER
    FASTAPI --> HYBRID
    MCP --> HYBRID
    MCP --> PARSER

    %% --- Eval edges ---
    GOLD --> RAGAS
    GRAPH_NODE --> RAGAS

    %% --- Styling ---
    classDef done fill:#e8f5e9,stroke:#2e7d32,color:#000
    classDef planned fill:#fff3e0,stroke:#ef6c00,color:#000,stroke-dasharray: 4 3
    classDef storage fill:#e3f2fd,stroke:#1565c0,color:#000
    classDef external fill:#fafafa,stroke:#616161,color:#000

    class CLI,OWASP_SRC,CIS_SRC,PIPE,CHUNK,EMB,BM25,DENSE,HYBRID,RERANK,PARSER,GRAPH_NODE,LLM,REPORT,GOLD,RAGAS,FASTAPI,MCP done
    class WAR_SRC planned
    class DB storage
    class USER,OWASP,CIS,WAR,PLAN external

Legend — green nodes and solid edges are implemented. Orange dashed nodes are planned (AWS Well-Architected as an additional source). The blue node is the storage layer; grey nodes are external actors / sources.

Six paths through the system:

  1. Ingestion (iac-copilot ingest): sources → pipeline → chunker + embedder → Postgres.

  2. Query (iac-copilot query): natural-language question → BM25 + Dense retrievers → RRF fusion → cross-encoder rerank → top chunks.

  3. Scan (iac-copilot scan): Terraform plan JSON → parser → LangGraph agent (per-resource parallel branches) → retrieves guidance + reasons with the configured LLM (Anthropic / OpenAI / Ollama / Stub) → structured ScanReport with citations.

  4. Eval (iac-copilot eval): runs the scan agent against every hand-curated golden/ case, scores each finding for resource match + severity bucket + keyword grounding + citation correctness, and emits a Markdown regression report.

  5. HTTP API (POST /v1/search, POST /v1/scan, GET /v1/sources): same retriever and scan agent reachable over HTTP, suitable for a deployed service.

  6. MCP server (iac-copilot mcp): same retriever and scan agent reachable as stdio MCP tools, suitable for Claude Code / Cursor.

The retrieval subgraph is the seam — the CLI's query command calls into it directly; the scan agent's per-resource branches call into the same retrievers; the FastAPI and MCP entry points wrap the same service.py factories so behaviour stays identical across protocols.

How a scan composes

The Mermaid diagram above shows the system at one altitude. Zooming into the iac-copilot scan path specifically:

Terraform plan JSON
    ↓  (parser — generic walker over the plan JSON, no per-resource code)
ResourceGraph
    ↓  (LangGraph agent — Send-based parallel fan-out)
    │
    │  for each create/update/replace resource:
    │      ↓  retrieve guidance         (hybrid retriever:
    │      │                             BM25 + Dense → RRF → cross-encoder rerank)
    │      ↓  analyze with LLM          (Anthropic via tool-use /
    │      │                             OpenAI via response_format strict /
    │      │                             Ollama via format=json-schema /
    │      │                             Stub via deterministic heuristics)
    │      ↓  structured Finding        (Pydantic-validated — severity, issue,
    │                                    why-it-matters, fix, citations, confidence)
    │
    ↓  (merge via Annotated[list, operator.add] reducers — N branches → 1 state)
ScanReport (JSON via --output / --json, or human-readable with severity colours)

Each box has a single owner module: the parser doesn't know about retrieval, the retriever doesn't know about LLMs, the agent doesn't know about Postgres. Adding a new input format (CloudFormation, Pulumi) means writing a new parser that produces the same ResourceGraph shape — nothing else changes. Swapping LLM providers is one env var.

Repository structure

iac-security-copilot/
├── Dockerfile                            # multi-stage build (uv + slim runtime, non-root, Cloud Run-ready)
├── docker-compose.yml                    # local Postgres 16 + pgvector
├── pyproject.toml                        # project metadata, deps, ruff config, CLI entry point
├── uv.lock                               # locked dependency graph (committed; used by --frozen builds)
├── .env.example                          # template for runtime config (copy to .env)
├── .gitignore                            # excludes .venv, .env, corpus caches, fastembed_cache
├── .dockerignore                         # keeps build context lean
├── .pre-commit-config.yaml               # hooks: ruff, detect-secrets, hadolint, hygiene checks
├── .secrets.baseline                     # detect-secrets allowlist (empty for now)
├── db/
│   └── init/
│       └── 01-extensions.sql             # CREATE EXTENSION vector; runs on first Postgres start
├── src/
│   └── iac_security_copilot/
│       ├── __init__.py                   # version constant
│       ├── main.py                       # FastAPI app — /health + /v1/search + /v1/scan + /v1/sources
│       ├── service.py                    # shared retriever + scan agent factory used by FastAPI and the MCP server
│       ├── mcp_server.py                 # MCP (stdio) server exposing search_corpus, scan_terraform_plan, list_sources
│       ├── cli.py                        # `iac-copilot` command surface (migrate/ingest/query/plan/scan/eval/mcp/stats/reset)
│       ├── config.py                     # pydantic-settings: env vars → typed Settings object
│       ├── db/
│       │   ├── client.py                 # context-managed psycopg connection with pgvector adapter registered
│       │   └── migrations.py             # schema (documents, chunks) parameterised by embedding dimension; generated tsvector column
│       ├── ingestion/
│       │   ├── embedder.py               # Embedder protocol + BGE-small + OpenAI implementations + factory
│       │   ├── chunker.py                # heading-aware Markdown chunker + Jekyll frontmatter stripper
│       │   ├── pipeline.py               # orchestration: fetch → chunk → embed → persist
│       │   └── sources/
│       │       ├── base.py               # Source protocol + FetchedDocument dataclass
│       │       ├── owasp_cicd.py         # OWASP Top 10 CI/CD Security Risks fetcher (GitHub raw)
│       │       └── cis_aws.py            # CIS AWS Foundations Benchmark — reads from corpus/cis-aws/ with provenance URLs
│       ├── retrieval/
│       │   ├── base.py                   # Retriever protocol + RetrievedChunk dataclass
│       │   ├── dense.py                  # pgvector cosine similarity via HNSW
│       │   ├── bm25.py                   # Postgres FTS (ts_rank_cd) with OR-rewritten plainto_tsquery
│       │   ├── hybrid.py                 # Reciprocal Rank Fusion across dense + BM25
│       │   └── reranker.py               # fastembed cross-encoder (Xenova/ms-marco-MiniLM-L-6-v2)
│       ├── plan/
│       │   ├── base.py                   # Resource, ResourceGraph, Action, ResourceMode — provider-agnostic
│       │   └── terraform.py              # parse_terraform_plan — generic walker over plan JSON, no per-resource code
│       ├── agent/
│       │   ├── schemas.py                # Finding, Citation, ResourceAnalysis, ScanReport, AgentState (TypedDict + reducer)
│       │   ├── prompts.py                # SYSTEM_PROMPT + build_user_prompt (redacts sensitive attributes)
│       │   ├── graph.py                  # LangGraph topology: load_plan → Send-based fan-out → analyze_one
│       │   └── llm/
│       │       ├── base.py               # LLMClient Protocol
│       │       ├── factory.py            # get_llm_client(settings)
│       │       ├── anthropic_client.py   # Claude via tool-use (structured output enforcement)
│       │       ├── openai_client.py      # OpenAI via response_format=json_schema strict
│       │       ├── ollama_client.py      # Local model via Ollama; format=schema for structured output
│       │       └── stub.py               # Deterministic heuristic provider (no API key needed)
│       └── eval/
│           ├── schemas.py                # GoldenCase, ExpectedFinding, FindingMatch, CaseResult, EvalReport
│           ├── golden.py                 # load_case + load_golden_set (reads golden/<id>/{plan,expected}.json)
│           ├── scorers.py                # score_case — resource-centric matching with severity tolerance + keyword grounding
│           ├── harness.py                # run_eval — orchestrates per-case scans with error isolation
│           └── reporter.py               # render_report — Markdown with aggregate + severity breakdown + per-case + diagnostic
├── corpus/
│   └── cis-aws/                           # 10 CIS AWS Foundations Benchmark control summaries (markdown)
│       ├── CIS-1.16-iam-admin-policy.md   # IAM: no full administrative privileges
│       ├── CIS-1.22-iam-wildcard-trust.md # IAM: no wildcard principals in trust policies
│       ├── CIS-2.1.1-s3-encryption.md     # S3: server-side encryption at rest
│       ├── CIS-2.1.2-s3-versioning.md     # S3: versioning + MFA delete
│       ├── CIS-2.1.5-s3-public-access.md  # S3: Block Public Access settings
│       ├── CIS-2.3.1-rds-encryption.md    # RDS: storage encryption
│       ├── CIS-2.3.2-rds-deletion-protection.md  # RDS: deletion protection
│       ├── CIS-2.3.3-rds-public-access.md # RDS: publicly_accessible = false
│       ├── CIS-4.1-sg-ssh-from-internet.md   # SG: no 0.0.0.0/0 on port 22
│       └── CIS-4.2-sg-rdp-from-internet.md   # SG: no 0.0.0.0/0 on port 3389
├── docs/
│   └── mcp.md                            # editor integration recipe — Claude Code + Cursor `.mcp.json` snippets
├── k8s/                                  # local Kubernetes demo via kind
│   ├── README.md                         # one-command bring-up + manifest walkthrough
│   ├── up.sh / down.sh                   # idempotent bring-up + tear-down scripts
│   ├── kind-cluster.yaml                 # kind config — single node + ingress port mapping
│   ├── 00-namespace.yaml                 # iac-copilot namespace
│   ├── 10-configmap.yaml                 # non-secret env: DATABASE_URL, LLM_PROVIDER, retrieval knobs
│   ├── 11-secret.example.yaml            # template; real Secret is created by up.sh from host env vars
│   ├── 20-postgres.yaml                  # pgvector/pgvector:pg16 Deployment + Service (emptyDir)
│   ├── 30-app-deployment.yaml            # FastAPI Deployment with startup/readiness/liveness probes
│   ├── 31-app-service.yaml               # ClusterIP service on 8080
│   ├── 32-app-hpa.yaml                   # HPA: 1–5 replicas at 70% CPU
│   ├── 33-app-ingress.yaml               # ingress-nginx rule routing host:80 → app
│   └── 90-ingest-job.yaml                # one-shot Job: migrate + ingest --all into in-cluster Postgres
├── examples/
│   └── terraform-plan-example.json       # 9-resource plan with intentional misconfigurations for demos + tests
├── golden/                                # 22 hand-curated eval cases — each dir has either plan.json (Terraform) or template.yaml (CloudFormation) + expected.json
│   ├── 01-s3-public-acl/                  # single-issue: public ACL bucket
│   ├── 02-s3-no-encryption/               # single-issue: missing SSE config
│   ├── 03-s3-no-versioning/               # single-issue: versioning + MFA delete disabled
│   ├── 04-s3-clean/                       # clean baseline — no findings expected
│   ├── 05-iam-admin-policy/               # single-issue: AdministratorAccess attached
│   ├── 06-iam-wildcard-principal/         # single-issue: Principal:* in assume_role_policy
│   ├── 07-iam-clean/                      # clean baseline
│   ├── 08-db-public-access/               # single-issue: publicly_accessible=true
│   ├── 09-db-unencrypted/                 # single-issue: storage_encrypted=false
│   ├── 10-db-no-deletion-protection/      # single-issue: deletion_protection=false
│   ├── 11-sg-open-ssh/                    # single-issue: 0.0.0.0/0 ingress on port 22
│   ├── 12-sg-clean/                       # clean baseline
│   ├── 13-multi-s3-iam/                   # multi-issue: public S3 + admin IAM
│   ├── 14-multi-mixed/                    # multi-severity: critical + medium + explicitly-clean
│   ├── 15-empty-plan/                     # edge case: no resource_changes
│   ├── 16-only-deletions/                 # edge case: agent skips delete actions
│   ├── 17-only-data-source/               # edge case: agent skips read actions
│   ├── cfn-01-s3-public-acl/              # CFN: S3 bucket with PublicRead access control
│   ├── cfn-02-iam-admin-policy/           # CFN: IAM role attached to AdministratorAccess managed policy
│   ├── cfn-03-sg-open-ssh/                # CFN: security group with SSH ingress from 0.0.0.0/0
│   ├── cfn-04-rds-unencrypted/            # CFN: RDS instance with StorageEncrypted=false
│   └── cfn-05-clean/                      # CFN: clean baseline — encryption + versioning + Block Public Access
├── evals/
│   ├── stub-baseline.md                  # deterministic regression report against the stub LLM
│   └── qwen3-baseline.md                 # real-quality report against ollama:qwen3:latest
└── tests/
    ├── test_chunker.py                   # 8 tests covering chunker + frontmatter strip
    ├── test_embedder.py                  # 3 tests covering embedder protocol + factory
    ├── test_retrieval.py                 # 5 tests covering RRF fusion semantics
    ├── test_plan.py                      # 12 tests covering parser + graph queries
    ├── test_agent.py                     # 14 tests covering schemas, prompts, stub (incl. guidance-scoping regression), graph topology
    ├── test_ollama_client.py             # 5 tests with httpx.MockTransport
    ├── test_eval.py                      # 17 tests covering scorers, golden-set loading, harness, reporter
    ├── test_cis_aws_source.py            # 6 tests covering the local CIS source: file discovery, provenance URLs, license metadata, error paths
    ├── test_http_api.py                  # 6 tests covering /v1/search, /v1/scan, /v1/sources via FastAPI TestClient (service layer monkeypatched)
    └── test_mcp_server.py                # 5 tests covering tool registration, tool plumbing, and JSON-serializable schemas

File-by-file explanation

Top-level

  • Dockerfile — Multi-stage build. Builder stage uses python:3.12-slim-bookworm with uv to install dependencies into a venv; runtime stage copies just the venv + source into a fresh slim image, drops to a non-root app user, and binds to $PORT (defaulting to 8080 for local). Final image ~189MB. Cloud Run-compatible without modification.

  • docker-compose.yml — Single service: pgvector/pgvector:pg16 on port 5433 with a named volume. Mounts ./db/init/ as /docker-entrypoint-initdb.d/ so the vector extension is created on first boot.

  • pyproject.toml — Project definition. Declares Python ≥3.12, runtime dependencies (FastAPI, psycopg, pgvector, fastembed, openai, tiktoken, typer, pydantic-settings, mistune, httpx), dev dependencies (ruff, pytest, pre-commit, detect-secrets), the ruff lint config (E, F, W, I, B, UP, S, RUF rule families), the hatchling build backend pointing at src/iac_security_copilot/, and the iac-copilot console script entry point.

  • uv.lock — Fully resolved dependency tree pinned to specific versions. Committed so Docker builds can use uv sync --frozen for reproducibility.

  • .env.example — Documents every environment variable the app reads. Copy to .env and adjust for local dev; .env is gitignored.

  • .pre-commit-config.yaml — Configures four hook repos: general hygiene (trailing-whitespace, EOF-fixer, YAML/TOML/JSON validation, merge conflict markers, large-file guard), ruff (lint with --fix plus format), Yelp's detect-secrets, and hadolint for the Dockerfile.

  • .secrets.baseline — Detect-secrets baseline file. Empty today; tracks known-safe secret-like strings if any are intentionally committed in the future.

Database

  • db/init/01-extensions.sql — Single statement: CREATE EXTENSION IF NOT EXISTS vector;. Runs once on the first container start because Postgres executes any .sql file in /docker-entrypoint-initdb.d/ during init. This avoids a chicken-and-egg problem where the Python client's register_vector call would fail against a fresh DB without the extension.

Source — application core

  • src/iac_security_copilot/__init__.py — Just __version__ = "0.1.0". Imported by main.py and surfaced via the FastAPI version field.

  • src/iac_security_copilot/main.py — FastAPI app exposing two endpoints: /health (returns {"status": "ok"}) and / (returns name + version). Imported by uvicorn at runtime. Will grow to host the retrieval and agent endpoints in later weeks.

  • src/iac_security_copilot/cli.py — Typer-based CLI exposing eight commands: migrate, ingest, query, plan, scan, eval, stats, reset. Registered as the iac-copilot console script in pyproject.toml.

  • src/iac_security_copilot/config.py — A single Settings class using pydantic-settings to load environment variables (with .env file support) into a strongly-typed config object. Defines three enums: EmbeddingProvider (bge-small, openai-small, openai-large), RetrievalMethod (dense, bm25, hybrid), and LLMProvider (stub, anthropic, openai). All other modules import get_settings() rather than reading env vars directly.

Source — database layer

  • src/iac_security_copilot/db/client.py — Provides a connection(database_url) context manager that opens a psycopg connection and registers the pgvector type adapter. Every caller in the codebase gets vector columns working without thinking about it; this is also the natural seam for adding pooling, retries, or instrumentation later.

  • src/iac_security_copilot/db/migrations.py — Holds the schema as a Python string template. The chunks.embedding column is sized by injecting the active embedder's dimension count (384 for BGE-small, 1536 for OpenAI small, 3072 for OpenAI large) at migration time. Switching embedding providers therefore requires re-running reset + migrate + ingest. Creates two indices: HNSW on embedding (fast cosine retrieval) and GIN on a generated content_tsv tsvector column (fast BM25 lookups via Postgres FTS).

Source — ingestion

  • src/iac_security_copilot/ingestion/embedder.py — Defines the Embedder protocol (name: str, dimensions: int, embed(texts) -> list[list[float]]) and three implementations: BGESmallEmbedder (fastembed, ONNX, ~30MB, CPU-friendly, default), OpenAIEmbedder (text-embedding-3-small / -large, requires OPENAI_API_KEY), and a get_embedder() factory that reads EMBEDDING_PROVIDER from settings. Swapping providers is a one-env-var change at call sites.

  • src/iac_security_copilot/ingestion/chunker.py — Heading-aware Markdown chunker. Strips Jekyll/YAML frontmatter (---\\n…\\n---) before processing — without this, every OWASP doc started with a layout: col-sidebar chunk that dominated BM25 with boilerplate. Then walks the document line by line, tracking the current heading path (h1 → h2 → h3). Each section between headings becomes a chunk if it fits in max_tokens; oversized sections slide with overlap_tokens of overlap. Each chunk carries its heading path so retrievers can surface where an answer came from. Token counting uses tiktoken's cl100k_base encoding for OpenAI/Claude-compatible token math.

  • src/iac_security_copilot/ingestion/pipeline.py — Orchestrates the full ingestion flow: fetches a source's documents, upserts them into the documents table (clearing any existing chunks for re-runs), chunks each document, embeds the chunks in one batch per document, and inserts the chunk rows with their embeddings. Returns an IngestStats dataclass with document and chunk counts.

  • src/iac_security_copilot/ingestion/sources/base.py — Defines the Source protocol (just name: str and fetch_all() -> list[FetchedDocument]) and the FetchedDocument dataclass that sources produce. Sources stay narrow — they fetch and identify content; chunking + embedding + persistence is the pipeline's job.

  • src/iac_security_copilot/ingestion/sources/owasp_cicd.py — Concrete source for OWASP Top 10 CI/CD Security Risks. Holds the list of 10 known markdown files in the OWASP GitHub repository and fetches each via raw githubusercontent.com URLs. Used today as the substitute for the (non-existent) "OWASP IaC Top 10" project; CICD-SEC-6/7/8 cover IaC security concerns directly.

  • src/iac_security_copilot/ingestion/sources/cis_aws.py — CIS AWS Foundations Benchmark source. Reads ten hand-curated control summaries from corpus/cis-aws/. Each summary is written for this project (not verbatim CIS material — the official benchmark is licensed CC BY-NC-SA 4.0) and carries the canonical CIS landing URL as the document url for citation provenance. Controls picked to overlap with the golden set so eval immediately gets multi-source signal: IAM (1.16, 1.22), S3 (2.1.1, 2.1.2, 2.1.5), RDS (2.3.1, 2.3.2, 2.3.3), security groups (4.1, 4.2).

Source — retrieval

  • src/iac_security_copilot/retrieval/base.py — Defines the Retriever protocol (name: str, search(query, top_k) -> list[RetrievedChunk]) and the RetrievedChunk dataclass. Every retriever returns the same shape: chunk content + heading path + document title/source/URL for citations + native score + rank + the method that produced it. The protocol is structural — anything that exposes name and search satisfies it.

  • src/iac_security_copilot/retrieval/dense.pyDenseRetriever. Embeds the query with the configured Embedder, then orders chunks by pgvector cosine distance (embedding <=> query). Returns the cosine similarity (1 − distance) as the score so 1.0 = identical, 0.0 = orthogonal. Uses the HNSW index from migrations.

  • src/iac_security_copilot/retrieval/bm25.pyBM25Retriever. Uses Postgres' ts_rank_cd cover-density ranker against the generated content_tsv column. Rewrites plainto_tsquery output from & to | so multi-word natural-language queries don't AND themselves into zero results — recall is the priority at the candidate-pool level; precision is the reranker's job downstream.

  • src/iac_security_copilot/retrieval/hybrid.pyHybridRetriever plus the standalone reciprocal_rank_fusion() function. RRF combines dense and BM25 result lists using only rank position (not native score, since cosine and BM25 are on incomparable scales). Standard formula: score = sum(1 / (k + rank+1)) with k=60. Pulls candidates_per_retriever from each retriever (default 30), then fuses.

  • src/iac_security_copilot/retrieval/reranker.pyCrossEncoderReranker. Wraps fastembed's TextCrossEncoder (Xenova/ms-marco-MiniLM-L-6-v2 by default — ~80MB ONNX model). Re-scores a candidate set jointly over (query, passage) pairs; significantly better top-k precision than bi-encoder retrieval because the model sees both pieces of context together. Lazy-imported in the CLI so users running --no-rerank don't pay the model-download cost.

Source — plan parser

  • src/iac_security_copilot/plan/base.py — Provider-agnostic data model: Resource (frozen + hashable, suitable for use as a graph node or set member), ResourceGraph (with by_address, by_type, with_action, managed, data_sources query helpers), Action enum (CREATE, UPDATE, DELETE, REPLACE, READ, NO_OP), and ResourceMode enum (MANAGED, DATA). The model is intentionally semantic-free — it captures what resources exist with what attributes, not what those attributes mean. The same shape is the target output for the Terraform parser today and for planned CloudFormation / Pulumi parsers later.

  • src/iac_security_copilot/plan/terraform.pyparse_terraform_plan(plan_json) -> ResourceGraph. Generic walker over resource_changes (the single source of truth — catches deletions, which planned_values misses). No per-resource code anywhere; new resource types are picked up automatically. Module path is extracted from the address regex rather than recursing into planned_values.root_module.child_modules. Dependency edges deliberately not extracted yet — added once the agent needs them.

Source — scan agent

  • src/iac_security_copilot/agent/schemas.pyFinding (resource address + severity + issue + why-it-matters + suggested fix + citations + confidence), Citation (document title + URL + heading path + verbatim excerpt), ResourceAnalysis (wrapper returned by the LLM per resource), ScanReport (aggregate across the plan), and AgentState (LangGraph TypedDict with Annotated[list, operator.add] reducers so parallel branches merge cleanly).

  • src/iac_security_copilot/agent/prompts.pySYSTEM_PROMPT (fixes the analyst role + guardrails: only flag what the guidance supports, cite verbatim, be conservative on severity) and build_user_prompt(resource, retrieved_chunks) (renders the per-resource payload + numbered citation blocks; redacts attributes marked sensitive before they reach the LLM).

  • src/iac_security_copilot/agent/graph.py — LangGraph topology with three nodes: load_plan (parses + filters to actionable resources), _fan_out (emits one Send per resource for parallel analysis), analyze_one (retrieves guidance + calls the LLM + normalises the address). The compiled graph is reusable — invoke it many times with different plan_json payloads. AgentDeps injects the retriever, LLM client, and retrieval_top_k; tests swap these for fakes.

  • src/iac_security_copilot/agent/llm/base.pyLLMClient Protocol (name, analyze(system, user) -> ResourceAnalysis). Structural typing — anything that exposes name and analyze satisfies it.

  • src/iac_security_copilot/agent/llm/factory.pyget_llm_client(settings) reads LLM_PROVIDER and instantiates Anthropic / OpenAI / Stub. Swapping providers is a one-env-var change.

  • src/iac_security_copilot/agent/llm/anthropic_client.py — Claude backend. Uses tool-use to enforce the structured ResourceAnalysis schema: defines a synthetic record_resource_analysis tool whose input schema is the Pydantic JSON schema, then forces the model to call it. More reliable than asking for free-form JSON.

  • src/iac_security_copilot/agent/llm/openai_client.py — OpenAI backend. Uses response_format=json_schema with strict: true — same intent as Anthropic's tool-use, different mechanism. Walks the schema once at construction time to add additionalProperties: false everywhere strict mode requires it.

  • src/iac_security_copilot/agent/llm/ollama_client.py — Local model via Ollama. Uses Ollama's /api/chat with format set to the Pydantic JSON schema (Ollama 0.5+ constrains generation to match) — same structured-output guarantee as Anthropic / OpenAI, running locally with no API key. Default model qwen3:latest (~5GB, 8B parameter Qwen 3). CLI auto-serialises the per-resource fan-out for Ollama because local models serve one request at a time per model — parallel branches just queue at Ollama and blow the per-request timeout.

  • src/iac_security_copilot/agent/llm/stub.py — Deterministic pattern-matching provider. Produces synthetic findings for well-known IaC misconfigurations (public ACL, unencrypted storage, open security groups, wildcard IAM, etc.) by regex-matching the user prompt. Used for local dev without an API key and for the agent's own test suite.

Source — eval framework

  • src/iac_security_copilot/eval/schemas.py — Pydantic models for the eval framework: ExpectedFinding (resource address + severity + keyword + citation document expectations), GoldenCase (plan + expectations + clean-resource list), FindingMatch (one of five outcomes: true_positive, false_negative, false_positive, wrong_severity, missing_citation), CaseResult (per-case aggregate with precision/recall/F1 properties), EvalReport (top-level aggregate across the full set).

  • src/iac_security_copilot/eval/golden.pyload_case (reads one golden/<id>/{plan.json,expected.json} pair) and load_golden_set (loads every case under golden/, sorted by id). The id defaults to the directory name; explicit id in expected.json wins.

  • src/iac_security_copilot/eval/scorers.pyscore_case does resource-centric matching: for each expected finding on a given resource address, greedily match the first actual finding on the same address and judge by severity bucket (with off-by-one tolerance), keyword presence in issue/suggested_fix, and citation document filter. Leftover actual findings on unexpected addresses are false positives. Severity calibration drift and citation-grounding failures are surfaced as sub-grade outcomes so the report distinguishes "missed the finding entirely" from "found it but mis-classified the severity" from "called it correctly but didn't cite the right document."

  • src/iac_security_copilot/eval/harness.pyrun_eval orchestrates the per-case scans, wraps each in try/except so one failure doesn't abort the run, aggregates per-case results into the EvalReport, and accepts an optional on_case_complete progress callback (the CLI uses it for the per-case OK / F1=N.NN line).

  • src/iac_security_copilot/eval/reporter.pyrender_report produces the Markdown that gets committed at evals/report.md. Two audiences: top-of-file aggregate + severity breakdown + per-case table (for skimmers ), and a diagnostic section for imperfect cases only (for the project author fixing regressions).

Tests

  • tests/test_chunker.py — Eight tests covering empty input, single-section under budget, heading-path nesting, oversized-section splitting with overlap, token counting, frontmatter stripping (presence + absence), and that the chunker actually strips frontmatter before emitting chunks.

  • tests/test_embedder.py — Three tests covering the OpenAI embedder's API-key requirement, rejection of unknown OpenAI model names, and protocol shape via a fake embedder.

  • tests/test_retrieval.py — Five tests covering the reciprocal_rank_fusion helper: empty-input handling, that chunks appearing in multiple lists outrank single-list chunks, that fusion ignores native scores in favour of rank position, that top_k is respected, and that returned scores are monotonically decreasing.

  • tests/test_plan.py — Twelve tests covering the Terraform plan parser: empty input, single resource extraction, action collapsing (create+delete → REPLACE), delete uses before attributes, data-source classification, module path extraction (including instance-keyed modules), sensitive-attribute flagging, graph query helpers, malformed-entry resilience, fallback for unknown action combinations, and the pure-function helpers.

  • tests/test_agent.py — Thirteen tests covering schemas (Pydantic round-trip), prompt construction (resource details, citations, redacted sensitive attributes), the stub LLM (public bucket, clean resource, critical open security group), end-to-end graph runs against the example plan, graceful behaviour on empty plans, the agent's address-normalisation of LLM output, action filtering (only CREATE/UPDATE/REPLACE are analysed), and resilience to LLM exceptions.

  • tests/test_ollama_client.py — Five tests using httpx.MockTransport to exercise the Ollama client without needing a running Ollama: schema-conformant response parsing, empty-content error, non-JSON content error, HTTP 5xx error, and the name property.

  • tests/test_eval.py — Seventeen tests covering the eval framework: scorer outcomes (true_positive, false_negative, false_positive, wrong_severity, missing_citation, off-by-one tolerance, multi-finding-same-resource greedy matching, citation grounding), golden-set loading (real cases + missing-file errors + round-trip), harness behaviour (aggregation + per-case error isolation), and reporter output (aggregate table + diagnostic detail section).

Quickstart

# 1. Install dependencies
uv sync

# 2. Copy env template (defaults work for local dev)
cp .env.example .env

# 3. Start Postgres + pgvector
docker compose up -d

# 4. Apply schema (dimensions inferred from EMBEDDING_PROVIDER)
uv run iac-copilot migrate

# 5. Ingest every seed source (OWASP CI/CD Top 10 + CIS AWS Foundations).
#    Single-source: `iac-copilot ingest owasp-cicd-top-10` or `cis-aws-foundations`.
uv run iac-copilot ingest --all

# 6. Try a query (downloads the cross-encoder reranker model on first use)
uv run iac-copilot query "how do I prevent credential leaks in CI/CD"

# 7. Scan the example Terraform plan (uses the deterministic stub LLM by default,
#    no API key required). Switch to real Claude / OpenAI / Ollama via LLM_PROVIDER in .env.
uv run iac-copilot scan examples/terraform-plan-example.json

# 8. Run the eval harness against the golden set. Produces evals/report.md.
uv run iac-copilot eval

On the first ingest run, fastembed downloads the BGE-small ONNX model (~30MB) and caches it under fastembed_cache/. The first query similarly downloads the cross-encoder (~80MB) before serving the first result; subsequent calls hit the cache. With both models warm, a typical end-to-end query runs in ~1.5 seconds, a scan of the 9-resource example plan completes in ~2 seconds with the stub LLM, and the 17-case Terraform eval runs in under a second with the stub (the 5 CFN cases are excluded by default — pass --include-tags cfn to run them against a real LLM provider).

Configuration

All runtime config is environment variables (loaded from .env if present). See .env.example for the full list.

Switching embedding providers

The chunks.embedding column is sized to match the active embedder's vector dimension. To switch:

# Set EMBEDDING_PROVIDER in .env, then:
uv run iac-copilot reset      # drops chunks + clears documents
uv run iac-copilot migrate    # recreates chunks with new dimension count
uv run iac-copilot ingest     # re-fetches and re-embeds with the new provider

Managing the local Postgres

The Postgres container declared in docker-compose.yml is a long-lived local dev service. Data persists in the named Docker volume iac-security-copilot_postgres_data — survives container restarts, Docker Desktop restarts, and machine reboots. It does NOT survive docker compose down -v (the -v removes named volumes).

Day-to-day commands

# Start (or resume) the Postgres container in the background
docker compose up -d

# Check status (running, exited, etc.)
docker compose ps

# Stop without removing — data stays
docker compose stop

# Restart after a stop
docker compose start

# Tear down container + network (keeps the named volume)
docker compose down

# Tear down EVERYTHING including the data volume — only when you want a clean slate
docker compose down -v

After a machine restart

By default the container stays stopped after a Docker Desktop / machine restart. You'll see Connection refused against 127.0.0.1:5433 until you run:

docker compose up -d

If you'd rather have it auto-resume, add a restart policy to the Postgres service in docker-compose.yml:

services:
  postgres:
    image: pgvector/pgvector:pg16
    restart: unless-stopped   # auto-restarts on Docker startup; respects explicit `docker compose stop`
    # ...

Three policies to choose from:

Policy

When the container auto-starts

When it stays down

(none)

Never — manual up/start only

Anytime it's not running

unless-stopped

After Docker / machine restart if it was running when Docker last shut down

After an explicit docker compose stop or docker stop

always

After any restart, regardless of how it stopped

Only after docker compose down

unless-stopped is the right default for a local dev DB — quiet during normal use, doesn't fight you when you intentionally stop it.

Health check

The compose file already declares a healthcheck (pg_isready). To see it:

docker compose ps        # shows "healthy" / "starting" / "unhealthy" in the STATUS column
docker inspect iac-copilot-postgres --format '{{.State.Health.Status}}'

If the container is up but the app can't connect, check docker compose logs postgres for startup errors.

CLI reference

The iac-copilot command is installed by uv sync as a console script. Available subcommands:

uv run iac-copilot migrate
# Applies the schema using the active embedder's dimension count.
# Idempotent — safe to run repeatedly.

uv run iac-copilot ingest [SOURCE] [--all]
# Fetches all documents from the named source(s), chunks, embeds, and persists.
# Known sources: `owasp-cicd-top-10`, `cis-aws-foundations`.
# `--all` ingests every known source sequentially.
# Re-running an ingest replaces existing chunks for matching
# (source, source_id) pairs, so it's safe to run repeatedly.
# Examples:
#   iac-copilot ingest                          # owasp-cicd-top-10 (default)
#   iac-copilot ingest cis-aws-foundations      # just CIS
#   iac-copilot ingest --all                    # both sources

uv run iac-copilot query "<question>" [--method dense|bm25|hybrid] [--top-k N] \
                                       [--candidates N] [--rerank|--no-rerank]
# Search the corpus and print the top matching chunks with provenance.
# Examples:
#   iac-copilot query "what is poisoned pipeline execution"
#   iac-copilot query "credential hygiene" --method bm25 --top-k 3
#   iac-copilot query "artifact integrity" --method hybrid --no-rerank
# Defaults: --method hybrid, --top-k 5, --candidates 30, --rerank.
# Hybrid pulls `candidates` chunks from each of BM25 and dense, fuses via RRF,
# then optionally reranks with a cross-encoder for top-k precision.

uv run iac-copilot plan <PATH> [--show-attributes]
# Parse an IaC configuration and print a summary of the resource graph —
# resource counts by action and type, plus a per-resource listing.
# Auto-detects the input shape from the path / file extension / content:
#   - *.tf file or directory of *.tf → Terraform HCL source
#   - *.yaml / *.yml / *.template    → CloudFormation YAML template
#   - *.json file with `resource_changes` key  → Terraform plan JSON
#   - *.json file with a `Resources` mapping   → CloudFormation JSON template
# The plan-JSON path sees fully-resolved attributes (post variable +
# module expansion); HCL and CloudFormation source paths see what was
# written — variables, intrinsic functions (`var.x`, `!Ref`, `!Sub`)
# survive as literal strings or their dict equivalents. For full
# resolution, prefer plan JSON.
# Examples:
#   iac-copilot plan examples/terraform-plan-example.json
#   iac-copilot plan path/to/main.tf
#   iac-copilot plan path/to/terraform-directory/
#   iac-copilot plan path/to/stack.yaml
#   iac-copilot plan path/to/cloudformation-template.json

uv run iac-copilot scan <PATH> [--json] [--output FILE]
# Scan an IaC configuration for security issues. Same input shapes as
# `plan` above — Terraform plan JSON, .tf files / directories, or
# CloudFormation YAML / JSON templates. Combines the hybrid retriever
# (BM25 + dense + RRF + rerank) with the parser to drive the LangGraph
# agent. The deterministic stub LLM's regex patterns target Terraform
# conventions; for CloudFormation templates, use a real LLM provider
# (Anthropic / OpenAI / Ollama) — the stub will miss findings the
# real LLMs catch. LLM provider is configured by LLM_PROVIDER:
#   - stub      (default) — deterministic heuristics, no API key needed
#   - anthropic — requires ANTHROPIC_API_KEY (uses tool-use for structure)
#   - openai    — requires OPENAI_API_KEY (uses response_format strict mode)
#   - ollama    — requires Ollama running locally + a model pulled
#                 (e.g. `ollama pull qwen3:latest`). No API key.
# Output is a structured ScanReport with findings, severities, suggested
# fixes, and citations into the corpus. --json prints raw JSON to stdout;
# --output writes it to a file.
# Examples:
#   iac-copilot scan examples/terraform-plan-example.json
#   iac-copilot scan path/to/main.tf
#   iac-copilot scan path/to/terraform-directory/ --output report.json
#   LLM_PROVIDER=ollama iac-copilot scan examples/terraform-plan-example.json

uv run iac-copilot eval [--output FILE] [--json-output FILE] [--golden-root DIR]
# Run the LangGraph scan agent against every case under `golden/` and emit
# a Markdown regression report. Uses the active LLM_PROVIDER, so the same
# command produces comparable reports across stub / Ollama / Anthropic / OpenAI.
# Defaults: --output evals/report.md. Golden cases live under golden/<id>/
# as (plan.json, expected.json) pairs.
# Examples:
#   iac-copilot eval                                    # stub baseline, < 1s
#   LLM_PROVIDER=anthropic iac-copilot eval             # real-quality run
#   iac-copilot eval --json-output evals/report.json    # also emit raw JSON

uv run iac-copilot stats
# Prints document and chunk counts, grouped by embedding provider.

uv run iac-copilot reset
# Drops the chunks table and clears documents. Prompts for confirmation.
# Required when changing embedding provider — see "Switching embedding
# providers" above.

Running the FastAPI service

For local development:

uv run uvicorn iac_security_copilot.main:app --reload --port 8080
curl http://localhost:8080/health     # → {"status":"ok"}

The service exposes three real endpoints on top of /health:

Route

Body

Purpose

POST /v1/search

{query, top_k}

Hybrid retrieval over the corpus — same engine the CLI's query command uses

POST /v1/scan

{plan_json}

Run the LangGraph scan agent against a Terraform plan JSON

GET /v1/sources

Document + chunk counts grouped by source

Example:

curl -s -X POST http://localhost:8080/v1/search \
  -H 'Content-Type: application/json' \
  -d '{"query":"s3 encryption","top_k":3}' | jq

curl -s -X POST http://localhost:8080/v1/scan \
  -H 'Content-Type: application/json' \
  -d "$(jq '{plan_json: .}' examples/terraform-plan-example.json)" | jq '.findings'

Via Docker:

docker build -t iac-security-copilot:0.1.0 .
docker run --rm -p 8080:8080 iac-security-copilot:0.1.0

The Docker image is Cloud Run-ready: listens on $PORT (defaults to 8080), binds to 0.0.0.0, runs as a non-root app user, and starts in under one second.

Editor integration (MCP)

The same retriever and scan agent are exposed as an MCP (Model Context Protocol) server, so editor agents like Claude Code and Cursor can call them as tools while reviewing IaC diffs in-editor.

uv run iac-copilot mcp      # boots a stdio MCP server

Three tools are registered: search_corpus, scan_terraform_plan, list_sources. The full editor config recipe (Claude Code .mcp.json snippet, Cursor ~/.cursor/mcp.json snippet, JSON-RPC smoke test) is in docs/mcp.md.

Local Kubernetes deploy (kind)

The k8s/ directory ships a one-command demo that brings the full stack up in a local kind cluster — Postgres + pgvector, the FastAPI app behind ingress-nginx, an HPA wired to a real metrics-server, and a one-shot ingest Job that loads the corpus:

./k8s/up.sh                                           # build + deploy + smoke test
curl http://localhost/v1/sources | jq                 # through the ingress
kubectl -n iac-copilot get pods,svc,hpa,ingress       # full surface
./k8s/down.sh

Manifest-by-manifest walkthrough lives in k8s/README.md. The same set of YAML (with a PVC instead of emptyDir and a real LoadBalancer) is what would ship to a managed cluster like GKE Autopilot.

Cloud deployment

Two end-to-end recipes for running this on Google Cloud:

  • docs/deploy-cloud-run.md — serverless HTTPS endpoint via Cloud Run + Cloud SQL. Scales to zero, no Kubernetes, one image push and one gcloud run deploy. Right tool when you want a cheap always-on demo URL.

  • docs/deploy-gke.md — full Kubernetes deploy on GKE Autopilot + Cloud SQL using the same manifests as the local kind cluster (with Workload Identity for Cloud SQL auth, a cloud-sql-proxy sidecar, and a managed LoadBalancer). Right tool when you actually want K8s semantics.

Each recipe has a gcloud walkthrough plus a fully-equivalent Terraform module under terraform/terraform apply instead of the shell calls when you'd rather have the infra in code. Both modules pin every upstream image version, generate the DB password with random_password, and put the connection string in Secret Manager.

The Dockerfile, the agent code, the eval harness, and the MCP server are unchanged in both — only the surrounding infrastructure differs.

Running the tests

All tests live under tests/ and use plain pytest.

# Full suite
uv run pytest

# Verbose, per-test output
uv run pytest -v

# Single file
uv run pytest tests/test_chunker.py

# Single test
uv run pytest tests/test_chunker.py::test_heading_path_tracks_nesting

The current suite is 118 tests, runs in under two seconds end-to-end. No external dependencies (no DB, no API calls) — the embedder tests use a fake implementation rather than spinning up BGE, the retrieval tests exercise the RRF fusion logic in isolation, the plan tests use synthetic JSON / HCL / CloudFormation fixtures inline (the HCL parser has 15 tests covering attribute shapes + module skipping; the CFN parser has 19 covering YAML intrinsics + multi-file merge), the agent tests inject a _StaticRetriever + the deterministic stub LLM rather than requiring Postgres or any LLM API, the Ollama client tests use httpx.MockTransport, the eval tests construct golden cases inline, the CIS source tests run against the committed corpus/cis-aws/ directory directly, and the HTTP API + MCP server tests monkeypatch the service-layer entrypoints so the route plumbing and tool plumbing can be tested independently of Postgres.

Development workflow

# Lint and auto-fix Python
uv run ruff check --fix

# Format Python
uv run ruff format

# Run every pre-commit hook over the whole tree
uv run pre-commit run --all-files

Pre-commit hook installation

The repo is set up to install pre-commit's hook into .git/hooks/pre-commit. If your machine has core.hooksPath set globally (common when using a shared dotfiles repo), you'll need a per-repo override:

git config --local core.hooksPath .git/hooks
uv run pre-commit install   # or `cp` the hook into .git/hooks/ manually

Subsequent git commit will run all hooks: hygiene checks, ruff lint + format, detect-secrets, hadolint on the Dockerfile.

What's not in the repo yet

  • Hosted demo URL — the FastAPI service is Cloud Run-ready, but no always-on instance is deployed.

  • CloudFormation / Pulumi / Bicep parsers — the ResourceGraph shape is provider-agnostic so adding one is a parser-only change, but only Terraform plans are wired up.

  • AWS Well-Architected as a source — listed under planned in the architecture diagram. The OWASP CI/CD and CIS AWS sources already cover the same ground for IaC-specific findings.

  • GitHub Action wrapper — auto-PR mode that comments findings on PRs touching *.tf files. Easy to layer on top of the CLI.

Sources

  • OWASP Top 10 CI/CD Security Risks — primary ingestion source for CI/CD-shaped IaC concerns. Substitutes for the (non-existent) "OWASP IaC Top 10".

  • CIS AWS Foundations Benchmark — ten hand-curated control summaries under corpus/cis-aws/, attributed to the canonical CIS reference (the benchmark itself is licensed CC BY-NC-SA 4.0 so verbatim reproduction is avoided).

  • AWS Well-Architected Framework — planned as an additional source.

  • pgvector — Postgres extension for vector similarity search.

  • fastembed — lightweight ONNX-based embedding library (BGE backend + cross-encoder reranker).

  • LangGraph — the agent topology library used for the scan agent's Send-based parallel fan-out.

  • Model Context Protocol — the open spec the MCP stdio server implements.

License

Apache 2.0

Available Tools

3 tools
list_sourcesList ingested corpus sourcesA

Return document and chunk counts grouped by source. Useful to verify the corpus an agent is grounding answers in before trusting its citations.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
sourcesYes

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the behavioral transparency burden. It clearly indicates a read-only aggregation behavior by describing returned counts, but it does not mention potential edge cases such as empty corpora, data freshness, or permission requirements. The core behavior is disclosed, but some context is missing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler. The first sentence states exactly what the tool returns, and the second gives practical usage context. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter list tool with an output schema, the description is fully sufficient. It tells the agent what data to expect and why the tool matters for grounding verification. No additional context is needed to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description correctly focuses on what the tool returns rather than trying to document nonexistent arguments.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: it returns document and chunk counts grouped by source. This clearly distinguishes it from the sibling tools search_corpus and scan_terraform_plan, which perform different functions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear use case: verify the ingested corpus before trusting an agent's citations. It does not explicitly name alternatives or state when not to use the tool, but the context provided is sufficient for a zero-parameter listing tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

scan_terraform_planScan a Terraform plan for security issuesA

Run the LangGraph scan agent against a Terraform plan JSON (output of terraform show -json). Returns structured findings with severity, suggested fixes and citations into the corpus. Defaults to LLM_PROVIDER=stub if no API key is configured.

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_jsonYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
findingsNo
llm_modelNo
embedding_modelNo
terraform_versionNo
resources_analyzedNo
plan_format_versionNo

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It adds useful context: it invokes an LLM-dependent agent, defaults to a stub provider when no API key is configured, and returns structured findings. However, it does not disclose potential side effects, network usage, or what happens when the stub provider is used.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action and input source, followed by return-value summary and configuration default. Every sentence adds distinct, useful information with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is a single-parameter operation with an output schema, so the description need not detail return values. It covers input provenance, output summary, and a key configuration default. The only minor gap is not elaborating on the stub-provider implication or environment expectations, but overall it is sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, but the description compensates by explaining that plan_json must be the output of 'terraform show -json'. Since there is only one parameter, this is sufficient to understand what to pass, despite the schema itself carrying no semantic description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Run the LangGraph scan agent') against a specific resource ('a Terraform plan JSON'), and clarifies the expected input format with the command that produces it. It clearly distinguishes the tool from its siblings, which search or list sources rather than scan.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies when to use the tool: when you have a Terraform plan JSON and need security findings. It also provides the exact command to generate the input. However, it does not explicitly compare with the sibling tools or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_corpusSearch the IaC security corpusA

Hybrid (BM25 + dense + RRF) retrieval over CIS AWS Foundations and OWASP CI/CD Top 10 guidance. Returns the top-k matching chunks with document title, heading path, URL and verbatim content for citation.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYes
top_kNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
queryYes
resultsYes

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses the retrieval algorithm (BM25 + dense + RRF), the targeted corpus, and the exact return payload (document title, heading path, URL, verbatim content). While it does not explicitly state side-effect freedom, the language is consistent with a non-mutating search operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a tight two-sentence structure. The first sentence states the method and corpus; the second defines the return format. Every sentence adds critical information, and the most distinguishing element (retrieval over specific guidance sources) is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the corpus scope, the retrieval method, and the exact contents of the returned chunks, which is sufficient given the simple two-parameter schema and existing output schema. It omits explicit usage guidance and caveats, but these are minor for a straightforward read-only search tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides no parameter descriptions, so the tool description must compensate. The phrase 'top-k matching chunks' clarifies the top_k parameter's role, and 'retrieval' implies query is the search text. However, the description does not explain query formulation, permissible input types, or top_k bounds, leaving the agent to infer most semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Hybrid (BM25 + dense + RRF) retrieval over CIS AWS Foundations and OWASP CI/CD Top 10 guidance.' This clearly differentiates the tool from siblings like scan_terraform_plan (scanning actions) and list_sources (listing sources), making the tool's purpose unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by describing what the tool searches and returns, but it never explicitly states when to use this tool versus the siblings or any exclusions. The purposeful phrasing 'retrieval over CIS AWS Foundations and OWASP CI/CD Top 10 guidance' makes the use case inferable, yet explicit guidance is absent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

A4.2/5.0
Disambiguation5/5

Each tool serves a clearly distinct purpose: searching the guidance corpus, scanning a Terraform plan, and inspecting corpus sources. There is no overlap or ambiguity between them.

Naming Consistency5/5

All tool names follow the same snake_case verb_noun pattern: search_corpus, scan_terraform_plan, list_sources. This is consistent and predictable.

Tool Count4/5

Three tools is on the lower end but still within the well-scoped range. Each tool earns its place for the server's focused IaC copilot purpose.

Completeness4/5

The core workflow of searching guidance, scanning a plan, and verifying corpus sources is covered. Minor gaps exist such as no direct tool for retrieving a full document or listing supported scan rules, but these are not major dead ends.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    D
    maintenance
    Allows developers to query security findings (SAST issues, secrets, patches) using natural language within AI-assisted tools like Claude Desktop, Cursor, and other MCP-compatible environments.
    17
    9
    MIT
  • A
    license
    A
    quality
    A
    maintenance
    MCP security server for AI coding agents. 12 tools: pre-install guardian, vulnerability audit, supply-chain attack detection via static code analysis, and CycloneDX 1.6 SBOM generation. Zero runtime dependencies.
    14
    43
    15
    Apache 2.0
  • A
    license
    A
    quality
    D
    maintenance
    Security scanning for AI coding tools (Claude Code, Cursor, Windsurf) including secrets detection, MCP config vulnerabilities, agent instruction checks, threat modeling, prompt injection testing, pre-commit security checks, and dependency vulnerability scanning.
    7
    50
    1
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables MCP-compatible agents to query security checklists, protocols, and framework mappings to safely build and operate AI agents, with tools for checklist runs, protocol lookup, mapping lookup, and threat search.
    9
    CC BY-4.0

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/togleid/iac-secopilot'

If you have feedback or need assistance with the MCP directory API, please join our Discord server