# Veritas MCP — evidence-grounded "Find Evil" DFIR triage

> *Veritas* (Latin, "truth"). A custom **Model Context Protocol** server that lets
> an AI agent triage a compromised host the way a senior DFIR analyst does — and
> that makes the two failure modes the judges care about most, **evidence
> spoliation** and **hallucinated findings**, *architecturally impossible* rather
> than merely discouraged by a prompt.

**SANS "Find Evil!" Hackathon submission — Approach #2 (custom MCP server).**

---

## The problem (and why prompts aren't enough)

The baseline approach to "agentic DFIR" hands a model a shell on a SIFT
workstation and a prompt that says *please be careful and don't make things up*.
Two things go wrong:

1. **Spoliation.** A shell can `dd`, `del`, mount read-write, or exfiltrate. One
   bad tool call destroys the evidence and the case. A prompt asking the model to
   behave is not a control an examiner can testify to.
2. **Hallucination.** LLMs assert tidy conclusions that the data never supported.
   In forensics an unsupported "finding" is not a minor error — it is the thing
   that gets a case thrown out.

Veritas removes both at the architecture level.

## How Veritas is different

| Concern | Prompt-only baseline | **Veritas** |
| --- | --- | --- |
| Destroying evidence | "Please don't." | **No tool exists that writes to, deletes, or mounts evidence.** Evidence handles are opened `O_RDONLY`; writes resolving inside an evidence root are refused by `EvidenceGuard`. |
| Running arbitrary commands | `execute_shell_cmd` | **No shell tool.** Only 8 typed, allow-listed, read-only forensic tools. Shell metacharacters and forbidden binaries are rejected in depth. |
| Hallucinated findings | hope | **Grounding enforcement:** a finding that does not cite the `exec_id` of a real, recorded tool run is *rejected by the server*. |
| "Confirmed" vs guess | undifferentiated prose | **Explicit confidence tiers** (`CONFIRMED`/`INFERRED`/`HYPOTHESIS`/`RETRACTED`); `CONFIRMED` requires ≥2 independent source families (disk/log/memory). |
| First-pass mistakes | shipped as final | **Self-correcting loop:** retracts false positives against a known-good baseline, fills gaps the correlation engine finds (timestomping), and promotes corroborated findings — iterating to a fixed point. |
| Context-window overload | raw tool text dumped into the model | Raw output is parsed **server-side** into compact typed records; the model reasons over fields, not megabytes. |
| Explainability | none | **Append-only JSONL audit log.** Every tool run gets a stable `exec_id`; every finding traces back to it. Full chain of custody. |
| "Trust me, it works" | — | **Reproducible accuracy benchmark** vs documented ground truth, runnable offline with zero third-party deps. |

## Headline result (reproducible: `python -m veritas.benchmark`)

On the bundled synthetic case `WEBSRV01`, the same engine with self-correction
turned **off** (the prompt-only baseline) vs **on** (Veritas):

| | TP | FP | FN | Precision | Recall | F1 | Hallucination |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Prompt-only baseline | 6 | 1 | 2 | 0.857 | 0.75 | 0.80 | 0.143 |
| **Veritas** | **8** | **0** | **0** | **1.00** | **1.00** | **1.00** | **0.00** |

The baseline misses **timestomping** and the **C2 channel**, and over-flags a
benign `OneDriveStandaloneUpdater.exe`. Veritas catches the timestomp via
`$SI`/`$FN` correlation, confirms the C2 across memory + network, and *retracts*
the OneDrive false positive against a known-good baseline.

## Quick start (60 seconds, no dependencies)

```powershell
# from the repo root — pure standard library, simulation mode
$env:PYTHONPATH = "src"
python -m veritas.cli triage        # human-readable findings report
python -m veritas.benchmark         # the accuracy table above
python -m pytest -q                 # 46 tests, incl. spoliation-impossibility
```

Or install it and use the console script:

```powershell
pip install -e .
veritas triage
veritas benchmark
```

To drive it from **Claude Code** as a real MCP server:

```powershell
pip install -e ".[mcp]"
# .mcp.json + claude/CLAUDE.md + claude/settings.json are included; see docs/TRY_IT_OUT.md
```

## Architecture at a glance

```
Host agent (Claude Code)
        │  MCP (stdio) — only typed, read-only tools are exposed
        ▼
┌──────────────────────── Veritas MCP server ────────────────────────┐
│  8 forensic tools ──► Executor (allow-list, shell=False) ──► EvidenceGuard │
│        │                         │                          (O_RDONLY)     │
│        ▼                         ▼                                          │
│   Parsers (raw→typed)      AuditLog (append-only JSONL, exec_id)            │
│        │                         ▲                                          │
│        ▼                         │ grounding check                         │
│   TriageLoop ──► FindingStore (rejects ungrounded; confidence tiers)        │
│        │            ▲                                                        │
│        ▼            │                                                        │
│   CorrelationEngine + KnownGoodBaseline (self-correction)                   │
└─────────────────────────────────────────────────────────────────────────┘
        │ writes ONLY to:  exports/  analysis/  reports/   (never evidence)
```

Full diagram, trust boundaries, and the **prompt-guardrails vs.
architectural-guardrails** distinction are in
[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).

## Repository map

| Path | What it is |
| --- | --- |
| `src/veritas/` | The engine (stdlib-only core; `mcp` is an optional extra) |
| `fixtures/case_websrv01/` | Synthetic but internally-consistent intrusion + simulated tool output |
| `fixtures/ground_truth/` | Documented ground truth used by the benchmark |
| `tests/` | 46 tests, including proofs that spoliation/hallucination are blocked |
| `logs/sample_agent_execution.jsonl` | A real captured audit trail from one triage run |
| `claude/` | `CLAUDE.md` operating instructions + `settings.json` permissions |
| `.mcp.json` | MCP server registration for Claude Code |
| `docs/` | The eight submission documents (see below) |

## Submission documents

1. **This README** — overview & quick start.
2. [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — design, trust boundaries, guardrail taxonomy.
3. [docs/ACCURACY_REPORT.md](docs/ACCURACY_REPORT.md) — methodology + reproducible numbers.
4. [docs/DATASET.md](docs/DATASET.md) — the synthetic case, how it was built, and why it's safe.
5. [docs/TRY_IT_OUT.md](docs/TRY_IT_OUT.md) — step-by-step, incl. Claude Code wiring.
6. [docs/DEMO_SCRIPT.md](docs/DEMO_SCRIPT.md) — the 5-minute demo-video script.
7. [docs/SUBMISSION.md](docs/SUBMISSION.md) — Devpost write-up / story & judging-criteria map.
8. `logs/sample_agent_execution.jsonl` — sample agent execution log (chain of custody).

## License

MIT — see [LICENSE](LICENSE). All bundled case data is synthetic; no real systems,
persons, or indicators are represented.