# Reliable Agentic RAG

A runnable, synthetic expense-policy workflow showing where model planning ends and deterministic decisions begin. It retrieves fictional policies, checks a claim, and requires a separate human step before recording a simulated payment.

**Status:** local portfolio demonstration. No real money, customer records, employer source, or legal rules. The default planner is scripted, not an LLM. An optional local Ollama planner selects validated tool calls.

## Ownership

Portfolio owner: Esteban H. Román Catafau. The initial implementation was developed with Codex assistance for this portfolio. It demonstrates inspectable engineering patterns; it is not presented as an independently authored historical project or a copy of any production system.

## Quick Start

Python 3.13 and [uv](https://docs.astral.sh/uv/getting-started/installation/) are required. Dependency download is the only network use in the default demo; no model download or API key is needed.

```bash
uv sync --frozen --extra dev
uv run --frozen policy-demo demo
uv run --frozen pytest -q
```

The sample claim requests 42.50 synthetic credits for a meal with a receipt. The tool returns 30.00, cites the fictional policy IDs, and records no payment.

```bash
uv run --frozen policy-demo demo --propose
uv run --frozen policy-demo review PROPOSAL_ID
uv run --frozen policy-demo approve PROPOSAL_ID
```

Replace `PROPOSAL_ID` with the full ID printed by the first command. The last command displays the proposal and asks you to type its complete ID in an interactive terminal. It records only a synthetic payment. Repeated approval fails; repeated identical proposals reuse the same ID. A claim ID cannot be reused with different contents.

## Architecture

```mermaid
flowchart TD
    C[Validated synthetic claim] --> P[Scripted or local model planner]
    P --> R[BM25 + LSA retrieval]
    R --> P
    P --> T[Deterministic Decimal policy tool]
    T --> O[Structured assessment and fixed citations]
    O --> S[Pending proposal in local SQLite]
    S --> H[Separate interactive human review]
    H --> A[Atomic simulated payment and audit events]
    M[MCP stdio client] --> R
    M --> T
```

- **Planning:** a maximum of five validated steps. Unknown tools and premature completion fail closed. Model output cannot replace the original claim or authorize approval.
- **Retrieval:** BM25 plus TF-IDF/SVD latent semantic vectors, fused with reciprocal rank fusion. LSA is trained on six fictional documents; it is not a pretrained neural embedding model. Out-of-vocabulary queries return no results.
- **Calculation:** Decimal arithmetic and fixed category caps. Missing receipts produce an ineligible decision. Retrieved prose is not parsed into executable policy; a small trusted policy module is authoritative.
- **Evidence:** output citations come from the deterministic tool, not generated model prose. Valid IDs here do not constitute a general solution to citation entailment.
- **Approval:** a SHA-256 ID binds the stored assessment to its contents. SQLite transactions enforce single-use approval and append lifecycle events atomically. The local database owner is trusted.

## MCP and Optional Model

```bash
uv run --frozen policy-mcp
```

This starts a real MCP stdio server using the official Python SDK. Exposed tools are `search_policy` and `assess_claim`. There is deliberately no approval tool. The integration test launches a subprocess, initializes a client session, discovers tools, and calls the assessment tool with valid and invalid inputs.

The implementation pins SDK 1.30.0 and uses its v1 FastMCP API; upgrading to SDK v2 requires migration. See the [SDK v1 branch](https://github.com/modelcontextprotocol/python-sdk/tree/v1.x).

For model-driven planning, install and run Ollama separately with a model supporting JSON schema output:

```bash
uv run --frozen policy-demo demo --ollama-model YOUR_INSTALLED_MODEL
```

Only the local loopback endpoint is used. Requests disable environment proxies and redirects. A model can fail to follow the plan; this is reported as failure, never converted into an approval. No real-model result is claimed by the offline demo.

## Evaluation and Limits

Tests cover cap boundaries, missing receipts, malformed inputs, bounded planning, retrieved policy relevance, fabricated assessments, storage tampering, duplicate proposals, repeated approvals, and the real MCP protocol. See [VALIDATION.md](VALIDATION.md) for the observed run. No production reliability rate is inferred from these tests.

This small corpus is not evidence of enterprise retrieval quality. The local approval step is a demonstration, not multi-user authentication: a person or agent with shell/database access can act as the owner. Audit rows are not externally tamper-proof. There is no payment integration, tenant isolation, OCR, live policy ingestion, or remote MCP deployment. Failed planner attempts are surfaced to the caller; the persistent event log covers proposal/payment lifecycle only.

## Files and Development

```text
src/policy_agent/
  policy.py       # input/output schemas and deterministic rules
  retrieval.py    # BM25 + LSA and rank fusion
  agent.py        # bounded planner/tool loop and local model adapter
  store.py        # proposals and transactional approval events
  server.py       # MCP stdio interface
  cli.py          # demo and human-review commands
tests/            # behavioral and protocol tests
```

```bash
uv run --frozen ruff check .
uv run --frozen ruff format --check .
uv build
```

See [SECURITY.md](SECURITY.md) for trust boundaries and [DISCLOSURE.md](DISCLOSURE.md) for data provenance. MIT license; dependency licenses remain their respective owners'.