# Vindex (`Ingesta`)

**Turn any video into searchable moments, keyframes, and AI-ready datasets—100% locally.**
*Give AI agents (Claude Code, Cursor, MCP) eyes and ears over your video library with zero cloud dependencies.*

---

## Why Vindex?

Most video tools either choke on long-form video, recompute expensive transcription/vision stages from scratch when you tweak a parameter, or require a heavy distributed stack (Docker + Redis + Celery + Qdrant + Postgres).

**Vindex (`Ingesta`)** compiles raw local video files, folders, or URLs into deterministic, content-addressed **`VideoBundle`s** using a single-node **SQLite WAL control plane**:

- **8-Op Content-Addressed Merkle DAG:** `fetch → probe ∥ audio ∥ scenes → transcribe ∥ frames → embed → bundle`. Changing `embed_model` reuses cached `probe`, `audio`, `scenes`, `transcribe`, and `frames` in `<5ms` per op (`INV-07`).
- **Crash-Only Worker Pool & Lease Reaper:** `BEGIN IMMEDIATE … RETURNING` atomic job claims, heartbeated leases, background lease reaper, and process-group (`os.setsid` / `os.killpg`) subprocess isolation (`INV-01`..`INV-10`).
- **Supervised Warm ML Sidecar:** Keeps heavy ML weights (`faster-whisper`, `open_clip`) warm across jobs over a local socket while completely isolating the control plane from OOM/segfault crashes.
- **Perceptual Keyframe Dedup + Contact-Sheet Filmstrips:** 64-bit color-aware `dHash` intra-shot deduplication selects `(start, medoid, end)` keyframes and stitches 3-tile contact-sheet filmstrips for VLM temporal grounding.
- **Hybrid Multimodal Retrieval (`RRF` + Temporal `MMR`):** Fuses `sqlite-vec` visual KNN over shot keyframes with `SQLite FTS5` BM25 over exact + `±1.5s` speech-to-shot windows, deduplicated via Temporal Maximal Marginal Relevance.
- **Zero-Config Interfaces:**
  - **CLI (`vindex`)**: `ingest`, `search`, `inspect`, `export`, `status`, `doctor`, `gc`, `chaos`, `server`, `mcp`.
  - **MCP Server (`vindex mcp`)**: 4 JSON-RPC 2.0 tools (`ingest_video`, `search_moments`, `get_shot_context`, `extract_clip`) for Claude Code, Cursor, and Antigravity.
  - **FastAPI + Live HTMX Dashboard (`vindex server`)**: Real-time DAG telemetry and a live `⚡ CHAOS: SIGKILL Random Worker` button.
  - **Dataset Exporters**: WebDataset (`.tar`), Apache Parquet (`.parquet`), JSONL (`.jsonl`), and Obsidian/Notion Markdown notes (`.md`).

---

## Quickstart

```bash
# 1. Install into Python 3.12 virtualenv
uv venv --python 3.12 .venv
uv pip install -e ".[dev]"

# 2. Ingest a video (or directory of videos)
uv run vindex ingest /path/to/video.mp4

# 3. Run hybrid multimodal search (RRF + Temporal MMR)
uv run vindex search "architecture diagram" --mode hybrid --top-k 5

# 4. Verify all 10 formal system invariants (INV-01 .. INV-10)
uv run vindex doctor

# 5. Export bundle to WebDataset, Parquet, JSONL, or Markdown
uv run vindex export <video_sha_prefix> --format parquet --out ./exports

# 6. Launch the FastAPI Control Plane & Live HTMX Dashboard
uv run vindex server --port 8000
```

---

## Repository Layout

See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the full 6-layer architecture (`core` → `models` → `engine` → `ops` → `index` & `exporters` → `interfaces`), the 8-op content-addressed Merkle DAG, and the 10 formal system invariants (`INV-01`–`INV-10`), and see [`CONTRIBUTING.md`](CONTRIBUTING.md) for development guidelines.

```bash
make check      # Run Ruff linter across src/ and tests/
make test-all   # Run all unit, integration, and chaos tests
make bench      # Run the 3-part reproducible systems benchmark suite
```