Skip to main content
Glama

Vindex (Ingesta)

Turn any video into searchable moments, keyframes, and AI-ready datasets—100% locally. Give AI agents (Claude Code, Cursor, MCP) eyes and ears over your video library with zero cloud dependencies.


Why Vindex?

Most video tools either choke on long-form video, recompute expensive transcription/vision stages from scratch when you tweak a parameter, or require a heavy distributed stack (Docker + Redis + Celery + Qdrant + Postgres).

Vindex (Ingesta) compiles raw local video files, folders, or URLs into deterministic, content-addressed VideoBundles using a single-node SQLite WAL control plane:

  • 8-Op Content-Addressed Merkle DAG: fetch → probe ∥ audio ∥ scenes → transcribe ∥ frames → embed → bundle. Changing embed_model reuses cached probe, audio, scenes, transcribe, and frames in <5ms per op (INV-07).

  • Crash-Only Worker Pool & Lease Reaper: BEGIN IMMEDIATE … RETURNING atomic job claims, heartbeated leases, background lease reaper, and process-group (os.setsid / os.killpg) subprocess isolation (INV-01..INV-10).

  • Supervised Warm ML Sidecar: Keeps heavy ML weights (faster-whisper, open_clip) warm across jobs over a local socket while completely isolating the control plane from OOM/segfault crashes.

  • Perceptual Keyframe Dedup + Contact-Sheet Filmstrips: 64-bit color-aware dHash intra-shot deduplication selects (start, medoid, end) keyframes and stitches 3-tile contact-sheet filmstrips for VLM temporal grounding.

  • Hybrid Multimodal Retrieval (RRF + Temporal MMR): Fuses sqlite-vec visual KNN over shot keyframes with SQLite FTS5 BM25 over exact + ±1.5s speech-to-shot windows, deduplicated via Temporal Maximal Marginal Relevance.

  • Zero-Config Interfaces:

    • CLI (vindex): ingest, search, inspect, export, status, doctor, gc, chaos, server, mcp.

    • MCP Server (vindex mcp): 4 JSON-RPC 2.0 tools (ingest_video, search_moments, get_shot_context, extract_clip) for Claude Code, Cursor, and Antigravity.

    • FastAPI + Live HTMX Dashboard (vindex server): Real-time DAG telemetry and a live ⚡ CHAOS: SIGKILL Random Worker button.

    • Dataset Exporters: WebDataset (.tar), Apache Parquet (.parquet), JSONL (.jsonl), and Obsidian/Notion Markdown notes (.md).


Related MCP server: klaket-mcp

Quickstart

# 1. Install into Python 3.12 virtualenv
uv venv --python 3.12 .venv
uv pip install -e ".[dev]"

# 2. Ingest a video (or directory of videos)
uv run vindex ingest /path/to/video.mp4

# 3. Run hybrid multimodal search (RRF + Temporal MMR)
uv run vindex search "architecture diagram" --mode hybrid --top-k 5

# 4. Verify all 10 formal system invariants (INV-01 .. INV-10)
uv run vindex doctor

# 5. Export bundle to WebDataset, Parquet, JSONL, or Markdown
uv run vindex export <video_sha_prefix> --format parquet --out ./exports

# 6. Launch the FastAPI Control Plane & Live HTMX Dashboard
uv run vindex server --port 8000

Repository Layout

See docs/ARCHITECTURE.md for the full 6-layer architecture (core → models → engine → ops → index & exporters → interfaces), the 8-op content-addressed Merkle DAG, and the 10 formal system invariants (INV-01–INV-10), and see CONTRIBUTING.md for development guidelines.

make check      # Run Ruff linter across src/ and tests/
make test-all   # Run all unit, integration, and chaos tests
make bench      # Run the 3-part reproducible systems benchmark suite

Available Tools

5 tools
get_ingest_statusB

Get per-op DAG status and progress for an ingest ID.

Inputs: ingest_id (required).

get_shot_contextB

Agentic Zoom (VideoTree/VideoAgent): expand a shot to inspect adjacent neighbor shots, keyframe paths, filmstrips, and continuous transcript.

Inputs: shot_id (required), video_sha256 (required), neighbor_shots.

get_transcriptC

Retrieve timestamped transcript segments for a video within [t_start, t_end].

Inputs: video_sha256 (required), t_end, t_start.

ingest_videoC

Ingest a local video file or YouTube URL into a searchable multimodal Vindex bundle.

Inputs: url_or_path (required), sync, embed_model, whisper_model.

search_corpusA

Hybrid visual (CLIP/SigLIP) + lexical (BM25) RRF search across ingested videos. Returns ranked shots with timestamps, transcript, and filmstrip image paths.

Inputs: query (required), k, mode, video_filter.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    B3.4/5.0

    Scored across 5 tools

    Disambiguation4/5

    The tools generally have distinct roles across ingest, status, search, shot inspection, and transcript retrieval. There is some overlap because search_corpus and get_shot_context both return transcript/filmstrip context, and get_transcript can be confused with transcript fields returned by other tools, but the descriptions make the intended boundaries reasonably clear.

    Naming Consistency5/5

    All tool names use consistent snake_case with a clear verb_noun or verb_noun_object pattern: ingest_video, get_ingest_status, search_corpus, get_shot_context, get_transcript. The naming is predictable and easy to infer.

    Tool Count5/5

    Five tools is well scoped for a video ingestion and retrieval server. Each tool covers a distinct stage of the workflow from ingestion through search and contextual inspection, with no obviously redundant tools.

    Completeness4/5

    The core lifecycle is covered: ingest, check status, search, inspect shot context, and retrieve transcripts. Minor gaps remain around corpus/video management, such as listing ingested videos or deleting/removing bundles, which agents may need for cleanup or discovery.

    Maintenance

    ActivityMaintained
    ResponsivenessLimited data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to query local video timelines by extracting speech, frame captions, and on-screen text into a SQLite store, exposing search and retrieval tools via MCP.
      PolyForm Noncommercial 1.0.0
    • A
      license
      A
      quality
      C
      maintenance
      Let AI agents watch videos: local transcripts, speaker labels, scenes, chapters and exact-moment search from any video URL or file. Fully local, no API keys.
      4
      3
      AGPL 3.0
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables an agent to process local video files and then read, search and inspect the results: timestamped transcripts, on-screen text and key frames as images. Everything runs locally, so an agent can answer questions like how to perform a task shown in a screencast or meeting recording.
      84 npm
      1
      MIT