Vindex MCP Server
Exports video bundle datasets to Apache Parquet files for analytics and downstream data processing.
Exports video bundle content as Markdown notes that can be imported into Notion.
Exports video bundle content as Markdown notes that can be used in Obsidian.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Vindex MCP Server find the moment in my video where the speaker shows the architecture diagram"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Vindex (Ingesta)
Turn any video into searchable moments, keyframes, and AI-ready datasets—100% locally. Give AI agents (Claude Code, Cursor, MCP) eyes and ears over your video library with zero cloud dependencies.
Why Vindex?
Most video tools either choke on long-form video, recompute expensive transcription/vision stages from scratch when you tweak a parameter, or require a heavy distributed stack (Docker + Redis + Celery + Qdrant + Postgres).
Vindex (Ingesta) compiles raw local video files, folders, or URLs into deterministic, content-addressed VideoBundles using a single-node SQLite WAL control plane:
8-Op Content-Addressed Merkle DAG:
fetch → probe ∥ audio ∥ scenes → transcribe ∥ frames → embed → bundle. Changingembed_modelreuses cachedprobe,audio,scenes,transcribe, andframesin<5msper op (INV-07).Crash-Only Worker Pool & Lease Reaper:
BEGIN IMMEDIATE … RETURNINGatomic job claims, heartbeated leases, background lease reaper, and process-group (os.setsid/os.killpg) subprocess isolation (INV-01..INV-10).Supervised Warm ML Sidecar: Keeps heavy ML weights (
faster-whisper,open_clip) warm across jobs over a local socket while completely isolating the control plane from OOM/segfault crashes.Perceptual Keyframe Dedup + Contact-Sheet Filmstrips: 64-bit color-aware
dHashintra-shot deduplication selects(start, medoid, end)keyframes and stitches 3-tile contact-sheet filmstrips for VLM temporal grounding.Hybrid Multimodal Retrieval (
RRF+ TemporalMMR): Fusessqlite-vecvisual KNN over shot keyframes withSQLite FTS5BM25 over exact +±1.5sspeech-to-shot windows, deduplicated via Temporal Maximal Marginal Relevance.Zero-Config Interfaces:
CLI (
vindex):ingest,search,inspect,export,status,doctor,gc,chaos,server,mcp.MCP Server (
vindex mcp): 4 JSON-RPC 2.0 tools (ingest_video,search_moments,get_shot_context,extract_clip) for Claude Code, Cursor, and Antigravity.FastAPI + Live HTMX Dashboard (
vindex server): Real-time DAG telemetry and a live⚡ CHAOS: SIGKILL Random Workerbutton.Dataset Exporters: WebDataset (
.tar), Apache Parquet (.parquet), JSONL (.jsonl), and Obsidian/Notion Markdown notes (.md).
Related MCP server: klaket-mcp
Quickstart
# 1. Install into Python 3.12 virtualenv
uv venv --python 3.12 .venv
uv pip install -e ".[dev]"
# 2. Ingest a video (or directory of videos)
uv run vindex ingest /path/to/video.mp4
# 3. Run hybrid multimodal search (RRF + Temporal MMR)
uv run vindex search "architecture diagram" --mode hybrid --top-k 5
# 4. Verify all 10 formal system invariants (INV-01 .. INV-10)
uv run vindex doctor
# 5. Export bundle to WebDataset, Parquet, JSONL, or Markdown
uv run vindex export <video_sha_prefix> --format parquet --out ./exports
# 6. Launch the FastAPI Control Plane & Live HTMX Dashboard
uv run vindex server --port 8000Repository Layout
See docs/ARCHITECTURE.md for the full 6-layer architecture (core → models → engine → ops → index & exporters → interfaces), the 8-op content-addressed Merkle DAG, and the 10 formal system invariants (INV-01–INV-10), and see CONTRIBUTING.md for development guidelines.
make check # Run Ruff linter across src/ and tests/
make test-all # Run all unit, integration, and chaos tests
make bench # Run the 3-part reproducible systems benchmark suiteAvailable Tools
5 toolsget_ingest_statusB
Get per-op DAG status and progress for an ingest ID.
Inputs: ingest_id (required).
get_shot_contextB
Agentic Zoom (VideoTree/VideoAgent): expand a shot to inspect adjacent neighbor shots, keyframe paths, filmstrips, and continuous transcript.
Inputs: shot_id (required), video_sha256 (required), neighbor_shots.
get_transcriptC
Retrieve timestamped transcript segments for a video within [t_start, t_end].
Inputs: video_sha256 (required), t_end, t_start.
ingest_videoC
Ingest a local video file or YouTube URL into a searchable multimodal Vindex bundle.
Inputs: url_or_path (required), sync, embed_model, whisper_model.
search_corpusA
Hybrid visual (CLIP/SigLIP) + lexical (BM25) RRF search across ingested videos. Returns ranked shots with timestamps, transcript, and filmstrip image paths.
Inputs: query (required), k, mode, video_filter.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 5 tools
The tools generally have distinct roles across ingest, status, search, shot inspection, and transcript retrieval. There is some overlap because search_corpus and get_shot_context both return transcript/filmstrip context, and get_transcript can be confused with transcript fields returned by other tools, but the descriptions make the intended boundaries reasonably clear.
All tool names use consistent snake_case with a clear verb_noun or verb_noun_object pattern: ingest_video, get_ingest_status, search_corpus, get_shot_context, get_transcript. The naming is predictable and easy to infer.
Five tools is well scoped for a video ingestion and retrieval server. Each tool covers a distinct stage of the workflow from ingestion through search and contextual inspection, with no obviously redundant tools.
The core lifecycle is covered: ingest, check status, search, inspect shot context, and retrieve transcripts. Minor gaps remain around corpus/video management, such as listing ingested videos or deleting/removing bundles, which agents may need for cleanup or discovery.
Maintenance
Related MCP Connectors
VideoDB lets an AI agent watch, search and clip video.
MCP server for Clipkit — gives AI agents a video toolbox via the Clipkit schema.
Open-source AI video editor: cut, caption, clip and animate real Revline projects from your agent.
Turn any video into context for AI agents: transcripts, metadata, chapters, timestamps.
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to query local video timelines by extracting speech, frame captions, and on-screen text into a SQLite store, exposing search and retrieval tools via MCP.PolyForm Noncommercial 1.0.0
- AlicenseAqualityCmaintenanceLet AI agents watch videos: local transcripts, speaker labels, scenes, chapters and exact-moment search from any video URL or file. Fully local, no API keys.43AGPL 3.0
- AlicenseNot gradedqualityDmaintenanceEnables LLM agents to process local videos into timestamped, citable text documents and then query them through tools for listing videos, retrieving transcripts, and fetching specific segments, all fully offline.21 PyPIMIT
- AlicenseNot gradedqualityBmaintenanceEnables an agent to process local video files and then read, search and inspect the results: timestamped transcripts, on-screen text and key frames as images. Everything runs locally, so an agent can answer questions like how to perform a task shown in a screencast or meeting recording.84 npm1MIT