Skip to main content
Glama
SourikAdhikary

Vindex MCP Server

by SourikAdhikary
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

Vindex (Ingesta)

Turn any video into searchable moments, keyframes, and AI-ready datasets—100% locally. Give AI agents (Claude Code, Cursor, MCP) eyes and ears over your video library with zero cloud dependencies.

Why Vindex?

Most video tools either choke on long-form video, recompute expensive transcription/vision stages from scratch when you tweak a parameter, or require a heavy distributed stack (Docker + Redis + Celery + Qdrant + Postgres).

Vindex (Ingesta) compiles raw local video files, folders, or URLs into deterministic, content-addressed VideoBundles using a single-node SQLite WAL control plane:

  • 8-Op Content-Addressed Merkle DAG: fetch → probe ∥ audio ∥ scenes → transcribe ∥ frames → embed → bundle. Changing embed_model reuses cached probe, audio, scenes, transcribe, and frames in <5ms per op (INV-07).

  • Crash-Only Worker Pool & Lease Reaper: BEGIN IMMEDIATE … RETURNING atomic job claims, heartbeated leases, background lease reaper, and process-group (os.setsid / os.killpg) subprocess isolation (INV-01..INV-10).

  • Supervised Warm ML Sidecar: Keeps heavy ML weights (faster-whisper, open_clip) warm across jobs over a local socket while completely isolating the control plane from OOM/segfault crashes.

  • Perceptual Keyframe Dedup + Contact-Sheet Filmstrips: 64-bit color-aware dHash intra-shot deduplication selects (start, medoid, end) keyframes and stitches 3-tile contact-sheet filmstrips for VLM temporal grounding.

  • Hybrid Multimodal Retrieval (RRF + Temporal MMR): Fuses sqlite-vec visual KNN over shot keyframes with SQLite FTS5 BM25 over exact + ±1.5s speech-to-shot windows, deduplicated via Temporal Maximal Marginal Relevance.

  • Zero-Config Interfaces:

    • CLI (vindex): ingest, search, inspect, export, status, doctor, gc, chaos, server, mcp.

    • MCP Server (vindex mcp): 4 JSON-RPC 2.0 tools (ingest_video, search_moments, get_shot_context, extract_clip) for Claude Code, Cursor, and Antigravity.

    • FastAPI + Live HTMX Dashboard (vindex server): Real-time DAG telemetry and a live ⚡ CHAOS: SIGKILL Random Worker button.

    • Dataset Exporters: WebDataset (.tar), Apache Parquet (.parquet), JSONL (.jsonl), and Obsidian/Notion Markdown notes (.md).

Related MCP server: klaket-mcp

Quickstart

# 1. Install into Python 3.12 virtualenv
uv venv --python 3.12 .venv
uv pip install -e ".[dev]"

# 2. Ingest a video (or directory of videos)
uv run vindex ingest /path/to/video.mp4

# 3. Run hybrid multimodal search (RRF + Temporal MMR)
uv run vindex search "architecture diagram" --mode hybrid --top-k 5

# 4. Verify all 10 formal system invariants (INV-01 .. INV-10)
uv run vindex doctor

# 5. Export bundle to WebDataset, Parquet, JSONL, or Markdown
uv run vindex export <video_sha_prefix> --format parquet --out ./exports

# 6. Launch the FastAPI Control Plane & Live HTMX Dashboard
uv run vindex server --port 8000

Repository Layout

See docs/ARCHITECTURE.md for the full 6-layer architecture (core → models → engine → ops → index & exporters → interfaces), the 8-op content-addressed Merkle DAG, and the 10 formal system invariants (INV-01–INV-10), and see CONTRIBUTING.md for development guidelines.

make check      # Run Ruff linter across src/ and tests/
make test-all   # Run all unit, integration, and chaos tests
make bench      # Run the 3-part reproducible systems benchmark suite

Available Tools

5 tools
get_ingest_statusB

Get per-op DAG status and progress for an ingest ID.

NameRequiredDescriptionDefault
ingest_idYes

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. 'Get' implies read-only and it names the returned concepts (status, progress), but it says nothing about terminal states, whether polling is expected, rate limits, or what an unknown ingest_id yields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One front-loaded sentence with no filler. It earns its place but is borderline under-specified rather than purely concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, no annotations, and an undocumented parameter, the description is the agent's entire information source, and it omits what statuses/progress look like, how completion is signaled, and when to re-call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and the single parameter has no description in the schema. The description at least ties the parameter to an ingest ID, but gives no format, origin, or validity guidance, so it only partially compensates.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Get') and resource ('per-op DAG status and progress') scoped to an ingest ID. It is clearly distinguishable from siblings like ingest_video or get_transcript, though the jargon 'per-op DAG' is left unexplained.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit when-to-use, when-not, or named alternatives. Usage is only implied: you call this to check on an ingest, presumably after invoking ingest_video, but the description never says so or mentions polling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_shot_contextB

Agentic Zoom (VideoTree/VideoAgent): expand a shot to inspect adjacent neighbor shots, keyframe paths, filmstrips, and continuous transcript.

NameRequiredDescriptionDefault
shot_idYes
video_sha256Yes
neighbor_shotsNo

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the behavioral disclosure burden. It reveals the kinds of content returned, but does not explicitly state that the operation is read-only, whether permissions are required, or what limitations apply.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single front-loaded sentence with no redundant padding. The parenthetical 'VideoTree/VideoAgent' is somewhat jargon-heavy, but overall the structure is efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema and no annotations, so the description needs to compensate by explaining parameters and return behavior. It partially describes return contents but leaves all parameter semantics undocumented, making it incomplete for a three-parameter retrieval tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for three parameters, and the description does not explain video_sha256, shot_id, or neighbor_shots. It only hints at 'shot' and 'adjacent neighbor shots,' leaving parameter formats, required status, and the default neighbor count undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource: 'expand a shot' and inspect neighbor shots, keyframe paths, filmstrips, and transcript. It is clear what the tool does, but it does not explicitly differentiate itself from siblings such as get_transcript, which also provides transcript content.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by 'expand a shot to inspect...', which suggests retrieving richer context around a shot. However, it does not state when to use this tool instead of get_transcript or search_corpus, nor does it give exclusions or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_transcriptC

Retrieve timestamped transcript segments for a video within [t_start, t_end].

NameRequiredDescriptionDefault
t_endNo
t_startNo
video_sha256Yes

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With zero annotations, the description carries the full behavioral burden but discloses almost nothing: no time units (seconds vs milliseconds), no indication of what happens when the video has no transcript or an unfinished ingest, and no sense of return volume or truncation for long videos. Only the bracketed range convention hints at inclusive-range behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence with the resource front-loaded and the scope qualifier attached. Nothing is padded or redundant, though the brevity is partly the cause of the coverage gaps.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-parameter tool with no annotations, no output schema, and no parameter documentation, the description leaves too much unsaid: units, defaults behavior (the schema defaults t_start=0 and t_end=999999 suggest full-video retrieval, but the description never confirms this), and result shape are all absent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does name all three concepts (video, t_start, t_end) and imply an inclusive range. However, it never specifies units or format for the numeric bounds, nor what video_sha256 actually identifies or where an agent obtains one, leaving real ambiguity.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource pair ("Retrieve timestamped transcript segments") scoped to a single video and a time window, which is meaningfully more precise than the name alone. It does not explicitly contrast itself with sibling tools like search_corpus or get_shot_context, so the distinction must be inferred from the word "transcript".

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to use this versus search_corpus (corpus-wide search) or get_shot_context (visual context around a moment). No prerequisites are given, such as whether the video must already be ingested via ingest_video, despite get_ingest_status being a sibling that implies an async pipeline.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ingest_videoC

Ingest a local video file or YouTube URL into a searchable multimodal Vindex bundle.

NameRequiredDescriptionDefault
syncNo
embed_modelNofake
url_or_pathYesLocal file path or video URL
whisper_modelNofake

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It implies creation/indexing of a bundle but does not disclose side effects, whether ingestion is synchronous/asynchronous, permission requirements, duplicate handling, or expected duration.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no wasted words. It is as concise as possible while still stating the core action and target artifact.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a potentially complex ingestion tool with no annotations, no output schema, and 75% of parameters undocumented, the description is too thin. It omits behavioral details and parameter semantics needed to invoke the tool confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 25%, so the description needs to compensate. It clarifies the required url_or_path can be a local file or YouTube URL, but it says nothing about sync, embed_model, or whisper_model, leaving three parameters undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ("Ingest") and resource (video file or YouTube URL) and names the resulting artifact (searchable multimodal Vindex bundle). It is clear, but it does not explicitly differentiate itself from siblings like get_ingest_status or search_corpus.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains what the tool does but gives no guidance on when to use it versus alternatives, nor does it mention prerequisites, follow-up status-checking, or when to prefer another sibling tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_corpusA

Hybrid visual (CLIP/SigLIP) + lexical (BM25) RRF search across ingested videos. Returns ranked shots with timestamps, transcript, and filmstrip image paths.

NameRequiredDescriptionDefault
kNo
modeNohybrid
queryYes
video_filterNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden; it usefully discloses the ranked-shot return payload and the fusion method, giving some sense of result semantics. However, it omits the read-only/safety profile, the meaning of the default k, and any note that results are bounded by what has been ingested.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense sentences that front-load the search method and then the return contract, with no filler. Every clause adds information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The return format is covered despite no output schema, which is a plus. But with four parameters at 0% description coverage and no annotation context, the definition leaves key invocation details (k semantics, video_filter target, mode selection) underspecified for a search tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, yet it only loosely gestures at the `mode` values via 'visual (CLIP/SigLIP)' and 'lexical (BM25)' without tying them to the enum. The `k` and `video_filter` parameters are left entirely unexplained in both schema and description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (search) and resource (corpus of ingested videos) and names the hybrid technique (CLIP/SigLIP + BM25 RRF). It also discloses the return shape (ranked shots with timestamps/transcript/filmstrips), which clearly differentiates it from ingest_video, get_ingest_status, get_transcript, and get_shot_context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this is the retrieval tool among the siblings, but it never states when to prefer it, when to fall back to get_transcript or get_shot_context, or when video_filter is warranted. Usage is inferable from the corpus/search framing rather than explicitly guided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 5 tool updatesv0.1.0
    • First observedget_ingest_status
    • First observedget_shot_context
    • First observedget_transcript
    • First observedingest_video
    • First observedsearch_corpus
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

B3.4/5.0

Scored across 5 tools

Disambiguation4/5

The tools generally have distinct roles across ingest, status, search, shot inspection, and transcript retrieval. There is some overlap because search_corpus and get_shot_context both return transcript/filmstrip context, and get_transcript can be confused with transcript fields returned by other tools, but the descriptions make the intended boundaries reasonably clear.

Naming Consistency5/5

All tool names use consistent snake_case with a clear verb_noun or verb_noun_object pattern: ingest_video, get_ingest_status, search_corpus, get_shot_context, get_transcript. The naming is predictable and easy to infer.

Tool Count5/5

Five tools is well scoped for a video ingestion and retrieval server. Each tool covers a distinct stage of the workflow from ingestion through search and contextual inspection, with no obviously redundant tools.

Completeness4/5

The core lifecycle is covered: ingest, check status, search, inspect shot context, and retrieve transcripts. Minor gaps remain around corpus/video management, such as listing ingested videos or deleting/removing bundles, which agents may need for cleanup or discovery.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • vidsight

    szepix
    A
    license
    Not graded
    quality
    B
    maintenance
    Enables AI agents to query local video timelines by extracting speech, frame captions, and on-screen text into a SQLite store, exposing search and retrieval tools via MCP.
    PolyForm Noncommercial 1.0.0

  • klaket-mcp

    huseyinstif
    A
    license
    A
    quality
    D
    maintenance
    Let AI agents watch videos: local transcripts, speaker labels, scenes, chapters and exact-moment search from any video URL or file. Fully local, no API keys.
    4
    3
    AGPL 3.0

  • vidx

    renjithrrkj
    A
    license
    A
    quality
    C
    maintenance
    Enables AI agents to index videos locally once and then browse a compact text map of shots, transcript, silences and visual embeddings, so they can query moments, cut dead air, apply edit scripts, render with ffmpeg and run QA checks without spending tokens on frames. Pixels are only fetched when the agent explicitly peeks at labeled contact sheets.
    9
    MIT