MCP-Powered Video RAG
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@MCP-Powered Video RAGWhat are the main topics discussed in my lecture video?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
🎬 MCP-Powered Video RAG
Ask natural language questions about your videos — powered by Whisper + ChromaDB + Groq + FastMCP. Fully free and runs locally.
Architecture & How It Works

Your Video → Whisper (transcription) → ChromaDB (vector store)
↓
Antigravity IDE ← MCP Server ← Your Question
↓
Groq LLM (free) → Answer + TimestampsRelated MCP server: video-agent-mcp
Stack (All Free)
Component | Tool |
Transcription | OpenAI Whisper (local) |
Embeddings | sentence-transformers (local) |
Vector DB | ChromaDB (local) |
MCP Framework | FastMCP |
LLM for Q&A | Groq (free tier) |
⚡ Quick Start
1. Install dependencies
uv sync2. Get your FREE Groq API key
Go to https://console.groq.com/ → create account → copy API key.
3. Set up environment
copy .env.example .env
# Edit .env and paste your GROQ_API_KEY4. Add the MCP server to Antigravity IDE
Copy mcp_config.json contents into your Antigravity IDE MCP settings.
Update GROQ_API_KEY with your actual key.
5. Place videos in the videos/ folder
Supports: .mp4, .mkv, .avi, .mov, .webm, .mp3, .wav
🛠️ Available MCP Tools
Tool | Description |
| Transcribe & index a video file |
| Semantic search over transcripts |
| Full RAG Q&A with timestamps |
| Show all indexed videos |
| Remove a video from the index |
📖 Example Usage (in Antigravity IDE)
ingest_video("videos/my_lecture.mp4")
ask_video("What are the main topics discussed?")
search_video("neural networks explained", n_results=3)
ask_video("What did the speaker say about backpropagation?", video_name="my_lecture.mp4")⚙️ Configuration (.env)
GROQ_API_KEY=your_key_here # Required — get free at console.groq.com
WHISPER_MODEL=base # tiny | base | small | medium | large
EMBEDDING_MODEL=all-MiniLM-L6-v2 # local embedding model
GROQ_MODEL=llama-3.1-8b-instant # Groq model for Q&A
CHROMA_DB_PATH=./chroma_db # where to persist the vector DB💡 Tips
Use
WHISPER_MODEL=tinyfor fastest transcription (less accurate)Use
WHISPER_MODEL=mediumfor high accuracy (slower)The first run downloads Whisper and embedding models (~200MB each) — subsequent runs are instant
ChromaDB data persists across restarts in
./chroma_db/
Available Tools
5 toolsask_videoA
Ask a natural language question about your videos and get an AI-generated answer.
This tool uses RAG: it retrieves the most relevant transcript chunks from ChromaDB, then sends them as context to a Groq LLM (free tier) to generate a precise answer with timestamps so you can jump directly to the relevant moment in the video.
| Name | Required | Description | Default |
|---|---|---|---|
| question | Yes | Your question in plain English (e.g. "What are the three main topics discussed?") | |
| video_name | No | Optional — restrict to a specific video file (e.g. "tutorial.mp4") | |
| n_context_chunks | No | Number of transcript chunks to use as context (default: 4) |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full behavioral burden and does a genuinely useful job: it discloses the RAG retrieval step (ChromaDB transcript chunks), that an external Groq LLM on a free tier generates the answer, and that responses include timestamps for jumping to the moment. That is real operational context beyond the schema, though it omits permission/auth needs and any rate-limit consequences implied by 'free tier'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short paragraphs, with the core purpose front-loaded in sentence one and pipeline detail following. The implementation specifics (ChromaDB, Groq free tier) are borderline but do serve transparency; nothing is repetitive or padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need no explanation, and the description covers purpose, mechanism, and the timestamped nature of answers. The main remaining gap is the unresolved relationship to the sibling search_video, which an agent selecting between them would want resolved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all three parameters (question, video_name, n_context_chunks) are already documented in the schema, including defaults and examples. The description adds no syntax, format, or tuning guidance for them, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: ask a natural-language question about your videos and receive an AI-generated answer. It clearly distinguishes an answering/synthesis tool from pure retrieval, but it never names or contrasts with the closest sibling, search_video, so the boundary between them must be inferred.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the phrasing 'get an AI-generated answer' versus search_video's likely raw-results behavior, but there is no explicit when-to-use, when-not-to-use, or alternative-naming guidance. An agent has to guess whether to reach for this or search_video when both would accept a question about a video.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_videoA
Remove a video and all its indexed data from the RAG system.
This deletes all transcript chunks for the specified video from ChromaDB. The original video file is NOT deleted — only the indexed data is removed.
| Name | Required | Description | Default |
|---|---|---|---|
| video_path | Yes | Path to the video file (same path used during ingestion). |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does useful work: it names the backing store (ChromaDB transcript chunks) and explicitly limits blast radius by stating the original video file is NOT deleted. It still omits irreversibility/reversibility, required permissions, and any side effects on in-flight searches.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, zero filler, and the most important constraint (file preserved) is front-loaded rather than buried.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return values need not be explained, and the destructive scope is well covered for a one-parameter tool. Only the irreversibility/permission angle is left open, which is a minor gap for a delete operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single video_path parameter, and the description adds no format or syntax detail beyond 'the specified video'. Baseline 3 applies when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (remove/delete) and resource (video + indexed data), and immediately disambiguates the scope: indexed data goes, the source file stays. An agent can distinguish this from ingest_video, search_video, ask_video and list_videos from the description alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use framing, no prerequisites, and no alternatives are named (e.g., when to delete vs. re-ingest, or whether removal requires re-indexing afterward). Usage must be inferred entirely from the verb.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
ingest_videoA
Ingest a video file into the RAG system.
This transcribes the video using Whisper (locally, for free), splits the transcript into timed chunks, embeds them, and stores them in a local ChromaDB vector database for semantic search.
| Name | Required | Description | Default |
|---|---|---|---|
| video_path | Yes | Absolute or relative path to the video file. Supports .mp4, .mkv, .avi, .mov, .webm, .mp3, .wav, etc. |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does disclose meaningful traits: local execution, no API cost, and persistent writes to a ChromaDB vector store. It stops short of mutation-relevant behavior — whether re-ingesting the same file duplicates entries, whether it overwrites, expected runtime, or failure behavior on unsupported media.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the purpose in the first sentence, then expands the pipeline in three short lines. Minor filler in the parenthetical '(locally, for free)', but overall tight and easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return values need not be explained, and the sole parameter is fully documented. The remaining gap is side-effect/idempotency information for an unannotated mutating tool, which an agent arguably needs before calling it repeatedly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter exists and schema description coverage is 100%, so the schema already documents video_path and its supported extensions. The description adds nothing beyond that, which matches the baseline 3 when structured data does the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Ingest a video file into the RAG system') and then names the concrete pipeline (Whisper transcription, timed chunking, embedding, ChromaDB storage). An agent can immediately tell this is the write/ingest counterpart to the read-oriented siblings, though no sibling is named explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the usage context — you must ingest before search_video/ask_video can find anything — but never states that sequencing, nor any prerequisites (file must exist on disk, one-time indexing cost, what to do with already-ingested files). No when-not guidance or alternatives are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_videosB
List all videos currently indexed in the RAG system.
Returns a summary of each ingested video including its name, path, and chunk count.
Returns: JSON list of indexed videos with metadata.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the behavioral burden. It implies a read-only inventory operation and describes the per-item fields, but says nothing about ordering, pagination, result limits, or behavior when the index is empty — meaningful gaps for an unannotated tool, though minor given the trivial zero-parameter surface.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The first sentence is well front-loaded, but the second and third sentences restate the same return information twice ('Returns a summary of each ingested video...' followed by 'Returns: JSON list of indexed videos with metadata'). One of those should be cut.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so the description is not obligated to describe return values, and the tool has no parameters — the definition is therefore close to sufficient. However, it omits ordering/scope details (does 'all' mean truncated or paginated?) that an agent would need to interpret the result set.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes no parameters, so there is no parameter semantics to document and the baseline of 4 applies. The description adds nothing parameter-related, which is appropriate here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (list) and resource (videos indexed in the RAG system), plus what each entry contains (name, path, chunk count). It does not explicitly contrast itself with siblings like search_video or ingest_video, but the distinction is clear enough from the verb alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: an agent can infer this is the tool for enumerating everything currently indexed rather than querying it. There is no explicit statement of when to prefer this over search_video or ask_video, and no exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_videoB
Semantically search across all indexed video transcripts.
Returns the most relevant timestamped transcript chunks for a given query. You can optionally filter to a specific video by name.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | Natural language search query (e.g. "what did they say about pricing?") | |
| n_results | No | Number of results to return (default: 5, max recommended: 10) | |
| video_name | No | Optional filename to restrict search to one video (e.g. "lecture.mp4") |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does disclose the return shape ("timestamped transcript chunks") which is useful. However, it omits whether results are ranked, pagination/truncation behavior, and the requirement that the video be indexed first.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, front-loaded with the core action and immediately followed by the return value. Little waste, though the optional-filter sentence is slight filler given the schema already documents video_name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return values need not be explained, and the description covers the core operation. It still lacks prerequisites (indexing state) and any routing hint against ask_video, leaving an agent to guess which tool answers a transcript query.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all three parameters are already documented in the schema, making 3 the baseline. The description adds only a light restatement of the video_name filter and does not clarify query syntax or ranking/pagination semantics beyond what the schema says.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and scope: "Semantically search across all indexed video transcripts." That is clear and concrete, but it does not differentiate this tool from the sibling ask_video, which an agent could easily confuse with a semantic transcript search.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description says the video_name filter is optional but gives no when-to-use guidance, no prerequisites (e.g. transcripts must already be indexed via ingest_video), and no comparison to ask_video or list_videos. Callers must infer the search-vs-answer distinction themselves.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
5 tool updates
v0.1.0- First observed
ask_video - First observed
delete_video - First observed
ingest_video - First observed
list_videos - First observed
search_video
TDQS
Scored across 5 tools
Each tool maps to a distinct action (ingest, search, ask, list, delete). The only potential overlap is search_video vs ask_video, since both retrieve transcript chunks, but the descriptions clearly differentiate raw chunk retrieval from LLM-generated answers with context.
All five tools follow a strict verb_noun snake_case pattern (ingest_video, search_video, ask_video, list_videos, delete_video). The convention is predictable and uniform.
Five tools is well-scoped for a video RAG system. Each tool covers a meaningful, non-redundant part of the workflow with no filler.
The surface covers the full lifecycle: ingestion, two retrieval modes (search and Q&A), listing indexed content, and deletion. Re-ingesting a video handles the update case, so there are no obvious dead ends.
Maintenance
Related MCP Connectors
Transcribe YouTube via Whisper. Summaries, chapters, semantic-search across your corpus.
MCP server for Clipkit — gives AI agents a video toolbox via the Clipkit schema.
Multimodal video analysis MCP — transcription, vision, and OCR for any video URL.
YouTube transcripts, search, channel browsing, and playlists for AI agents via MCP.
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to query local video timelines by extracting speech, frame captions, and on-screen text into a SQLite store, exposing search and retrieval tools via MCP.PolyForm Noncommercial 1.0.0
- AlicenseNot gradedqualityCmaintenanceEnables automated video learning workflows by ingesting video URLs, managing remote GPU ASR transcription, pulling transcripts, generating digest summaries, and searching local notes via MCP tools.MIT
- AlicenseAqualityCmaintenanceEnables MCP clients to transcribe audio/video files locally, generate SRT subtitles, and burn captions into videos via tool calls, without a cloud API.3MIT
- AlicenseNot gradedqualityAmaintenanceEnables AI agents to search, retrieve, transcribe, and summarize a local audio/video library over MCP, with timestamps and saved reports.4Apache 2.0