Video RAG MCP Server
The Video RAG MCP Server enables video-based retrieval-augmented generation through semantic search and video processing capabilities:
Ingest video data: Load and process video files from a specified directory into the Ragie index, including audio-video analysis for indexing
Perform semantic search: Query indexed video content using natural language to retrieve relevant text excerpts with timestamps and source file information
Extract video chunks: Create and save specific video segments from original videos based on document name, start time, and end time
AI assistant integration: Provide these capabilities as MCP tools (
ingest_data_tool,retrieve_data_tool,show_video_tool) for use in Cursor IDE and other MCP-compatible environments
Provides environment variable management for storing and accessing API keys securely.
Allows cloning and management of the video RAG project repository.
Enables issue tracking and repository management for the video RAG project.
Enables interactive development and experimentation with video RAG capabilities through notebook support.
Integrates with Python 3.12 or higher for video processing and analysis capabilities.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Video RAG MCP Server find the part where they explain quantum computing concepts"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Video RAG (Retrieval-Augmented Generation)
A powerful video retrieval and analysis system that uses the Ragie API to process, index, and query video content with natural language. This project enables semantic search through video content, extracting relevant video chunks based on text queries.
π― MCP-powered video-RAG using Ragie
This project demonstrates how to build a video-based Retrieval Augmented Generation (RAG) system powered by the Model Context Protocol (MCP). It uses Ragie's video ingestion and retrieval capabilities to enable semantic search and Q&A over video content and integrate them as MCP tools via Cursor IDE.
Tech Stack
Ragie for video ingestion + retrieval (video-RAG)
Cursor as the MCP host
Model Context Protocol (MCP) for AI assistant integration
Related MCP server: YouTube MCP Server
π― Features
Video Processing: Upload and process video files with audio-video analysis
Semantic Search: Query video content using natural language
Video Chunking: Extract specific video segments based on search results
MCP Integration: Model Context Protocol (MCP) server for AI assistant integration
Jupyter Notebook Support: Interactive development and experimentation
Automatic Indexing: Clear and rebuild video indexes as needed
π Quick Start
Prerequisites
Python 3.12 or higher
Ragie API key
Video files to process
Cursor IDE (for MCP integration)
Setup and Installation
1. Install uv
First, let's install uv and set up our Python project and environment:
MacOS/Linux:
curl -LsSf https://astral.sh/uv/install.sh | shWindows:
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"2. Clone and Setup Project
# Clone the repository
git clone https://github.com/FiloHany/Video_RAG_MCP.git
cd video_rag
# Create virtual environment and activate it
uv venv
source .venv/bin/activate # MacOS/Linux
# OR
.venv\Scripts\activate # Windows
# Install dependencies
uv sync3. Configure Environment Variables
Create a .env file in the project root:
RAGIE_API_KEY=your_ragie_api_key_here4. Add Your Video Files
Place your video files in the video/ directory.
MCP Server Setup with Cursor IDE
1. Configure MCP Server in Cursor
Go to Cursor settings
Select MCP Tools
Add new global MCP server
In the JSON configuration, add:
{
"mcpServers": {
"ragie": {
"command": "uv",
"args": [
"--directory",
"/absolute/path/to/project_root",
"run",
"server.py"
],
"env": {
"RAGIE_API_KEY": "YOUR_RAGIE_API_KEY"
}
}
}
}Note: Replace /absolute/path/to/project_root with the actual absolute path to your project directory.
2. Connect MCP Server
In Cursor MCP settings, make sure to toggle the button to connect the server to the host
You should now see the MCP server listed in the MCP settings
3. Available MCP Tools
Your custom MCP server provides 3 tools:
ingest_data_tool: Ingests the video data to the Ragie indexretrieve_data_tool: Retrieves relevant data from the video based on user queryshow_video_tool: Creates a short video chunk from the specified segment from the original video
You can now ingest your videos, retrieve relevant data and query it all using the Cursor Agent. The agent can even create the desired chunks from your video just with a single query!
π Usage
Basic Usage
Run the main script to process videos and perform queries:
python main.pyThis will:
Clear the existing index
Ingest all videos from the
video/directoryPerform a sample query
Interactive Development
Use the Jupyter notebook for interactive development:
jupyter notebook video_rag.ipynbMCP Server
Start the MCP server for AI assistant integration:
python server.pyπ§ API Reference
Core Functions
clear_index()
Removes all documents from the Ragie index.
ingest_data(directory: str)
Processes and uploads all video files from the specified directory to the Ragie index.
Parameters:
directory(str): Path to the directory containing video files
retrieve_data(query: str)
Performs semantic search on the indexed video content.
Parameters:
query(str): Natural language query to search for in video content
Returns:
List of dictionaries containing:
text: The retrieved text contentdocument_name: Name of the source video filestart_time: Start timestamp of the video segmentend_time: End timestamp of the video segment
chunk_video(document_name: str, start_time: float, end_time: float, directory: str = "videos")
Extracts a specific video segment and saves it as a new file.
Parameters:
document_name(str): Name of the source video filestart_time(float): Start time in secondsend_time(float): End time in secondsdirectory(str): Directory containing the source video (default: "videos")
Returns:
Path to the created video chunk file
MCP Tools
The project includes an MCP server with the following tools:
ingest_data_tool(directory: str)
MCP wrapper for the ingest_data function.
retrieve_data_tool(query: str)
MCP wrapper for the retrieve_data function.
show_video_tool(document_name: str, start_time: float, end_time: float)
MCP wrapper for the chunk_video function.
π Examples
Example 1: Basic Video Processing and Query
from main import clear_index, ingest_data, retrieve_data
# Clear existing index
clear_index()
# Ingest videos from directory
ingest_data("video")
# Query the video content
results = retrieve_data("What is the main topic of the video?")
print(results)Example 2: Extract Video Chunks
from main import retrieve_data, chunk_video
# Get search results
results = retrieve_data("Show me the goal scoring moments")
# Extract video chunks for each result
for result in results:
if result['start_time'] and result['end_time']:
chunk_path = chunk_video(
result['document_name'],
result['start_time'],
result['end_time']
)
print(f"Created chunk: {chunk_path}")Example 3: Jupyter Notebook Workflow
# Load environment and initialize Ragie
import os
from dotenv import load_dotenv
from ragie import Ragie
load_dotenv()
ragie = Ragie(auth=os.getenv('RAGIE_API_KEY'))
# Upload a video
file_path = "video/messi-goals.mp4"
result = ragie.documents.create(request={
"file": {
"file_name": "messi-goals.mp4",
"content": open(file_path, "rb"),
},
"mode": {
"video": "audio_video"
}
})
# Query the video
response = ragie.retrievals.retrieve(request={
"query": "Give detailed description of the video with timestamp of the events"
})
# Process results
for chunk in response.scored_chunks:
print(f"Time: {chunk.metadata.get('start_time')} - {chunk.metadata.get('end_time')}")
print(f"Content: {chunk.text}")
print("-" * 50)ποΈ Project Structure
video_rag/
βββ main.py # Core functionality and main script
βββ server.py # MCP server implementation
βββ video_rag.ipynb # Jupyter notebook for development
βββ pyproject.toml # Project configuration and dependencies
βββ README.md # This file
βββ video/ # Directory for video files
β βββ messi-goals.mp4 # Example video file
βββ video_chunks/ # Output directory for video chunks (created automatically)π Environment Variables
Variable | Description | Required |
| Your Ragie API authentication key | Yes |
π¦ Dependencies
ragie: Video processing and retrieval API
moviepy: Video editing and manipulation
python-dotenv: Environment variable management
mcp: Model Context Protocol implementation
ipykernel: Jupyter notebook support
π€ Contributing
Fork the repository
Create a feature branch (
git checkout -b feature/amazing-feature)Commit your changes (
git commit -m 'Add some amazing feature')Push to the branch (
git push origin feature/amazing-feature)Open a Pull Request
π License
This project is licensed under the MIT License - see the LICENSE file for details.
π Acknowledgments
Ragie for providing the video processing API
MoviePy for video manipulation capabilities
MCP for AI assistant integration
π Support
If you encounter any issues or have questions:
Check the Issues page
Create a new issue with detailed information
Include your Python version, error messages, and steps to reproduce
π Changelog
v0.1.0
Initial release
Basic video processing and retrieval functionality
MCP server integration
Jupyter notebook support
Video chunking capabilities
Available Tools
3 toolsingest_data_toolB
Loads data from a directory into the Ragie index. Wait until the data is fully ingested before continuing.
Args:
directory (str): The directory to load data from.
Returns:
str: A message indicating that the data was loaded successfully.
Inputs: directory (required).
retrieve_data_toolB
Retrieves data from the Ragie index based on the query. The data is returned as a list of dictionaries, each containing the following keys:
- text: The text of the retrieved chunk
- document_name: The name of the document the chunk belongs to
- start_time: The start time of the chunk
- end_time: The end time of the chunk
Args:
query (str): The query to retrieve data from the Ragie index.
Returns:
list[dict]: The retrieved data.
Inputs: query (required).
show_video_toolB
Creates and saves a video chunk based on the document name, start time, and end time of the chunk.
Returns a message indicating that the video chunk was created successfully.
Args:
document_name (str): The name of the document the chunk belongs to
start_time (float): The start time of the chunk
end_time (float): The end time of the chunk
Returns:
str: A message indicating that the video chunk was created successfully
Inputs: document_name (required), start_time (required), end_time (required).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelogβ¦
TDQS
Scored across 3 tools
Each tool has a clearly distinct purpose with no overlap: ingest_data_tool loads data into an index, retrieve_data_tool queries the index for information, and show_video_tool generates video chunks from retrieved metadata. The descriptions clearly differentiate their functions, making misselection unlikely.
All tools follow a consistent verb_noun pattern with snake_case naming: ingest_data_tool, retrieve_data_tool, and show_video_tool. This predictable naming convention makes the tool set easy to understand and navigate.
With only 3 tools, the set feels thin for a video RAG system. While the core operations (ingest, retrieve, show) are covered, typical RAG workflows might benefit from additional tools like index management, query refinement, or batch processing. The count is borderline but functional.
The tools cover the essential RAG lifecycle: ingestion, retrieval, and video generation. However, there are minor gaps such as missing update/delete operations for the index, query history, or configuration tools. Agents can work around these, but the surface is not fully comprehensive.
Maintenance
Related MCP Connectors
Transcripts, summaries, chapters and timestamped search for video links, for AI agents.
VideoDB lets an AI agent watch, search and clip video.
Turn any video into context for AI agents: transcripts, metadata, chapters, timestamps.
YouTube transcripts, search, channels, playlists and bulk transcript jobs for AI agents. 14 tools.
Related MCP Servers
AlicenseAqualityDmaintenanceEnables AI agents to upload, index, search, and analyze videos through the Reka Vision API, supporting natural language search, visual question answering, and extraction of transcripts and captions.1727 PyPI1Apache 2.0- AlicenseNot gradedqualityDmaintenanceProvides tools for searching YouTube videos, retrieving transcripts, and performing semantic search over video content.MIT
- AlicenseNot gradedqualityDmaintenanceEnables LLM agents to process local videos into timestamped, citable text documents and then query them through tools for listing videos, retrieving transcripts, and fetching specific segments, all fully offline.21 PyPIMIT
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to read videos from YouTube links, web pages, media URLs, or local files by returning timestamped transcripts, labelled frame contact sheets, and lossless audio or spectrograms. Also searches YouTube, channels, pages, and folders to find videos worth reading.MIT