MCP Video Parser
A powerful video analysis system that uses the Model Context Protocol (MCP) to process, analyze, and query video content using AI vision models.
🎬 Features
- AI-Powered Video Analysis: Automatically extracts and analyzes frames using vision LLMs (Llava)
- Natural Language Queries: Search videos using conversational queries
- Time-Based Search: Query videos by relative time ("last week") or specific dates
- Location-Based Organization: Organize videos by location (shed, garage, etc.)
- Audio Transcription: Extract and search through video transcripts
- Chat Integration: Natural conversations with Mistral/Llama while maintaining video context
- Scene Detection: Intelligent frame extraction based on visual changes
- MCP Protocol: Standards-based integration with Claude and other MCP clients
🚀 Quick Start
Prerequisites
- Python 3.10+
- Ollama installed and running
- ffmpeg (for video processing)
Installation
- Clone the repository:
git clone https://github.com/michaelbaker-dev/mcpVideoParser.git
cd mcpVideoParser
- Install dependencies:
pip install -r requirements.txt
- Pull required Ollama models:
ollama pull llava:latest # For vision analysis
ollama pull mistral:latest # For chat interactions
- Start the MCP server:
python mcp_video_server.py --http --host localhost --port 8000
Basic Usage
- Process a video:
python process_new_video.py /path/to/video.mp4 --location garage
- Start the chat client:
python standalone_client/mcp_http_client.py --chat-llm mistral:latest
- Example queries:
- "Show me the latest videos"
- "What happened at the garage yesterday?"
- "Find videos with cars"
- "Give me a summary of all videos from last week"
🏗️ Architecture
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Video Files │────▶│ Video Processor │────▶│ Frame Analysis │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ MCP Server │◀────│ Storage Manager │◀────│ Ollama LLM │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│
▼
┌─────────────────┐
│ HTTP Client │
└─────────────────┘
🛠️ Configuration
Edit config/default_config.json to customize:
- Frame extraction rate: How many frames to analyze
- Scene detection sensitivity: When to capture scene changes
- Storage settings: Where to store videos and data
- LLM models: Which models to use for vision and chat
See Configuration Guide for details.
The server exposes these MCP tools:
process_video - Process and analyze a video filequery_location_time - Query videos by location and timesearch_videos - Search video content and transcriptsget_video_summary - Get AI-generated summary of a videoask_video - Ask questions about specific videosanalyze_moment - Analyze specific timestamp in a videoget_video_stats - Get system statisticsget_video_guide - Get usage instructions
🛠️ Utility Scripts
Video Cleanup
Clean all videos from the system and reset to a fresh state:
# Dry run to see what would be deleted
python clean_videos.py --dry-run
# Clean processed files and database (keeps originals)
python clean_videos.py
# Clean everything including original video files
python clean_videos.py --clean-originals
# Skip confirmation and backup
python clean_videos.py --yes --no-backup
This script will:
- Remove all video entries from the database
- Delete all processed frames and transcripts
- Delete all videos from the location-based structure
- Optionally delete original video files
- Create a backup of the database before cleaning (unless
--no-backup)
Video Processing
Process individual videos:
# Process a video with automatic location detection
python process_new_video.py /path/to/video.mp4
# Process with specific location
python process_new_video.py /path/to/video.mp4 --location garage
📖 Documentation
🚦 Development
Running Tests
# All tests
python -m pytest tests/ -v
# Unit tests only
python -m pytest tests/unit/ -v
# Integration tests (requires Ollama)
python -m pytest tests/integration/ -v
Project Structure
mcp-video-server/
├── src/
│ ├── llm/ # LLM client implementations
│ ├── processors/ # Video processing logic
│ ├── storage/ # Database and file management
│ ├── tools/ # MCP tool definitions
│ └── utils/ # Utilities and helpers
├── standalone_client/ # HTTP client implementation
├── config/ # Configuration files
├── tests/ # Test suite
└── video_data/ # Video storage (git-ignored)
🤝 Contributing
We welcome contributions! Please see CONTRIBUTING.md for guidelines.
📝 Roadmap
- ✅ Basic video processing and analysis
- ✅ MCP server implementation
- ✅ Natural language queries
- ✅ Chat integration with context
- 🚧 Enhanced time parsing (see INTELLIGENT_QUERY_PLAN.md)
- 🚧 Multi-camera support
- 🚧 Real-time processing
- 🚧 Web interface
🐛 Troubleshooting
Common Issues
- Ollama not running:
ollama serve # Start Ollama
- Missing models:
ollama pull llava:latest
ollama pull mistral:latest
- Port already in use:
# Change port in command
python mcp_video_server.py --http --port 8001
📄 License
MIT License - see LICENSE for details.
🙏 Acknowledgments
- Built on FastMCP framework
- Uses Ollama for local LLM inference
- Inspired by the Model Context Protocol specification
💬 Support
Version: 0.1.1
Author: Michael Baker
Status: Beta - Breaking changes possible