# Qwen Video Understanding MCP Server
An MCP (Model Context Protocol) server that enables Claude and other AI agents to analyze videos and images using **Qwen3-VL** deployed on Modal.
## Highlights
- **Hours-long video** support with full recall
- **Timestamp grounding** - second-level precision
- **256K context** (expandable to 1M)
- **32-language OCR** support
- **Free/self-hosted** on Modal serverless GPU
## Features
- **Video Analysis**: Analyze videos via URL with custom prompts
- **Image Analysis**: Analyze images via URL
- **Video Summarization**: Generate brief, standard, or detailed summaries
- **Text Extraction**: Extract on-screen text and transcribe speech
- **Video Q&A**: Ask specific questions about video content
- **Frame Comparison**: Analyze changes and progression in videos
## Architecture
```
Claude/Agent → MCP Server → Modal API → Qwen3-VL (GPU)
```
The MCP server acts as a bridge between Claude and your Qwen2.5-VL model deployed on Modal's serverless GPU infrastructure.
## Prerequisites
1. **Modal Account**: Sign up at [modal.com](https://modal.com)
2. **Deployed Qwen Model**: Deploy the video understanding model to Modal (see below)
3. **Python 3.10+**
## Quick Start
### 1. Deploy the Model to Modal (if not already done)
```bash
cd ~/qwen-video-modal
modal deploy qwen_video.py
```
### 2. Install the MCP Server
```bash
cd ~/qwen-video-mcp-server
pip install -e .
```
Or with uv:
```bash
uv pip install -e .
```
### 3. Configure Environment
```bash
cp .env.example .env
# Edit .env with your Modal workspace name
```
### 4. Add to Claude Desktop
Add to your Claude Desktop config (`~/Library/Application Support/Claude/claude_desktop_config.json`):
```json
{
"mcpServers": {
"qwen-video": {
"command": "uv",
"args": [
"--directory",
"/Users/adamanz/qwen-video-mcp-server",
"run",
"server.py"
],
"env": {
"MODAL_WORKSPACE": "adam-31541",
"MODAL_APP": "qwen-video-understanding"
}
}
}
}
```
### 5. Restart Claude Desktop
The `qwen-video` tools should now be available.
## Available Tools
### `analyze_video`
Analyze a video with a custom prompt.
```
analyze_video(
video_url="https://example.com/video.mp4",
question="What happens in this video?",
max_frames=16
)
```
### `analyze_image`
Analyze an image with a custom prompt.
```
analyze_image(
image_url="https://example.com/image.jpg",
question="Describe this image"
)
```
### `summarize_video`
Generate a video summary in different styles.
```
summarize_video(
video_url="https://example.com/video.mp4",
style="detailed" # brief, standard, or detailed
)
```
### `extract_video_text`
Extract text and transcribe speech from a video.
```
extract_video_text(
video_url="https://example.com/presentation.mp4"
)
```
### `video_qa`
Ask specific questions about a video.
```
video_qa(
video_url="https://example.com/video.mp4",
question="How many people appear in this video?"
)
```
### `compare_video_frames`
Analyze changes throughout a video.
```
compare_video_frames(
video_url="https://example.com/timelapse.mp4",
comparison_prompt="How does the scene change?"
)
```
### `check_endpoint_status`
Check the Modal endpoint configuration.
### `list_capabilities`
List all server capabilities and supported formats.
## Configuration
| Environment Variable | Description | Default |
|---------------------|-------------|---------|
| `MODAL_WORKSPACE` | Your Modal workspace/username | `adam-31541` |
| `MODAL_APP` | Name of the Modal app | `qwen-video-understanding` |
| `QWEN_IMAGE_ENDPOINT` | Override image endpoint URL | Auto-generated |
| `QWEN_VIDEO_ENDPOINT` | Override video endpoint URL | Auto-generated |
## Supported Formats
**Video**: mp4, webm, mov, avi, mkv
**Image**: jpg, jpeg, png, gif, webp, bmp
## Limitations
- Videos must be accessible via public URL
- Maximum 64 frames extracted per video
- Recommended video length: under 10 minutes for best results
- First request may have cold start delay (Modal serverless)
## Cost
The Modal backend uses A100-40GB GPUs:
- ~$3.30/hour while processing
- Scales to zero when idle (no cost)
- Only charged for actual processing time
## Troubleshooting
### "Request timed out"
- Video may be too large
- Try a shorter video or reduce `max_frames`
### "HTTP error 502/503"
- Modal container is starting up (cold start)
- Wait a few seconds and retry
### "Video URL not accessible"
- Ensure the URL is publicly accessible
- Check for authentication requirements
## Development
```bash
# Install dev dependencies
pip install -e ".[dev]"
# Run tests
pytest
```
## License
MIT
