VideoScan MCP

An MCP (Model Context Protocol) server for comprehensive video analysis — AI-powered transcription, visual frame analysis, and metadata extraction from 1000+ platforms.

Features

• Full video analysis — combines transcription, frame extraction, and metadata in a single call

• AI vision analysis — describes frames and extracts on-screen text (OCR) using GPT-4o, Claude, or Gemini

• Audio transcription — Whisper-based transcription with timestamps and language detection

• Auto-tuning — automatically adjusts frame extraction density, interval, and detail level based on video duration

• Smart frame extraction — scene-change detection, interval sampling, or combined strategy

• Deduplication — perceptual hashing removes near-duplicate frames before analysis

• Metadata extraction — title, duration, chapters, tags, view count, and more without full download

• Multi-provider — OpenAI, Anthropic, and Google vision providers with per-request override

• Caching — persistent cache for downloads, frames, and results to minimize repeat costs

• 1000+ platforms — powered by yt-dlp (YouTube, Vimeo, Twitter/X, TikTok, and more)

Related MCP server: YOLO-FFMPEG-MCP

Installation

pip install videoscan-mcp

System dependencies

VideoScan requires ffmpeg for video processing and yt-dlp for downloading from URLs.

# macOS
brew install ffmpeg yt-dlp

# Ubuntu/Debian
apt install ffmpeg
pip install yt-dlp

# Windows — install ffmpeg from https://ffmpeg.org/download.html, then:
pip install yt-dlp

Configuration

Copy .env.example to .env and fill in at minimum one API key:

# Vision provider (frame analysis)
VISION_PROVIDER=openai          # openai | anthropic | google
VISION_MODEL=                   # optional — defaults: gpt-4o / claude-sonnet-4-20250514 / gemini-2.0-flash

# Transcription provider
TRANSCRIPTION_PROVIDER=openai   # openai only for now
TRANSCRIPTION_MODEL=whisper-1

# Concurrency
VISION_CONCURRENCY=5            # max parallel vision API calls

# API keys — only need the key for your chosen provider
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=AIza...

# Cache
CACHE_ENABLED=true
CACHE_DIR=~/.videoscan/cache
CACHE_MAX_SIZE_GB=5
CACHE_DOWNLOAD_TTL=3600         # 1 hour
CACHE_FRAMES_TTL=86400          # 24 hours
CACHE_RESULTS_TTL=604800        # 7 days

# Safety limits (set to 0 for unlimited)
MAX_VIDEO_DURATION=3600         # 60 minutes in seconds
MAX_DOWNLOAD_SIZE=2147483648    # 2 GB in bytes
MAX_ANALYZED_FRAMES=100
DOWNLOAD_TIMEOUT=300
FRAME_ANALYSIS_TIMEOUT=30

Quick Start — Claude Code

Add VideoScan to your Claude Code settings.json (usually at ~/.claude/settings.json):

{
  "mcpServers": {
    "videoscan": {
      "command": "videoscan",
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

Or using uvx without a global install:

{
  "mcpServers": {
    "videoscan": {
      "command": "uvx",
      "args": ["videoscan-mcp"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

Once connected, you can ask Claude things like:

• "Analyze this YouTube video: https://youtube.com/watch?v=..."

• "Transcribe the audio from this video file"

• "What's on screen at the 2:30 mark of this video?"

• "Extract frames from this video and describe what you see"

Auto-Tuning

When max_frames and interval are not explicitly set, VideoScan automatically adjusts frame extraction parameters based on video duration to optimize cost and coverage:

Short videos get dense frame extraction for maximum detail, while longer videos use lighter sampling to keep costs down. You can always override by setting max_frames or interval explicitly.

Tool Reference

analyze_video

Full pipeline — transcription + AI frame analysis + metadata in one call. Uses auto-tuning by default.

transcribe

Transcribe video or audio to text with timestamps.

extract_frames

Extract and AI-analyze frames from a video.

analyze_moment

Deep-dive analysis on a specific time range.

get_frame_at

Get a single frame at a specific timestamp, optionally analyzed by AI.

get_metadata

Fetch video metadata without downloading the full video.

Supported Platforms

VideoScan uses yt-dlp under the hood, which supports 1000+ video platforms including:

• YouTube, YouTube Shorts, YouTube Live

• Vimeo, Dailymotion, Twitch

• Twitter/X, Instagram, TikTok, Facebook

• Reddit, LinkedIn, Pinterest

• BBC iPlayer, CNN, NBC, CBS

• SoundCloud, Bandcamp (audio)

• And hundreds more — see the yt-dlp supported sites list

Local files in any format supported by ffmpeg (mp4, mov, avi, mkv, webm, mp3, wav, etc.) are also supported.

Cost Estimates

Costs depend on your chosen provider and usage:

A typical 10-minute video analyzed with analyze_video (30 frames + transcription) costs approximately $0.45–$0.51 with OpenAI.

Development

git clone https://github.com/guguborbh/videoscan-mcp
cd videoscan-mcp
pip install -e ".[dev]"
pytest

License

MIT License — see LICENSE for details.