Use this server to safely inspect local media and transcribe its audio offline.
Probe a local absolute media path with ffprobe for normalized metadata (duration, container, streams, ffprobe version, SHA-256, cache hit).
Transcribe the first audio stream with whisper.cpp and return timestamped segments plus durable JSON/Markdown transcript paths.
Enforce absolute paths and allowed read/write roots; transcript output directories must already exist.
Reuse content-addressed caches and report cache hits.
README also documents frame extraction with provenance, vu-doctor, and bounded/cancellable subprocess execution; the provided schema exposes only video_probe and video_transcribe.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@video-understanding-mcp Transcribe /home/user/video.mp4 and save transcript to /home/user/transcripts"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Local stdio MCP server for extracting bounded, reproducible evidence from video.
Early development: Milestones 1 through 3 provide safe video inspection, offline transcription, and timestamp-accurate visual evidence extraction.
The server provides:
video_probe, backed by ffprobe
video_transcribe, backed by FFmpeg and whisper.cpp
video_extract_frames, with source-PTS scene and cadence sampling
timestamped transcript JSON and Markdown written to caller-selected roots
absolute-path and allowed-root enforcement
symlink-escape protection
bounded, cancellable subprocess execution
content-addressed probe, transcript, and frame caching with retention limits
vu-doctor readiness checks for FFmpeg, FFprobe, and whisper.cpp
synthetic media fixtures and MCP stdio contract tests
Node.js 22 or newer
ffmpeg
ffprobe
whisper-cli
a local whisper.cpp GGML model
On macOS with Homebrew:
brew install whisper-cpp
npm run bootstrap:model
export VU_WHISPER_MODEL_PATH="$HOME/.local/share/video-understanding-mcp/models/ggml-large-v3-turbo-q5_0.bin"The bootstrap script downloads the official
ggml-large-v3-turbo-q5_0.bin
model and verifies its Git LFS SHA-256 digest,
394221709cd5ad1f40c46e6031ca61bce88931e6e088c188294c6d5a55ffa7e2,
before reporting the configuration value. A failed checksum leaves no accepted
model. Pass a different destination directory when required:
npm run bootstrap:model -- /absolute/model/directoryThe server never downloads a model or makes any other network request while processing media.
Related MCP server: ollos-mcp
git clone https://github.com/kwacky1/video-understanding-mcp.git
cd video-understanding-mcp
npm install
npm run build
VU_ALLOWED_READ_ROOTS="$HOME/Videos" \
VU_ALLOWED_WRITE_ROOTS="$HOME/Documents/transcripts" \
VU_WHISPER_MODEL_PATH="$HOME/.local/share/video-understanding-mcp/models/ggml-large-v3-turbo-q5_0.bin" \
npm startVU_ALLOWED_READ_ROOTS is a path-delimited list. If it is omitted, the server
allows reads only within its current working directory.
Optional configuration:
Variable | Default | Purpose |
| current directory | Readable filesystem roots |
| readable roots | Writable transcript roots |
|
| Maximum accepted input size |
|
| FFprobe executable |
|
| FFmpeg executable |
|
| whisper.cpp executable |
| none | Absolute path to a local GGML model |
| platform cache directory | Content-addressed cache root |
|
| Maximum cache entry age |
|
| Maximum total cache size with LRU eviction |
Both root variables are path-delimited lists (: on macOS/Linux, ; on
Windows). Output directories must already exist, resolve inside a configured
writable root, and be writable.
video_transcribe accepts:
{
"path": "/absolute/path/demo.mp4",
"output_dir": "/absolute/path/transcripts",
"language": "en"
}It extracts the first audio stream to a temporary 16 kHz mono WAV, runs
whisper-cli locally, and removes scratch files on success, failure, or
cancellation. The result includes timestamped segments and durable
transcript_json_path and transcript_markdown_path values.
The transcript cache key includes the input SHA-256, transcription schema,
canonical parameters, whisper executable fingerprint, and model SHA-256.
Repeating the same call reuses cached transcript artefacts and returns
cache_hit: true; changing the input, language, model, or executable
configuration invalidates the cache.
video_extract_frames accepts:
{
"path": "/absolute/path/demo.mp4",
"output_dir": "/absolute/path/evidence",
"interval_seconds": 10,
"scene_threshold": 0.4,
"max_frames": 24,
"return_inline": false
}The sampler combines the first frame, source-time cadence, scene changes, and
FFmpeg mpdecimate near-duplicate removal. It never uses the timestamp-
rewriting fps filter. The server records every selected frame's source PTS,
timebase, duration, selection reason, and SHA-256 in a durable
provenance.json file.
max_frames cannot exceed 24. Inline images are disabled by default; when
requested, no more than four JPEG thumbnails are returned and their longest
edge is at most 512 pixels. Full extracted frames remain in the durable output
directory.
All cache stages are pruned to the configured age and total-size limits. Recent cache hits update the entry timestamp used for least-recently-used eviction.
npm run doctor
npm run doctor -- --jsonThe doctor exits non-zero until all three executables are ready. Whisper is reported now so Milestone 2 prerequisites are visible before transcription is added.
Inspect a local media file with ffprobe and return normalised metadata. The path must be absolute and inside an allowed read root.
Inputs: path (required). Returns: streams, cache_hit, container, duration_ms, +3 more
Transcribe the first audio stream of a local media file with whisper.cpp. Processing is offline and writes timestamped JSON and Markdown into an allowed output directory.
Inputs: path (required), output_dir (required), language. Returns: engine, language, segments, cache_hit, +5 more
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Scored across 2 tools
The two tools have completely distinct purposes: video_probe inspects metadata, while video_transcribe performs audio transcription. There is no overlap or ambiguity between them.
Both tool names follow the same verb_noun snake_case pattern (video_probe, video_transcribe), with the action first and the domain prefix consistent. The naming is uniform and predictable.
With only two tools, the server feels thin for the broad domain of video understanding. The count is borderline but not extreme, as both operations are relevant and non-trivial.
The tool surface is severely incomplete for a server named 'video-understanding'. It only covers two basic operations: metadata inspection and speech transcription. There are no tools for common video-understanding tasks such as scene detection, object recognition, action classification, or even audio extraction.
Transcode, host and caption video from a prompt. Fifteen tools, nine read-only, nothing deletes.
Transcribe YouTube via Whisper. Summaries, chapters, semantic-search across your corpus.
YouTube captions (chunks, timestamps), PDFs (page numbers) and web pages as clean Markdown.
Transform video, audio and images, and generate media from prompts. FFmpeg, captions, models.
