video-storyboard-mcp
Enables processing Bilibili videos from URLs, including b23.tv and related CDN hosts, to generate storyboard sheets, aligned transcripts, zooms, subtitle strips, clips, and cache cleanup, with optional cookies for CC tracks and higher bitrates.
Enables processing YouTube videos from URLs to generate storyboard sheets, aligned transcripts, zooms, subtitle strips, clips, and cache cleanup, with optional cookies for platform CC tracks and higher bitrates.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
video-storyboard-mcp
Portable MCP server, CLI, and Antigravity plugin built on vendored SAOS-3624/dsh-video-storyboard (v0.7.6, Apache-2.0). Watch long local or online videos (Bilibili, YouTube, etc.) in Antigravity, Claude Code, Cursor, Gemini CLI, or Codex using a single compact storyboard image + timestamped transcript, with zero third-party npm dependencies.
Features
Variable-Tier Scene Storyboards (
1/1..1/8): One representative frame per shot, scaled by cut-point density (1/1for long takes,1/8for rapid cuts) and DP row-packed onto tightly cropped canvases (~222 vision tokens per default sheet).Audio + Multi-Source Transcript Alignment: Renders a bottom loudness waveform strip (
audio="present") and resolves transcripts cleanly from site CC subtitles, embedded subtitle tracks, local ASR, or targeted burned-in RapidOCR.Built-In Offline
sherpa-onnxASR Lifecycle: Includes a zero-framework stdlib HTTP ASR server (server/asr_server.py) and one-command installer (setup-asr) supporting SenseVoice (zh/en/ja/ko/yue) and Paraformer + Silero VAD with automatic launch and 5-second idle shutdown.Native Clip & Cache Hygiene Extensions: Adds
video_clip(export short.mp4/.mp3slices for native multimodal inspection) andvideo_cache_clean(wipe downloaded video caches after watching).Bilibili Proxy Split & Error Surfacing: Automatically routes domestic Bilibili hosts (
bilibili.com,b23.tv,bilivideo.com,hdslb.com) aroundHTTP_PROXY/HTTPS_PROXYwhile preserving proxies for international CDNs, with--proxy ""fallback.
Related MCP server: watch-mcp
8 Tools Exposed Over MCP & CLI
Tool | Purpose | Image Tokens |
| Inspect duration, resolution, FPS, tracks, and available link heights (no download) |
|
| Generate variable-tier DP-packed storyboard sheet(s) + aligned transcript + | ~222 / sheet |
| Re-read a | ~222 |
| Fuzzy search spoken transcript lines to find exact timestamps |
|
| Crop the burned-in subtitle band vertically across timestamps into one strip image | ~222 |
| Print the upstream parameter, strategy, |
|
| Export trimmed |
|
| Delete cached downloaded videos, clips, and preview images |
|
Prerequisites
Node.js
>= 20ffmpeg+ffprobe(winget install Gyan.FFmpeg/brew install ffmpeg/apt install ffmpeg)yt-dlp(optional, for URLs such as Bilibili / YouTube:pip install --user -U yt-dlp)Python
>= 3.9(optional, forsetup-asroffline speech transcription and optionalrapidocrhard-subtitle checking)
3-Step Quick Start
1. Install into Antigravity or print your MCP config
# Antigravity global plugin (~/.gemini/config/plugins/video-storyboard/)
node scripts/install.mjs --antigravity-global
# Or install into a specific workspace (.agents/plugins/video-storyboard/)
node scripts/install.mjs --antigravity-workspace /path/to/workspace
# Or print ready-to-paste MCP config for Claude Code / Cursor / Gemini CLI / Codex
node scripts/install.mjs --print-mcp2. Run diagnostics
node server/cli.mjs doctor3. (Optional) Enable offline speech transcription (sherpa-onnx)
By default, videos without subtitle tracks report visual storyboards cleanly without transcribing audio (asr-status shows not configured). To add free offline transcription (downloads ~156 MB SenseVoice int8 + Silero VAD into .asr-runtime/):
# Preview exact URLs and paths without downloading
node server/cli.mjs setup-asr --dry-run
# Install local sherpa-onnx + SenseVoice int8 + Silero VAD and configure auto-lifecycle
node server/cli.mjs setup-asr
# Check active ASR state
node server/cli.mjs asr-statusUses
DSH_ASR_FALLBACK_CONFIGwhen neither~/.dsh/asr.jsonnor$DSH_HOME/asr.jsonexists, so any existing user ASR config takes precedence automatically.
CLI Examples
node server/cli.mjs probe "https://b23.tv/..." --allow-anonymous
node server/cli.mjs storyboard "/path/to/video.mp4" --strategy auto
node server/cli.mjs zoom "/path/to/video.mp4" --cell 2
node server/cli.mjs locate "/path/to/video.mp4" "keyword"
node server/cli.mjs subtitles "/path/to/video.mp4" --max-strips 12
node server/cli.mjs clip "/path/to/video.mp4" --cell 2 --audio-only
node server/cli.mjs clean --all --include-scan-cacheNotes on Auth Cookies & Cost
Anonymous Bilibili/YouTube links may be capped or rate-limited (
HTTP 403/567). For higher bitrates or platform CC tracks, export a Netscapecookies.txtand configure{"cookies": "/path/to/cookies.txt"}in~/.dsh/video.json(or pass--cookies). Never commitcookies.txtto version control.Use
video_probefirst anddryRun: trueon long videos before committing sheets. Usefrom+segment: 300to scan long videos in 5-minute windows.
Running Tests
npm testIncludes offline smoke coverage for MCP stdio JSON-RPC (8 tools), setup-asr --dry-run, install.mjs --print-mcp, and end-to-end asr_server.py --mock auto-launch + transcription + idle shutdown.
Attribution & License
Licensed under the Apache License, Version 2.0. Built on vendored source from SAOS-3624/dsh-video-storyboard (v0.7.6). See LICENSE, NOTICE, and vendor/dsh-video-storyboard/CREDITS.md for upstream copyright, third-party credits, and documented modifications.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Hosted video MCP server for branded explainers and reference-driven edits.
Watch and share RAW media files, search, read transcripts, and auto-assemble edits.
AI video editor for agents and humans: timeline, captions, color, audio and generation as MCP tools.
Plan, compare, price, generate, and recover AI video from compatible MCP clients.
Related MCP Servers
AlicenseNot gradedqualityBmaintenanceMCP server enabling video analysis via scene detection, audio transcription, visual description, and stylistic fingerprinting, with tools for full pipeline execution and storyboard generation.1MIT- AlicenseAqualityBmaintenanceEnables Claude Code, Codex, Claude Desktop and Cursor to see and hear MP4/MOV/MKV/WebM videos and screen recordings through timestamped keyframe storyboards, zoom and full-resolution frame drill-down, and a locally generated Whisper transcript. It also works with recording bundles that capture clicks and typed context, and runs entirely on the user's machine with no uploads or API keys.51MIT
- AlicenseNot gradedqualityBmaintenanceEnables AI assistants and MCP clients to ingest audiovisual media from local files or yt-dlp-supported URLs, producing timestamped transcripts and scene-selected keyframes with exact-frame drill-down and transcript search.102 PyPI3MIT
- AlicenseNot gradedqualityCmaintenanceEnables local video understanding by generating numbered storyboard sheets with timecoded transcripts, zooming into selected segments, locating spoken phrases, probing media metadata, and tracking playback progress. It uses local ffmpeg and optional offline speech recognition.MIT