yueying
Allows the agent to process Bilibili videos by fetching timestamped transcripts and keyframe contact sheets, including support for multi-part videos and collections.
Allows the agent to process TikTok videos by fetching timestamped transcripts and keyframe contact sheets.
Allows the agent to process Vimeo videos by fetching timestamped transcripts and keyframe contact sheets.
Allows the agent to process Xiaohongshu videos by fetching timestamped transcripts and keyframe contact sheets.
Allows the agent to process YouTube videos by fetching timestamped transcripts and keyframe contact sheets.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@yueyingAnalyze this video and give me a timestamped summary: https://youtu.be/dQw4w9WgXcQ"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
yueying — let AI watch videos
Point Claude, Cursor or any MCP client at a video and get back a timestamped transcript plus keyframe contact sheets — offline, no API key. Local files first; URLs (YouTube, Bilibili, Douyin, Xiaohongshu, TikTok, Vimeo, …) are videos you are entitled to process, fetched via yt-dlp at ≤720p and deleted after processing by default.
Yueying (阅影) means "read video". One package gives you an MCP server, a CLI and an agent skill.
What you get

Claude Desktop with yueying connected: paste a link, wait about forty seconds, get the video back as
timestamped notes. This video ships captions, so speech recognition never ran, and the model asked for
the transcript only. Keyframes and contact sheets come back through get_frames when it needs to see
the screen. Demo video: GitInGifs: Git Branches by GitLab, CC BY.

Contact sheet from a 24-second demo clip (four app screenshots with Chinese narration). The yellow label on every tile is the keyframe number and timestamp; the model cites them back to you.
The transcript of the same clip — local speech recognition, language auto-detected as Chinese:
[00:00] 这是阅读,一个安静的桌面小说阅读器。整本书连续滚动,按段落记住进度。
第二个画面是桌面模式,窗口变透明,只留文字浮在桌面上。
第三个画面是伪装皮肤,一键变成代码编辑器。
最后是伪装成表格的样子。(The app is called 月读; ASR heard the homophone 阅读. Speech recognition does that to names — the model corrects it from the on-screen text in the frames.)
Every video becomes one folder:
report.md index for the model: metadata, chapters, contact sheets, keyframes, transcript
transcript.txt paragraphs with [mm:ss] timestamps
transcript.srt subtitles for any player
grid_01.jpg … 3x3 contact sheets, 9 keyframes each, in time order
frames/ full-size keyframes, e.g. f003_00m15s.jpg
manifest.json machine-readable result (paths, segments, chapters, options)Related MCP server: claude-real-video
Why yueying
Captions first, Whisper only when needed. Platform subtitles are used when they exist. Otherwise local faster-whisper:
large-v3-turboon an NVIDIA GPU,smallon CPU, automatic CPU fallback — nothing is uploaded, no key.ffmpeg bundled. Works on Windows 11 out of the box (imageio-ffmpeg); no PATH fiddling.
Token-efficient. Keyframes are taken at scene changes, near-duplicates dropped, then packed into 3x3 contact sheets with burned-in timestamps. One sheet ≈ 1–2K tokens for nine moments; one transcript with
[mm:ss]paragraphs.Chinese platforms and the rest. Bilibili (multi-part, collections, member videos with your browser login), Douyin, Xiaohongshu — and YouTube, TikTok, Vimeo, X and every other yt-dlp site.
Zero API keys, zero telemetry. The only network traffic is the video site you name and one Whisper model download. See the privacy policy.
Benchmark: a 6-minute Bilibili video → report in ~90 s on an RTX 5060 laptop; on CPU with model=small expect ~1–2 min per 10 min of speech.
Quick start
Install uv (Python is not required):
winget install astral-sh.uv # Windows brew install uv # macOS curl -LsSf https://astral.sh/uv/install.sh | sh # Linux / macOSWarm up and check everything once (installs the package, probes the GPU, downloads the speech model, runs a 2-second smoke test, prints config to paste):
uvx yueying mcp --setupAdd the server to your client (below), then ask: "Watch C:\videos\lecture3.mp4 and turn the steps into notes" or "What does this video say about docker compose: https://www.bilibili.com/video/BV…".
Claude Desktop
%APPDATA%\Claude\claude_desktop_config.json (Windows) · ~/Library/Application Support/Claude/claude_desktop_config.json (macOS). Fully quit and reopen Claude afterwards.
{ "mcpServers": { "yueying": { "command": "uvx", "args": ["yueying", "mcp"], "env": { "PYTHONUTF8": "1" } } } }Windows note: Claude Desktop does not always see your PATH — if the server fails to start ("spawn uvx ENOENT"), use the absolute path, e.g. "command": "C:\\Users\\<you>\\.local\\bin\\uvx.exe" (where uvx prints it). Logs: %APPDATA%\Claude\logs\mcp-server-yueying.log (~/Library/Logs/Claude/ on macOS). Keep wait_seconds at its default there; see the RUNNING rule.
Claude Code
claude mcp add --transport stdio --scope user yueying --env PYTHONUTF8=1 -- uvx yueying mcpOr drop this repo's .mcp.json into a project (it ships with "timeout": 1800000 so one watch_video call can wait for a long video). To raise Claude Code's tool timeout globally, set MCP_TOOL_TIMEOUT=1800000 (ms) in your environment. The repo is also a Claude Code plugin (.claude-plugin/plugin.json: server + skill).
Cursor
Click the Add to Cursor badge above, or put the same JSON in ~/.cursor/mcp.json (global) or .cursor/mcp.json (project):
{ "mcpServers": { "yueying": { "command": "uvx", "args": ["yueying", "mcp"], "env": { "PYTHONUTF8": "1" } } } }Cline
MCP Servers → Configure (cline_mcp_settings.json). timeout is in seconds; the five read-only tools are safe to auto-approve. Step-by-step agent instructions: llms-install.md.
{
"mcpServers": {
"yueying": {
"type": "stdio",
"command": "uvx",
"args": ["yueying", "mcp"],
"env": { "PYTHONUTF8": "1" },
"timeout": 1800,
"autoApprove": ["get_transcript", "search_transcript", "get_frames", "get_frame_at", "list_videos"]
}
}
}Windsurf
~/.codeium/windsurf/mcp_config.json:
{ "mcpServers": { "yueying": { "command": "uvx", "args": ["yueying", "mcp"], "env": { "PYTHONUTF8": "1" } } } }VS Code (Copilot agent mode)
Click the Install in VS Code badge above, or create .vscode/mcp.json (note the root key servers):
{ "servers": { "yueying": { "type": "stdio", "command": "uvx", "args": ["yueying", "mcp"], "env": { "PYTHONUTF8": "1" } } } }Direct links for hosts that accept custom URL schemes: cursor://anysphere.cursor-deeplink/mcp/install?name=yueying&config=eyJjb21tYW5kIjoidXZ4IiwiYXJncyI6WyJ5dWV5aW5nIiwibWNwIl0sImVudiI6eyJQWVRIT05VVEY4IjoiMSJ9fQ== and vscode:mcp/install?%7B%22name%22%3A%22yueying%22%2C%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22yueying%22%2C%22mcp%22%5D%2C%22env%22%3A%7B%22PYTHONUTF8%22%3A%221%22%7D%7D.
Without uv (pip / pipx) and Windows one-click
pip install yueying # or: pipx install yueying
yueying mcp --setup # prints a config with the absolute path of the yueying-mcp executableUse that absolute path as "command" with no args (Windows: ...\Scripts\yueying-mcp.exe; also works as python -m yueying mcp). Windows users without Python tooling can double-click install.cmd from a checkout: it creates %LOCALAPPDATA%\yueying\venv, installs the Claude Code skill, runs yueying mcp --setup and prints the JSON block with the right path.
GPU
uvx --from "yueying[cuda]" yueying mcp # NVIDIA: adds the CUDA runtime wheels (cuBLAS, cuDNN)
pip install "yueying[cuda]"Device and model are chosen automatically (model=auto: large-v3-turbo on CUDA, small on CPU); if the GPU trial fails, recognition falls back to CPU by itself.
Docker
docker build -t yueying .
docker run --rm -i -v yueying-data:/data -v "$PWD/videos:/videos:ro" yueyingThe image is CPU-only (containers get no GPU by default), so it defaults to the small model.
Mount your videos read-only and give the tools container paths (/videos/lesson.mp4); results and
the downloaded Whisper weights live in the /data volume. In a client config the command is
docker and args are ["run", "--rm", "-i", "-v", "yueying-data:/data", "-v", "/your/videos:/videos:ro", "yueying"].
Tools
Tool | When the agent uses it | What it returns | Limits |
| First call for any video: an absolute local path or a URL. |
| Blocks up to |
| The overview was truncated, a specific time range, or exporting subtitles ( | Header + |
|
| "When does he mention X?" | Hits with time, surrounding sentences, nearest keyframe number and contact-sheet number. | Space-separated terms, any matches, more terms rank higher; |
| See what is on screen: contact sheets first ( | Absolute path + JPEG per image, in time order; | ≤ 3 images per call (default 2, keep ≤ 2 in Claude Desktop); ~150 KB per sheet at 1280 px. |
| Read code, a slide, a chart or UI at one moment. | The exact frame (extracted from the source when the local file still exists) or the nearest cached keyframe, plus the paragraphs spoken around then. | One image, ~100 KB at 960 px. |
| The user refers to an earlier video, or to check disk use. | Table: video_id, title, duration, text source, date, size, folder; running jobs. | Instant, read-only. |
video for the read tools accepts the video_id from watch_video/list_videos, the results folder, or the same path/URL you gave watch_video.
The RUNNING rule
Processing can take minutes, and most hosts cap a tool call at about a minute. So watch_video waits at most wait_seconds, then answers RUNNING video_id=… · stage 2/4 speech recognition 40% · elapsed 46 s · est. ~1 min remaining. The agent simply calls watch_video again with the same video — it re-attaches to the same job (options are ignored while it runs; refresh=true restarts). Recommended wait_seconds:
Host |
| Why |
Claude Desktop | 45 (default) | hard ~60 s client timeout |
Cursor | 45 (default) | 60–120 s |
Claude Code | up to 1500 | with |
Cline | up to 1500 | with |
Progress notifications are sent every 1.5 s for hosts that display them. One video is processed at a time per server; extra requests queue.
First run: the first speech recognition downloads a Whisper model once (~480 MB small on CPU, ~1.6 GB large-v3-turbo on GPU). uvx yueying mcp --setup does this ahead of time; otherwise the RUNNING line says "first run downloads ~… this can take several minutes".
Where files go
Root: $YUEYING_OUT_DIR if set, else ~/yueying_out. One entry per video, named <slug>-<video_id> (yt-<id>, bili-<BV>, or the file name) — never renamed; the title lives in manifest.json.
~/yueying_out/
└── bili-BV1xx-3f9a2c1e/
├── report.md transcript.txt transcript.srt manifest.json
├── grid_01.jpg … grid_07.jpg
├── frames/ f001_00m02s.jpg … (+ frames/extra/ for get_frame_at)
├── .job only while a job runs
└── _download/ only with YUEYING_KEEP_SOURCE=1Environment variable | Meaning | Default |
| root folder for results (absolute, |
|
| default for the |
|
|
|
|
| language of report.md written by the server ( |
|
| default browser for cookies ( | unset |
|
| unset |
| pipelines running at once per server |
|
| hard limit per video, seconds |
|
| Hugging Face cache (Whisper weights live here) | HF default |
| mirror, e.g. | huggingface.co |
| set to | — |
Disk budget: ≈ 25 MB per hour of video; 300–600 MB/h more with YUEYING_KEEP_SOURCE=1. Nothing is deleted automatically — list_videos shows sizes; delete a folder to free space; watch_video(refresh=true) reprocesses one video. Editing a local file changes its size/mtime and therefore gets a new entry.
Supported sources
Local files: anything ffmpeg reads — mp4, mkv, mov, webm, avi, flv, ts, and audio (mp3, m4a, wav, …). Audio-only input gives a transcript without frames.
URLs: every site yt-dlp supports. Fetched at ≤720p and deleted after processing unless
YUEYING_KEEP_SOURCE=1.Bilibili: without login Bilibili serves 480p — enough for slides and code. For HD or member-only videos pass
cookies_from_browser="chrome"(oredge,firefox,brave,chromium,safari); on Windows close Chrome first, it locks its cookie database. Multi-part videos and collections:p=links are separate entries; the CLI's--allprocesses them all.Not for live streams or images. Short links (
b23.tv,v.douyin.com) are processed but not de-duplicated against their long form (the server never resolves URLs itself).
Also a CLI and an agent skill
yueying video.mp4
yueying "https://www.bilibili.com/video/BVxxxx" --ui-lang en
yueying "https://www.youtube.com/watch?v=xxxx" --out ./notes/xxx
yueying lesson1.mp4 lesson2.mp4 "https://www.bilibili.com/video/BVyyyy" # several at once, one folder each + index.md
yueying "https://www.bilibili.com/video/BVxxxx" --all # every part of a multi-part video / collection
yueying --install-skill # Claude Code skill -> ~/.claude/skills/yueyingDefault output: ./yueying_out/<name>/ (parent folder when several inputs). CLI log lines and report.md are Chinese by default (--ui-lang en for English); 0.3 will flip the default to English.
Flag | Meaning |
| output folder (default |
| when the URL is a Bilibili multi-part video / collection / playlist, process every entry (default: only the first) |
| spoken language code ( |
| Whisper model: |
| force CPU ( |
| roughly one keyframe every N seconds. Default by duration: 2 s under 1 min, 3 s under 3 min, 6 s under 10 min, 12 s under 30 min, 20 s beyond |
| maximum number of keyframes (default by duration, cap 150; 300 with |
| scene-change sensitivity 0–1, lower = more sensitive (default 0.3) |
| keep frames that are almost identical to the previous one (default drops them: < 2 % of thumbnail pixels changed) |
| no speech recognition even without subtitles (pictures only) |
| no keyframes (text only) |
| run speech recognition even when subtitles exist |
| download with your browser login (Bilibili HD / member videos, sign-in-gated YouTube) |
| keep the downloaded source video |
| language of report.md headings and labels: |
| print one line of manifest JSON at the end (for scripts) |
| install the agent skill into |
| run the MCP server ( |
The skill (src/yueying/skill/SKILL.md) tells a coding agent to prefer the MCP tools when present and otherwise run the CLI and read report.md plus the contact sheets. Tools that support the Agent Skills standard can copy ~/.claude/skills/yueying/SKILL.md into their own skills folder.
Compared with similar projects (September 2026)
yueying | claude-video | claude-real-video | mcp-video-analyzer | |
MCP server | yes | no (skill only) | no (skill) | yes (Node) |
Offline speech recognition | yes — subtitles first, local faster-whisper otherwise | cloud Whisper fallback | ASR-first | whisper installed separately |
GPU auto-detect + CPU fallback | yes | – | – | – |
ffmpeg bundled | yes | – | manual ffmpeg | – |
Windows tested | yes (Windows 11) | – | – | – |
Contact sheets (3x3) | yes | – | – | – |
Burned-in timestamps on frames | yes | – | – | – |
Bilibili / Douyin / Xiaohongshu | yes | – | – | – |
"–" means the project did not advertise the feature when we looked; check their READMEs, they may have moved on.
Privacy policy
yueying collects nothing and has no telemetry, analytics, crash reporting or update checks. All processing is local. The only network connections are (1) to the video site of the URL you pass, through yt-dlp, and (2) to Hugging Face (or HF_ENDPOINT) to download a Whisper model once. Outputs are stored in your folder until you delete them. The transcript and any frames you request are sent only to the model your MCP client is configured to use — that transfer is governed by your client's and provider's terms, not by yueying. Questions: GitHub issues. Full text: docs/privacy.md.
Troubleshooting / FAQ
"No result received" in Claude Desktop. Keep
wait_secondsat 45 (the agent then re-callswatch_video), and runuvx yueying mcp --setuponce so the first call is not also the model download.spawn uvx ENOENT/ server fails to start. The host cannot see your PATH: use the absolute path touvx(where uvx/which uvx) or toyueying-mcpas"command".Console windows flash on Windows. Update to 0.2.0+: child processes are started without a window. If you still see them, you are running an old install (
uv cache clean yueying).Mojibake /
?????in titles. Add"env": { "PYTHONUTF8": "1" }to the server config (all snippets above include it).Bilibili error 412 / "-352". The site wants a login:
cookies_from_browser="edge"or"chrome"(close Chrome first on Windows).YouTube "Sign in to confirm you're not a bot". Same fix:
cookies_from_browser. Also try updating yt-dlp:uv cache clean yueyingorpip install -U yt-dlp.Slow on CPU.
model="small"is already the automatic choice without an NVIDIA GPU; usemode="frames"when only the pictures matter, ormode="transcript"to skip keyframes.Names, numbers and code are wrong in the transcript. Expected with any ASR — the agent is told to trust on-screen text; ask it to
get_frame_atthe moment.Model download is slow or blocked (mainland China). Set
HF_ENDPOINT=https://hf-mirror.comin the serverenv(the CLI switches to the mirror automatically when huggingface.co is unreachable).GPU error (CUDA / cuDNN / out of memory).
model="small"orYUEYING_DEVICE=cpu; install the CUDA wheels withyueying[cuda].Want to reprocess with different settings.
watch_video(video=…, refresh=true, …)— it kills a running job for that video, deletes the entry and starts again.
How it works
video / URL ──► yt-dlp (≤720p) + platform subtitles
──► ffmpeg 16 kHz audio ──► faster-whisper (skipped when subtitles exist)
──► ffmpeg scene detection ──► keyframes (near-duplicates dropped)
──► Pillow: burn "#n mm:ss", pack 3x3 contact sheets
──► report.md · transcript.txt · transcript.srt · manifest.json
MCP client ──stdio──► mcp_server.py ──spawns──► python -m yueying.cli <video> --json --ui-lang en
│ parses the child's progress lines, long-polls, caches per video
└─► store.py (cache keys / folders) · query.py (paging, search, frames)The server process never loads yt-dlp, Whisper or CUDA itself; all heavy work runs in a child process that is killed with the server. Everything lives in src/yueying/:
File | Responsibility |
| command-line entry; runs the pipeline; |
| the MCP server: six tools, job runner, progress parsing, DONE/RUNNING/ERROR rendering, |
| output root, canonical URLs, per-video cache keys and folder names, |
| pure functions over a manifest: transcript paging, search, nearest frame, contact-sheet ranges, image shrinking |
| Whisper model names, sizes and Hugging Face repos (no heavy imports) |
| yt-dlp download, subtitle language choice, playlist/collection listing |
| ffmpeg wrapper: probe, audio extraction, embedded subtitles, timeouts |
| srt / vtt / Bilibili JSON subtitle parsing, YouTube auto-caption de-duplication |
| faster-whisper transcription, GPU/CPU selection, |
| scene detection, frame timing, extraction, de-duplication, timestamp burn-in, contact sheets |
| report.md, transcript files, manifest.json (zh/en labels), manifest loading |
| the agent skill |
Roadmap
next —
.mcpbone-click bundle for Claude Desktop, Smithery listing.0.3 —
forget_video/ prune tools, English as the CLI default report language,--all(playlists) over MCP.
Credits
Burned-in timestamps on contact sheets follow video-vision-mcp; duration-based frame intervals follow video-analyzer-skill
License
MIT — see LICENSE.
中文说明
阅影(yueying)让 AI 看懂视频。 给 Claude Desktop、Claude Code、Cursor 等支持 MCP 的工具一个本地视频文件或视频链接(B站 / YouTube / 抖音 / 小红书 …),它就能拿到带时间戳的文字稿和关键帧九宫格:字幕优先,没有字幕就本地 faster-whisper 识别,全程离线,不上传、不要 API key。链接通过 yt-dlp 以 ≤720p 下载,处理完默认删除原视频。完整文档见上方英文部分。
安装
# 1. 装 uv(不需要先装 Python)
winget install astral-sh.uv # Windows;macOS: brew install uv
# 2. 预热:安装、探测显卡、下载模型、跑 2 秒冒烟测试、打印配置
uvx yueying mcp --setup不用 uv:pip install yueying(有 NVIDIA 显卡再加 pip install "yueying[cuda]"),然后 yueying mcp --setup 会打印 yueying-mcp 的绝对路径。Windows 也可以下载仓库后双击 install.cmd,它会建独立环境、装 Claude Code 技能、跑 --setup 并打印可粘贴的配置。
配置
Claude Desktop(%APPDATA%\Claude\claude_desktop_config.json,改完完全退出再打开;Windows 下 uvx 找不到就写绝对路径,如 C:\\Users\\<你>\\.local\\bin\\uvx.exe)、Cursor(~/.cursor/mcp.json)、Windsurf(~/.codeium/windsurf/mcp_config.json)都是同一段:
{ "mcpServers": { "yueying": { "command": "uvx", "args": ["yueying", "mcp"], "env": { "PYTHONUTF8": "1" } } } }Claude Code 一行:
claude mcp add --transport stdio --scope user yueying --env PYTHONUTF8=1 -- uvx yueying mcpCline 在同一段里加 "type": "stdio"、"timeout": 1800,并把 get_transcript、search_transcript、get_frames、get_frame_at、list_videos 放进 autoApprove。VS Code 的 .vscode/mcp.json 根键是 servers 并加 "type": "stdio"。
六个工具
工具 | 用途 |
| 看一个视频:本地绝对路径或链接。返回 |
| 按时间段读文字稿; |
| 找「哪里提到了 X」,给出时间、上下文、最近的关键帧号和九宫格号 |
| 看画面:先看九宫格( |
| 看某一时刻:代码、PPT、图表、界面;本地文件还在就精确抽帧 |
| 列出处理过的视频(video_id、标题、时长、文字来源、日期、大小、目录)和正在跑的任务 |
Claude Desktop / Cursor 里 wait_seconds 保持默认 45;Claude Code / Cline 配好 timeout 后可以给到 1500,一次调用就等到结果。第一次语音识别要下载模型(CPU 约 480 MB 的 small,显卡约 1.6 GB 的 large-v3-turbo),--setup 会提前下好。
输出目录与环境变量
结果在 ~/yueying_out/<名字>-<video_id>/(YUEYING_OUT_DIR 可改),每个视频一个文件夹:report.md、transcript.txt、transcript.srt、grid_01.jpg …、frames/、manifest.json。约每小时视频 25 MB;不会自动删,删文件夹即可。常用环境变量:YUEYING_OUT_DIR(输出根目录)、YUEYING_MODEL(默认 auto:有 NVIDIA 显卡用 large-v3-turbo,否则 small)、YUEYING_DEVICE(auto/cuda/cpu)、YUEYING_LANG(服务端 report.md 语言,默认 en,中文写 zh)、YUEYING_KEEP_SOURCE=1(保留下载的原视频)、YUEYING_JOB_TIMEOUT(单个视频上限秒数,默认 7200)、PYTHONUTF8=1(Windows 防乱码)。
B站 cookie
不登录 B站 只给 480p,看 PPT 和代码够用;高清或会员视频传 cookies_from_browser="chrome"(或 edge),Windows 下先关掉 Chrome,否则读不到 cookie 数据库。遇到 412 / -352 也是同样的处理。
国内镜像
下载模型慢或被墙:在服务器配置的 env 里加 "HF_ENDPOINT": "https://hf-mirror.com"。命令行版在 huggingface.co 连不上时会自动切到镜像。
命令行用法
yueying 视频.mp4
yueying "https://www.bilibili.com/video/BVxxxx"
yueying "https://www.youtube.com/watch?v=xxxx" --out ./notes/xxx
yueying 第1课.mp4 第2课.mp4 "https://www.bilibili.com/video/BVyyyy" # 多个一起,各出各的文件夹 + index.md
yueying "https://www.bilibili.com/video/BVxxxx" --all # B站 分 P / 合集 全部处理
yueying --install-skill # 装 Claude Code 技能默认输出 ./yueying_out/<视频名>/,日志和 report.md 默认中文(--ui-lang en 切英文)。常用参数:--lang zh 指定语言、--model small 换小模型(auto 自动选)、--device cpu、--interval 3 抽帧间隔、--frames 30 最多帧数、--scene 0.2 场景灵敏度、--no-dedupe 不去重、--no-asr 只要画面、--no-frames 只要文字、--force-asr 有字幕也识别、--cookies-from-browser chrome、--keep 保留原视频、--json 末尾打印一行 manifest。完整说明见上方英文表格。
Available Tools
6 toolsget_frame_atARead-onlyIdempotent
Look closely at one moment of an already-watched video, e.g. to read code, a slide, a chart or a UI. For local files that still exist the exact frame at that time is extracted from the video; otherwise the nearest cached keyframe is returned and the caption says so. Also returns the transcript paragraphs spoken around that time. Returns one image (~100 KB at 960 px).
| Name | Required | Description | Default |
|---|---|---|---|
| time | Yes | Timestamp: seconds, mm:ss or h:mm:ss, e.g. '03:15' | |
| video | Yes | video_id, results folder, or the path/URL given to watch_video | |
| max_width | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With annotations already marking readOnlyHint and idempotentHint, the description adds meaningful behavioral detail: exact frames are extracted from existing local files, otherwise the nearest cached keyframe is returned and the caption notes this. It also discloses the transcript inclusion and approximate image size, going beyond what annotations capture.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded, starting with the core purpose before covering fallback behavior and output details. Every sentence contributes useful information, though it is slightly longer than the minimum needed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description adequately explains what the agent receives: one image, approximate size, transcript paragraphs, and the keyframe caption caveat. It covers the important edge case of missing local files and gives enough context for a correct call, though it does not discuss failure modes or prerequisites at length.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 67%, and the description mostly restates that the frame corresponds to a specific time rather than adding new parameter-level semantics. It does connect the default 960 px to the ~100 KB output, but it does not substantially enrich the meaning of video, time, or max_width beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Look closely at one moment') with a clear resource ('already-watched video') and concrete use cases like reading code, slides, or UI. It also explicitly says it returns one image, distinguishing it from siblings like get_frames or watch_video.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description conveys a clear context for use: when you need a close, detailed look at a specific moment of a previously watched video. It does not explicitly name alternatives or give when-not-to-use guidance, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_framesARead-onlyIdempotent
See what is on screen in an already-watched video. Returns contact-sheet images (3x3 keyframes in time order, every tile labelled '#number mm:ss' bottom-left) or individual keyframes. Read the contact sheets first to get the visual storyline, then request single frames only when you need to read code, slides or UI text. At most 3 images per call (default 2), downscaled to max_width; page with start/count. Every image is preceded by its absolute file path so hosts that can read files may open the full-size original instead.
| Name | Required | Description | Default |
|---|---|---|---|
| kind | No | grids = 3x3 contact sheets, 9 keyframes per image (start here); frames = individual full-size keyframes | grids |
| count | No | Images per call; keep 2 or fewer in Claude Desktop | |
| start | No | 1-based number of the first image: contact-sheet number for grids, keyframe number (as printed on the tiles) for frames | |
| video | Yes | video_id, results folder, or the path/URL given to watch_video | |
| max_width | No | Downscale width in pixels; 1280 is about 150 KB per contact sheet |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, and the description adds substantial behavioral detail: tile labels with '#number mm:ss', the 3-image cap, downscaling to max_width, paging via start/count, and the absolute-file-path prefix. This goes well beyond the structured annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but well organized: purpose first, then usage strategy, then limits and paging, then file-path behavior. Every sentence contributes actionable information, and there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description must carry the burden of explaining return values; it does so thoroughly, including image layout, labels, ordering, size limits, and host file access. The tool is simple enough that nothing needed for correct invocation is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds real value by explaining the paging relationship between start/count, the meaning of kind ('grids' vs 'frames'), and the practical effect of max_width. This helps an agent use the parameters more correctly than the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Returns contact-sheet images... or individual keyframes') and the resource ('an already-watched video'), and it distinguishes the two output kinds, grids vs. frames. This makes the tool's purpose concrete and separates it from related frame-retrieval tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says to 'Read the contact sheets first... then request single frames only when you need to read code, slides or UI text,' which is strong when-to-use guidance within the tool. It also gives paging and count guidance, though it does not explicitly name sibling alternatives like get_frame_at for single-frame needs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_transcriptARead-onlyIdempotent
Read part of an already-watched video's transcript with [mm:ss] timestamps. Use when the watch_video overview was truncated, when the user asks about a specific time range, or to export subtitles (format='srt'). Returns at most max_chars; when truncated the last line gives next_start so you can continue from there. Paragraph format is the cheapest.
| Name | Required | Description | Default |
|---|---|---|---|
| end | No | End time (same formats); None = to the end | |
| start | No | Start time: seconds, mm:ss or h:mm:ss | 0 |
| video | Yes | video_id from watch_video/list_videos, the results folder, or the same path/URL given to watch_video | |
| format | No | paragraphs = compact [mm:ss] text (fewest tokens); segments = one line per subtitle cue with start-end; srt = subtitle blocks | paragraphs |
| max_chars | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses truncation and pagination behavior: "Returns at most max_chars; when truncated the last line gives next_start so you can continue from there." It also notes cost behavior ("Paragraph format is the cheapest"). The readOnlyHint and idempotentHint annotations already cover safety, so the description adds meaningful behavioral detail beyond them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three dense sentences, each earning its place: purpose, use cases, and return/truncation behavior. The most important scope is front-loaded and there is no redundant restating of the parameter schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only transcript tool with no output schema, the description covers the key operational details: time-range reading, format options, truncation, continuation, and cost. It is slightly less explicit about the exact shape of the transcript lines, but enough for an agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is high and each parameter already has a description, so the baseline is 3. The description adds value beyond the schema by explaining the practical cost/format tradeoff and the continuation mechanism tied to max_chars and start/end usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: "Read part of an already-watched video's transcript with [mm:ss] timestamps." It clearly distinguishes this from the sibling tools by explicitly linking it to watch_video and by its time-range-focused scope rather than search or frame extraction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use conditions: "Use when the watch_video overview was truncated, when the user asks about a specific time range, or to export subtitles (format='srt')." This tells the agent exactly which scenario selects this tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_videosARead-onlyIdempotent
List videos already processed by yueying on this machine (newest first) and jobs currently running, with video_id, title, duration, text source, date, folder and size. Use when the user refers to a video watched earlier, to get a video_id for the other tools, or to see how much disk space results use. Instant and read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so the safety profile is covered. The description adds value by specifying the scope ('on this machine') and mentions disk space usage, which is additional context beyond annotations. However, it doesn't detail behavior like live updates of running jobs or error cases, so a 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact: it states what is listed, the fields, and the purpose in two sentences. It front-loads the core function and lists key return fields, then gives usage reasons. Every clause earns its place; no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, the rich sibling context, and clear schema, the description covers everything needed for correct invocation. It specifies what is returned, ordering, usage cases, and even performance characteristics (instant, read-only). No gaps for an agent to make mistakes.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the parameter is simple (limit with default, min, max). The description does not explain the limit semantics, but since it's a common parameter with schema constraints, baseline 3 is given; the description could add 'Use limit to control result count' but doesn't need much.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists videos already processed by yueying, plus running jobs, with a specific set of fields returned, and notes ordering (newest first). It distinguishes itself from siblings by focusing on already-processed videos and running jobs, not on watching, transcript retrieval, or frame lookup.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: when user refers to a video watched earlier, to fetch video_id for other tools, or to check disk space. It also implicitly guides away from siblings by mentioning the video_id is for the other tools, and characterizes the tool as instant and read-only, reinforcing usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_transcriptARead-onlyIdempotent
Find where something is said in an already-watched video. Each hit shows the time, the surrounding sentences, and the nearest keyframe number and contact-sheet number so you can follow up with get_frame_at or get_transcript. Use this instead of paging the whole transcript when the user asks 'when does he mention X' or 'find the part about Y'.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| query | Yes | Words to look for; separate alternatives with spaces (any term matches, more terms rank higher). Case-insensitive | |
| video | Yes | video_id, results folder, or the path/URL given to watch_video | |
| context_seconds | No | Seconds of surrounding text to include around each hit |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish readOnlyHint and idempotentHint. The description adds meaningful context on top: the 'already-watched' prerequisite and the exact per-hit result shape (time, surrounding sentences, keyframe number, contact-sheet number). It does not contradict annotations, though it leaves out result ordering and empty-result behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, then result contents, then usage guidance. Every clause earns its place and no information is repeated from the schema or annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description tells the agent exactly what each hit returns and how to follow up with get_frame_at or get_transcript. Combined with the well-described parameters and annotations, the description gives the agent everything needed to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 75% and the schema itself gives rich descriptions for video, query, and context_seconds. The tool description does not add parameter-level semantics and does not explain the undocumented limit parameter. This is a fair baseline score because the schema carries the semantic burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Find where something is said') targeting a specific resource ('an already-watched video' transcript) and describes the result contents. It clearly differentiates itself from get_transcript by framing itself as the targeted-search alternative to paging the full transcript.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says to use this tool instead of paging the whole transcript when the user asks a specific mention question, with concrete example phrasing ('when does he mention X'). It also names follow-up tools, giving clear routing context. This is strong, actionable when-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
watch_videoAIdempotent
Turn a video into a timestamped transcript plus a keyframe overview so you can summarize it, answer questions about it, extract steps, commands or code, or write notes. video is an absolute path to a local video/audio file or a YouTube / Bilibili / Douyin / Xiaohongshu / TikTok / Vimeo URL. Fully offline: platform subtitles are used when they exist, otherwise local Whisper speech recognition (GPU when available, CPU otherwise); no API key. Results are cached per video, so calling again with the same video is instant. Typical time: 5–60 s for short subtitled videos, a few minutes for long videos without subtitles; the very first speech recognition downloads a model once (~480 MB on CPU, ~1.6 GB on GPU). If the reply starts with RUNNING, call watch_video again with the same video — it resumes waiting for the same job; do not change options and do not start other videos meanwhile. The reply is an English overview: metadata, chapters, the list of contact sheets (see them with get_frames), file paths, and the transcript with [mm:ss] timestamps (truncated at max_chars with a start time for get_transcript). Cite timestamps like (03:15). Not for live streams or images.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | full = transcript + keyframes (default); transcript = no keyframes; frames = no speech recognition (platform subtitles still used) | full |
| model | No | Whisper model used only when the video has no subtitles. auto = large-v3-turbo on an NVIDIA GPU, small on CPU | auto |
| video | Yes | Absolute path to a local video/audio file, or a video page URL (YouTube, Bilibili, Douyin, Xiaohongshu, TikTok, Vimeo, X and other yt-dlp sites) | |
| refresh | No | Discard cached results (and any running job) for this video and process it again | |
| language | No | Spoken language; auto detects it. Set it when you know it for better accuracy | auto |
| max_chars | No | Maximum characters of transcript included in the reply | |
| output_dir | No | Absolute folder for the results; default $YUEYING_OUT_DIR/<name>-<id> | |
| wait_seconds | No | How long this call may block before answering RUNNING. Keep 45 in Claude Desktop/Cursor (60 s client timeout); Claude Code/Cline may use up to 1500 | |
| cookies_from_browser | No | Reuse a browser login for HD or member-only Bilibili / sign-in-gated YouTube. Close Chrome first on Windows | |
| frame_interval_seconds | No | Approximate seconds between keyframes; None = automatic by duration (2–20 s). Use 2 for code/slide-heavy screencasts |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond the annotations. It discloses that the tool is fully offline, uses platform subtitles when available otherwise local Whisper, caches results per video, has specific time expectations (5–60s short subtitled, minutes for long unsubtitled), downloads a model on first run (~480MB CPU / ~1.6GB GPU), and explains the RUNNING reply behavior. It also states the reply format (English overview with metadata, chapters, contact sheets, file paths, transcript with [mm:ss] timestamps). This is rich behavioral context that annotations alone (readOnlyHint=false, idempotentHint=true, destructiveHint=false) do not provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but well-organized, front-loading the core purpose and then covering input types, offline behavior, caching, timing, the RUNNING protocol, reply format, and exclusions. Every sentence carries useful information. It loses a point for being somewhat long and for burying the 'Not for live streams or images' exclusion at the end, which is important routing information that could be more prominent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with 10 parameters, no output schema, and no sibling differentiation in the schema, the description is remarkably complete. It covers input types, platform support, offline behavior, caching, timing, model download, the RUNNING retry protocol, reply format, timestamp citation format, and exclusions. It also references sibling tools (get_frames, get_transcript) for follow-up actions. Nothing an agent needs to call this tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds value beyond the schema by explaining the video parameter's accepted URL platforms (YouTube, Bilibili, Douyin, Xiaohongshu, TikTok, Vimeo) and clarifying that results are cached per video, which gives meaning to the refresh parameter. It also explains the max_chars truncation behavior and its relationship to get_transcript. However, it doesn't add much detail on mode, model, language, or output_dir beyond what the schema already documents, so a 4 rather than 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Turn a video into a timestamped transcript plus a keyframe overview' and immediately lists downstream uses (summarize, answer questions, extract steps/commands/code, write notes). It also names the input types (local file or specific platforms) and explicitly excludes live streams and images, which distinguishes it from sibling tools like get_frames or get_transcript.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use context: use it to summarize, answer questions, extract steps/commands/code, or write notes. It also states when NOT to use it ('Not for live streams or images') and references sibling tools for follow-up actions ('see them with get_frames', 'start time for get_transcript'). It even provides operational guidance about the RUNNING reply and not starting other videos meanwhile.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
6 tool updates
v0.2.1- First observed
get_frame_at - First observed
get_frames - First observed
get_transcript - First observed
list_videos - First observed
search_transcript - First observed
watch_video
TDQS
Scored across 6 tools
Each tool has a clear primary purpose, but get_frames and get_frame_at have overlapping names and both can return individual frames, and get_transcript/search_transcript both access transcript content. The descriptions are detailed enough to resolve most ambiguity, so misselection is unlikely.
All tool names follow a consistent lowercase snake_case verb_noun pattern: list_videos, watch_video, get_transcript, search_transcript, get_frames, get_frame_at. The only slight variation is the prepositional suffix in get_frame_at, but it still reads as verb_object and maintains overall consistency.
Six tools is a well-scoped set for the video-analysis lifecycle: one ingestion tool, two transcript accessors, two frame accessors, and one listing utility. No tool feels redundant, and the count is appropriate for the server's purpose.
The tool surface covers the full workflow of ingesting a video, listing processed items, reading/searching transcripts, and inspecting frames. Minor gaps exist, such as no delete/cancel operation and no dedicated metadata retrieval, but these do not block the core use cases.
Maintenance
Related MCP Connectors
Video, audio, and image processing for AI agents: convert, transcribe, upscale - 150+ operations.
Fetch transcripts, subtitles, chapters, metadata and frames from YouTube and 10+ video platforms
PDF, image, video, OCR, screenshot, SQL, QR and text tools for agents. No API key, no signup.
15 media & data tools for AI agents: search, transcribe, subtitles, voiceover, translate & more.
Related MCP Servers
- AlicenseAqualityBmaintenanceLet AI agents watch videos: local transcripts, speaker labels, scenes, chapters and exact-moment search from any video URL or file. Fully local, no API keys.42AGPL 3.0
- AlicenseNot gradedqualityAmaintenanceLets any LLM agent actually watch videos: a watch_video tool takes a URL or local file and returns scene-aware keyframes fused with a timestamped transcript, processed 100% locally with per-source caching.2,139MIT
- AlicenseNot gradedqualityBmaintenanceEnables LLM agents to process local videos into timestamped, citable text documents and then query them through tools for listing videos, retrieving transcripts, and fetching specific segments, all fully offline.MIT
- AlicenseNot gradedqualityCmaintenanceEnables AI agents to download, convert, and visually analyze videos by providing timestamped frames and locally transcribed speech, all without API keys.MIT