MultiMediaGrep
Configures ElevenLabs as an ASR backend to transcribe local audio/video files, with configurable API key, base URL, model, language, and timeout.
Supports OpenAI /audio/transcriptions-compatible ASR services for transcribing local audio/video, using a configurable API key, base URL, and model.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@MultiMediaGrepprobe /Users/me/Movies/demo.mp4 and show me a storyboard overview"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
MultiMediaGrep
给 agent 的本地媒体感知工具(MCP server)。
把本地已有的视频 / 录音 / 文档,变成 agent 能直接消化的材料:可直接看的图(分镜宫格、特写裁切) + 按时间/页码对齐的文本清单。server 自己不调 LLM——理解由 agent 的多模态模型完成。
这是什么 / 不是什么
是 | 不是 |
一个 stdio MCP server,只吃本地绝对路径 | 不下载、不检索、不登录、不碰 cookie / 凭据(取数属于 IntelligenceAtlas) |
只做「材料化」:出图 + 出带时间码/页码的文本清单 | 不调 LLM:不做总结、不做问答、不做转述 |
ASR 与文档解析/OCR 走你自己配的外部服务与 key | 不内置本地 ASR / OCR 模型(mlx-whisper、rapidocr 都不再维护) |
缓存只写自己的缓存目录 | 绝不在你的文件旁边写任何东西 |
失败也是明说的:没配 ASR 就只给画面,扫描件没配 OCR 就只给页图,并在结果里写清楚原因—— 宁可说「没有文字」,也不编一段台词。
Related MCP server: video-tools-mcp
工具(5 个,按动作命名,跨媒体复用)
工具 | 干什么 |
| 探媒体:时长/分辨率/页数/编码、可用轨道、能否做 ASR/OCR |
| 总览:视频出分镜宫格图,录音出带时间轴转写,文档出页面图 + 每页文本;额外返回全文落盘路径 |
| 特写:取指定时间点/区间的单帧或文档某页,可裁切放大,用于读字 |
| 定位:给一句文本/关键词,返回它在视频/录音里的时间戳、在文档里的页码 |
| 字幕:读本地 srt/vtt/ass 或视频内嵌字幕轨,解析成带时间轴的段落 |
用法指引(先 probe → 再 overview → 只 zoom 重要部分 → 长素材先收窄范围)写在工具描述与 server instructions 里。
环境要求
依赖 | 必需性 | 缺了会怎样 |
Node.js ≥ 20(ESM,零构建步骤) | 必需 | 起不来 |
ffmpeg + ffprobe(同一个发行版里通常一起装) | 视频 / 录音 / 文档都需要 | 返回 |
poppler( | 只有读 PDF 时需要 | PDF 相关调用返回 |
安装示例:
# macOS
brew install ffmpeg poppler # 文档需要 poppler;只要视频可省略 poppler
# Debian / Ubuntu
sudo apt install ffmpeg poppler-utils
# Windows(winget)
winget install Gyan.FFmpeg # PDF 另装 poppler(无官方 winget 包时用 scoop install poppler)可选:把可执行文件放在 PATH 之外时,用 MMGREP_FFMPEG_PATH / MMGREP_FFPROBE_PATH 指定绝对路径。
外部服务(可选):ASR(ElevenLabs 或任何 OpenAI /audio/transcriptions 兼容服务)与文档解析(MinerU)
由你用环境变量配置;不配也能用,只是拿不到文字层(详见下面「环境变量」)。
安装与运行
# 1) 免安装(发布到 npm 之后)
npx -y multimediagrep-mcp
npx -y multimediagrep-mcp会拉起 server 并阻塞等待 stdio 上的 MCP 消息——它没有交互界面, 通常由 MCP client 自动拉起。想确认它起没起得来,就手动跑一次,看 stderr 有没有启动横幅:[mmgrep] info: multimediagrep@0.1.0 ready on stdio [mmgrep] info: cache: /Users/you/.cache/multimediagrep (max 10737418240 bytes) [mmgrep] info: asr: none (no API key) · doc: none · ocr: none [mmgrep] info: backends: video=ready audio=ready document=ready [mmgrep] info: image budget: 4000 tokens/call, maxEdge 1600px
# 2) 从 checkout 直接跑
git clone https://github.com/flmaximwang/MultiMediaGrep.git && cd MultiMediaGrep
npm install
node src/index.js# 3) 从 GitHub 装(npm 还没发布时)
npx -y github:flmaximwang/MultiMediaGrep # 临时拉取并启动
npm i -g github:flmaximwang/MultiMediaGrep # 装成本地命令 multimediagrep-mcp
npm i -y https://github.com/flmaximwang/MultiMediaGrep/releases/download/v0.1.0/multimediagrep-mcp-0.1.0.tgz # 只装 release 里的运行时包从 GitHub 源码装会把整个仓库拉下来(含 vendor 源码与文档),首次安装比 npm 慢; release 里的
.tgz只含运行时(src + vendor 运行时 + 署名材料)。
只有 stdio 传输。 没有 HTTP / SSE 端点,也没有 --port 之类的开关——它被谁拉起、怎么拉起,
由 MCP client 决定。
接入宿主
Claude Code
# 用户级(所有项目可用)
claude mcp add --scope user multimediagrep \
-e MMGREP_ASR_PROVIDER=elevenlabs \
-e MMGREP_ASR_API_KEY=sk-你的key \
-- npx -y multimediagrep-mcp
# 项目级(写进项目里的 .mcp.json,随仓库走)
claude mcp add --scope project multimediagrep -- npx -y multimediagrep-mcp.mcp.json 片段(claude mcp add --scope project 生成的就是这个形状):
{
"mcpServers": {
"multimediagrep": {
"command": "npx",
"args": ["-y", "multimediagrep-mcp"],
"env": {
"MMGREP_ASR_PROVIDER": "elevenlabs",
"MMGREP_ASR_API_KEY": "sk-替换成你的key",
"MMGREP_DOC_PROVIDER": "mineru",
"MMGREP_DOC_API_KEY": "替换成你的token"
}
}
}
}Hermes
Hermes 用同名的 hermes mcp add 流程(stdio server):
hermes mcp add multimediagrep \
--command npx \
--env MMGREP_ASR_PROVIDER=elevenlabs MMGREP_ASR_API_KEY=sk-你的key \
--args -y multimediagrep-mcp它会先连一次、列出工具让你勾选,然后写进 ~/.hermes/config.yaml 的 mcp_servers:。
加完用 hermes mcp test multimediagrep 验连通,用 /reload-mcp 让已经开着的会话也加载上。
--args之后的参数原样透传给命令,所以--args必须放在最后。
其它 MCP client
任何吃标准 MCP client 配置的宿主都用同一段 JSON(键名通常是 mcpServers):
可复制的一份见 examples/mcp-config.json,
各宿主的完整写法见 docs/install.md §2。
给第一次来的 agent:5 分钟上手
给一个没有上下文、只想「看看这个工具能干什么」的 agent:
装起来(宿主负责拉起它,不用自己开端口):
Claude Code:
claude mcp add --scope user multimediagrep -- npx -y github:flmaximwang/MultiMediaGrep其它 client:把
examples/mcp-config.json里的command/args换成上面那条命令即可。
确认环境:宿主里需要
ffmpeg+ffprobe在 PATH 上;读 PDF 还要poppler/mutool/gs任意一个。 缺了会返回TOOL_NOT_FOUND,不会静默出错。先探后看(省图片 token):
media_probe({ path })→ 类型、时长、有没有字幕轨、能不能 ASR/OCR。不花任何图片 token,先做这一步。media_overview({ path })→ 视频出分镜宫格图 + 时间对齐文本;录音出带时间轴转写;文档出页图 + 每页文字。media_zoom({ path, at })→ 只在「画面上有字要读」时用,最贵(约 1900 token/张)。media_locate({ path, query })→ 「那句话在哪说的」,纯文本、零图片。
记住三件事:
只吃本地绝对路径,它不下载任何东西;
server 自己不调 LLM,理解由你的模型完成;
字幕优先于 ASR:字幕是作者原话(可引用),ASR 是机器转写(人名、数字容易错,别当原话引用)。
长素材先收窄:视频/录音用
start/end,文档用pages(如"3-10,15"),再跟着结果里的continuation往下走。
没有 ASR key 也能用:只是拿不到文字层(文件自带字幕就照样出文本)。
环境变量
全部前缀 MMGREP_,用 env | grep MMGREP_ 查看当前值。key 只写在你自己的配置文件里,
server 不会打印它(日志与错误里只出现 set / unset)。
变量 | 默认值 | 作用 |
|
| 缓存根目录(抽帧、页图、转写、全文落盘都在这)。 |
|
| 缓存上限,支持 |
|
| ASR 后端: |
| 空 | ASR 的 API key。选了 provider 就必填——自建本地服务也要给一个非空值(可以随便写,如 |
|
| ASR 的 API 基址。默认值与 provider 无关——用 |
|
| ASR 模型名。默认值与 provider 无关——用 |
|
| 强制语言(BCP-47,如 |
|
| ASR 单次请求超时 |
|
| 文档解析后端: |
| 空 | 文档解析服务的 token(MinerU:https://mineru.net/apiManage/token) |
|
| 文档解析的 API 基址 |
| 空 | 解析模型名;留空 = 用服务端默认(MinerU 可选 |
|
| 文档解析单次请求超时(3 分钟)。MinerU 是「上传 → 排队 → 解析 → 下载」整段墙钟时间,排队久就调大 |
|
| 预留给独立的本地 OCR 通道(契约 §12.4)。当前版本未实现:OCR 由文档解析 provider(MinerU)承担,这个变量只出现在启动横幅,不影响任何工具行为 |
|
| 单次调用的图片 token 软预算。超了自动模式会降形,显式模式只警告 |
|
| 单张图最长边的全局上限(px)。请求超过 → |
|
| 日志级别: |
另有 MMGREP_FFMPEG_PATH / MMGREP_FFPROBE_PATH(默认按 PATH 查找)、
MMGREP_MAX_TEXT_INLINE_CHARS(默认 24000)、MMGREP_MAX_CONCURRENCY(默认 2)等,
完整清单见 docs/mcp-contract.md §12。
第一次跑一遍
假设有一个本地视频 /abs/path/demo.mp4:
1. 探(不花图片 token,几毫秒返回)
{ "path": "/abs/path/demo.mp4" }返回里看四件事:媒体类型、时长、有没有字幕轨、capabilities.asr.available。
录音还会给一段 audioHealth 体检——如果结论是 too-quiet / silent,
转写会故意是空的(避免 ASR 编出一整段幻觉台词),这时该做的是换一份清楚的录音,而不是继续追问。
2. 总览(主体动作)
{ "path": "/abs/path/demo.mp4", "cells": 9, "images": 2 }出 2 张 3×3 的分镜宫格图 + 一份按格对齐的文本清单(每行末尾标了来源:subtitle / asr / ocr / none)。
全文不内联,而是落盘成三个文件并把路径返回给你:
文件 | 内容 |
| 人读 Markdown,按时间区间分节 |
| 一行一个 segment(JSON),机器读 |
| 工具、窗口、转写来源、图片描述、警告、计数 |
读这些文件要自己分块总结,不要为了拿更多内联文本反复调工具。
3. 特写(贵,只在「这里有事」时用)
{ "path": "/abs/path/demo.mp4", "at": 72.5 }宫格一格只有几百像素,里面的字基本看不清;要读画面上的字必须 zoom,
必要时再加 crop({x, y, width, height},全是 0..1 的比例,原点左上)裁出那块区域。
4. 定位(纯文本,零图片)
{ "path": "/abs/path/demo.mp4", "query": "那句话是在哪说的" }返回它在视频里的时间戳(毫秒 + 秒两种表示)、在文档里的页码。
长素材先收窄。 超过 30 分钟的视频、超过 30 页的文档,先用
start/end(视频、录音)或pages(文档,如"3-10,15")取一小段,再跟着结果里的continuation提示一页页往下走。 一次 overview 硬上限是 4 张图、每张 16 格、单次共 64 格、单图最长边 1600px—— 一张「两小时电影的大宫格」既看不清又剧透。
依赖、许可与上游致谢
运行时依赖:
@modelcontextprotocol/sdk(MIT)、zod(MIT)。没有构建步骤,纯 ESM。系统依赖:ffmpeg / ffprobe(不打包、不链接,按独立子进程调用)、poppler 或 mutool 或 gs(只读 PDF 时用)。
可选外部服务:ElevenLabs(ASR)、MinerU(文档解析)。都是用户自备 key,代码里不含它们的凭据。
许可:本项目 Apache-2.0(
LICENSE)。vendored 源码:视频部分基于 Apache-2.0 的 dsh-video-storyboard v0.5.9 改造而来, 源码在
vendor/dsh-video-storyboard/(VENDOR.md逐文件说明改了什么; 上游LICENSE/NOTICE/CREDITS.md逐字复制在同目录下)。 被改动的文件顶部都有Modifications:段落(Apache-2.0 §4(b))。 完整署名见NOTICE与CREDITS.md。
开发
纯 ESM JavaScript(Node ≥20),零构建步骤。规格与决策只在 beads(bd),不在本仓库写交接文档。
npm install
npm run check # 语法检查(src/ 与 vendor/ 全量 node --check)
npm test # node --test
npm run fixtures # 生成测试素材(ffmpeg 脚本生成,仓库不含任何第三方媒体)工具契约(参数、返回结构、错误码、图片预算)见 docs/mcp-contract.md;
安装、provider 配置、排障与验证清单见 docs/install.md。
Available Tools
5 toolsmedia_locateLocateARead-only
Find where something appears in a local media file and return its position: a timestamp (milliseconds and seconds) for a video or recording, a page number for a document. (定位:给一段文本或关键词,返回它在视频 / 录音里的时间戳,在文档里的页码。)
Use it when you know roughly what was said but not when ("where does it mention X"), or when you need to know which page a phrase is on. Matching ignores punctuation, spacing and case, and works for both Chinese and English. Long phrases match better than single characters.
Text only by default — no image, no pixels spent. Add includeFrame: true to also get the picture at the best match (one image, roughly 1,900 vision tokens).
A transcript is required: the embedded subtitle track, a transcriptFile you pass in, or ASR configured via MMGREP_ASR_PROVIDER. Subtitles win over ASR. If none is available this tool says so instead of guessing. Use media_overview if you only need the visuals.
Each match is tagged with its source. A match whose source is "asr" came from machine transcription, so the wording may be wrong (names, numbers, jargon) — do not present it as the speaker's exact words.
| Name | Required | Description | Default |
|---|---|---|---|
| end | No | Window end in seconds (closed interval, both ends included). Applicable to video and audio only. Defaults to the end of the media; a value past the end is clipped and reported. (终点,秒) | |
| max | No | Maximum number of matches to return: 1-20, default 5. (最多返回几条匹配) | |
| path | Yes | Absolute path to a LOCAL file: video, audio recording, or document. URLs and relative paths are rejected — this server never downloads anything. (本地文件绝对路径) | |
| pages | No | Page range for document media: comma-separated 1-based page numbers and closed ranges, e.g. "3-10,15". Not applicable to video or audio. (页码范围) | |
| query | Yes | Text or keyword to find. Matching ignores punctuation, spacing and case; Chinese and English both work. (要找的文本 / 关键词) | |
| start | No | Window start in seconds (closed interval, both ends included). Applicable to video and audio only; document media uses "pages" instead. Default 0. (起点,秒) | |
| segment | No | Window length in seconds (30-3600, default 300), used when only "start" is given. Applicable to video and audio only. | |
| transcript | No | "auto" (default): use subtitles first, ASR only as a fallback. "off": no transcription at all, visuals only. Applicable to video and audio only. | |
| includeFrame | No | Also return one picture at the best match (default false). Applicable to video and document only. (是否附带最佳匹配处的画面) | |
| transcriptFile | No | Absolute path to a local subtitle/transcript file (.srt, .vtt, .ass, .ssa). Applicable to video and audio only. (本地字幕/转写文件) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnlyHint and openWorldHint, so the description carries the rest: text-only by default, includeFrame costs roughly 1,900 vision tokens for one image, transcript precedence (subtitles > transcriptFile > ASR via MMGREP_ASR_PROVIDER), graceful failure rather than guessing, and the caveat that ASR-sourced matches may misquote names and numbers. The only missing piece is a fuller account of the per-match record fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose and return format, then usage, cost, and prerequisites in short paragraph blocks. It runs longer than most definitions, but nearly every sentence carries decision-relevant information (token cost, transcript requirement, ASR caveat); the bilingual restatements add length without adding much for an English-reading agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 10-parameter tool with no output schema, the description covers the critical unknowns: what is returned (position units, source tag), what it costs, what prerequisites must hold, and how failure is surfaced. It stops short of enumerating all result fields (snippet text, match metadata), leaving some return-shape uncertainty.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3; the description adds genuine meaning on top — matching ignores punctuation, spacing and case, works in Chinese and English, and long phrases match better than single characters — which the schema's query field does not say. It also ties the transcript/transcriptFile parameters to the ASR fallback logic.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource — find where a given text appears in a local media file — and specifies the return shape (timestamp in ms/seconds for video/audio, page number for documents). It also names the sibling it is not (media_overview for visuals only), so an agent can choose without opening schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit use case ("you know roughly what was said but not when") plus an explicit alternative ("Use media_overview if you only need the visuals"). It also warns when the tool cannot work (no transcript available), which is a real when-not condition.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
media_overviewOverviewARead-only
Build an overview of a local media file so you can read it without guessing. (生成媒体总览:视频出分镜宫格图与按时间对齐的文本清单;录音出带时间轴的转写;文档出页面缩略图与每页文字。)
What you get depends on the media:
video : one or more numbered storyboard grids (each cell carries its cell number) plus a time-aligned text index of every cell
audio : a timestamped transcript, segment by segment
document : a grid of page thumbnails plus the text of each page, when a document parser is configured
TEXT SOURCE ORDER — subtitles first, ASR only as a fallback:
a transcriptFile you pass in (external subtitle file),
the subtitle track embedded in the container,
a subtitle file with the same base name in the same directory (a.srt next to a.mp4),
ASR — only when none of the above exists and an ASR provider is configured. Every segment is tagged with its source. subtitle = the author's own words, safe to quote. asr = machine transcription: names, numbers and jargon are often wrong, so do NOT quote an ASR segment verbatim and do not present it as the speaker's exact words. visual = there is no text at all, only the picture.
The storyboard grid is for FINDING things, not for reading them: cells are small, so text inside a cell is usually unreadable. To read text on screen, call media_zoom on that moment and read the cropped single frame.
The full text is NOT inlined. It is written to disk and the returned paths point at it: content.md (readable), segments.jsonl (one segment per line, machine-readable), manifest.json (what was produced and how). Read those files section by section and summarise them in chunks yourself — do not ask for the same range again just to get more inline text.
Always call media_probe first: it tells you how long the file is, whether a subtitle source exists, and which transcript source applies, for free. Long media must be narrowed with start/end (video, audio) or pages (document) — one giant grid of a two-hour film gives you illegible cells and spoils the ending. Follow the continuation hint in the result to page through the rest.
| Name | Required | Description | Default |
|---|---|---|---|
| end | No | Window end in seconds (closed interval, both ends included). Applicable to video and audio only. Defaults to the end of the media; a value past the end is clipped and reported. (终点,秒) | |
| path | Yes | Absolute path to a LOCAL file: video, audio recording, or document. URLs and relative paths are rejected — this server never downloads anything. (本地文件绝对路径) | |
| cells | No | Cells per puzzle image: storyboard shots for video, page thumbnails for document. 1-16; default 9 for video and 6 for document. Applicable to video and document only. (每张拼图的格数) | |
| pages | No | Page range for document media: comma-separated 1-based page numbers and closed ranges, e.g. "3-10,15". Not applicable to video or audio. (页码范围) | |
| start | No | Window start in seconds (closed interval, both ends included). Applicable to video and audio only; document media uses "pages" instead. Default 0. (起点,秒) | |
| images | No | How many puzzle images to return: 1-4, default 2. Applicable to video and document only. (返回几张拼图) | |
| maxEdge | No | Longest edge of each image in pixels: 256-1600. Default 1280 for overview, 1600 for zoom. Applicable to video and document only. (单张图最长边) | |
| segment | No | Window length in seconds (30-3600, default 300), used when only "start" is given. Applicable to video and audio only. | |
| envelope | No | Attach an audio envelope waveform image (default false). Applicable to audio recordings only. (音频包络图) | |
| transcript | No | "auto" (default): use subtitles first, ASR only as a fallback. "off": no transcription at all, visuals only. Applicable to video and audio only. | |
| transcriptFile | No | Absolute path to a local subtitle/transcript file (.srt, .vtt, .ass, .ssa). Applicable to video and audio only. (本地字幕/转写文件) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnlyHint and closed-world, and the description is fully consistent while adding substantial behavior: a four-step text-source precedence order, per-segment source tags, a strong warning not to quote ASR verbatim, and the fact that full text is written to disk (content.md, segments.jsonl, manifest.json) rather than inlined. It also discloses the key failure mode — storyboard cells are too small to read — that annotations cannot express.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Structure is strong — scoping sentence, per-media output list, source-precedence block, then caveats and return-value paths — and the most important warnings are front-loaded or visually separated. It is nonetheless verbose (bilingual restatement plus some overlap between the output list and the file-path paragraph) where a tighter version would lose nothing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Eleven parameters with no output schema make this a complex tool, and the description covers what an agent needs: what is produced per media type, that text lives at returned disk paths and should be read section by section, how to page through long media, and the quoting caveat for ASR. Nothing material is left for the agent to guess.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already carries parameter mechanics and the baseline would be 3. The description still adds motive that guides parameter choice: why windows must be used (illegible cells, spoilers), that start/end are video/audio-only while documents use pages, and the subtitle-vs-ASR consequence of transcript="auto" vs "off" via the source order. It stops short of adding format specifics beyond what the schema states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opening sentence names a specific verb and resource ("build an overview of a local media file") and immediately enumerates the three media types with the exact artifact produced for each. It also disambiguates the outputs from the sibling tools by naming media_zoom for reading text and media_probe for probing, so an agent can place this tool in the workflow without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit sequencing and exclusions: "Always call media_probe first", "To read text on screen, call media_zoom on that moment", and long media "must be narrowed with start/end or pages". It also tells the agent to follow the continuation hint rather than re-requesting the same range, which is precisely the kind of when/when-not guidance that prevents bad calls.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
media_probeProbe mediaARead-only
Inspect a local media file — video, audio recording, or document — and report what it is: duration, resolution, page count, codec, available tracks (audio / subtitle / text layer), and whether transcription (ASR) and OCR are available for this file. (读取本地媒体的元信息:时长、分辨率、页数、编码、可用轨道(音轨 / 字幕轨 / 文本层), 以及能否做 ASR / OCR 的初判。)
Call this FIRST on any file you have not inspected yet. It costs no image tokens and decodes nothing. Then pick the right next tool:
video / audio -> media_overview to read it
document -> media_overview to see page thumbnails and per-page text
a specific moment or page -> media_zoom for a close-up
"where was X said / on which page" -> media_locate for a timestamp or a page number
For audio it also reports a quick health check — average level, speech ratio and silence ratio — so you know whether the recording actually contains audible speech before you spend an ASR call on it. A recording that is effectively silent is reported as such and is never sent to ASR (no hallucinated transcript).
This server never downloads anything and never calls an LLM: you always pass a local absolute path, and it only prepares images and text for your own model to look at.
| Name | Required | Description | Default |
|---|---|---|---|
| path | Yes | Absolute path to a LOCAL file: video, audio recording, or document. URLs and relative paths are rejected — this server never downloads anything. (本地文件绝对路径) | |
| transcript | No | "auto" (default): use subtitles first, ASR only as a fallback. "off": no transcription at all, visuals only. Applicable to video and audio only. | |
| transcriptFile | No | Absolute path to a local subtitle/transcript file (.srt, .vtt, .ass, .ssa). Applicable to video and audio only. (本地字幕/转写文件) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnlyHint and openWorldHint=false; the description adds substantial context beyond that: it costs no image tokens, decodes nothing, never downloads, never calls an LLM, and requires a local absolute path. It also discloses the silent-recording gate (silent audio is never sent to ASR, preventing hallucinated transcripts), which is behavior an agent could not predict from the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well front-loaded: purpose first, routing second, behavioral caveats last. Nearly every sentence earns its place, though the parenthetical Chinese restatement of the purpose line is pure duplication and the routing list is somewhat verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries the return-value burden and does so thoroughly: metadata fields, track list, ASR/OCR readiness, and the audio health check (average level, speech ratio, silence ratio). It also covers the operational constraints (local path only, no downloads, no LLM) that an agent needs to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents path, transcript enum, and transcriptFile. The description reinforces the local-absolute-path constraint and mentions the ASR judgment, but adds no syntax, default, or interaction detail for the transcript/transcriptFile parameters beyond what the schema states. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (inspect/probe) and resource (local media file) and enumerates the concrete outputs it reports: duration, resolution, page count, codec, tracks, ASR/OCR availability. It also names each sibling routing target (media_overview, media_zoom, media_locate), so an agent can distinguish it from all of them without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Call this FIRST on any file you have not inspected yet" gives an unambiguous trigger condition, followed by explicit if/then routing to each sibling for video/audio, documents, specific moments, and 'where was X said' queries. Both when-to-use and which-alternative are stated, not inferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
media_zoomZoomARead-only
Get a close-up of one moment in a local video, or one page of a local document, returned as a single image. (取本地视频某一时刻的特写单帧,或本地文档某一页的整页 / 局部放大图。)
Use it after media_overview when one cell or page needs a closer look: "what exactly is on screen at 01:12?" "what does line 3 of page 7 say?"
For video, pass at (a single timestamp) or start/end (a window — the sharpest frame inside it is returned, and the exact timestamp is reported back). For a document, pass page, optionally with crop. crop is {x, y, width, height}, each a fraction 0..1 of the frame or page, origin top-left.
Keep it to one frame. It costs roughly 1,900 vision tokens — far more than a whole overview cell. Zoom the parts that matter instead of re-scanning the file.
This is the ONLY tool whose image is meant for reading. A storyboard cell is a few hundred pixels wide and the text inside it is unreadable; a cropped single frame at up to 1600 px is readable. So: locate in the overview grid, then zoom to read. The returned image is always source "visual" — what you read off it is your own reading of pixels, not a transcript, so treat it as an observation rather than a quotation.
| Name | Required | Description | Default |
|---|---|---|---|
| at | No | Exact timestamp in seconds for a video frame. Mutually exclusive with start/end. Applicable to video only. (精确时刻,秒) | |
| end | No | Window end in seconds (closed interval, both ends included). Applicable to video and audio only. Defaults to the end of the media; a value past the end is clipped and reported. (终点,秒) | |
| crop | No | Crop rectangle {x, y, width, height}: all four required, each a fraction 0..1 of the frame or page, origin top-left. Applicable to video and document only. (裁切区域,比例制) | |
| page | No | 1-based page number to render. Required for document media; not applicable to video. (页码) | |
| path | Yes | Absolute path to a LOCAL file: video, audio recording, or document. URLs and relative paths are rejected — this server never downloads anything. (本地文件绝对路径) | |
| start | No | Window start in seconds (closed interval, both ends included). Applicable to video and audio only; document media uses "pages" instead. Default 0. (起点,秒) | |
| maxEdge | No | Longest edge of each image in pixels: 256-1600. Default 1280 for overview, 1600 for zoom. Applicable to video and document only. (单张图最长边) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnly and closed-world annotations, the description discloses cost (~1,900 vision tokens), output nature (single image, source 'visual'), and an important interpretive limitation: reading pixels is an observation, not a transcript or quotation. It also explains that for video windows the sharpest frame is returned and the exact timestamp is reported back.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The English portion is front-loaded, clearly structured, and every sentence carries useful guidance about purpose, routing, cost, and interpretive caveats. The main inefficiency is the full bilingual duplication, which roughly doubles the description size without adding new semantic content for an agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 7-parameter tool with no output schema, the description is nearly complete: it explains the returned image, its visual source, the video/document modes, and the read-after-overview workflow. One gap is that the schema accepts audio paths and start/end for audio, while the description only frames the tool around video and documents.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description still adds meaningful semantics beyond the schema: at versus start/end behavior, the sharpest-frame-inside-window rule, exact timestamp reporting, and crop as fractional coordinates with top-left origin.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: get a close-up of one moment in a local video or one page of a local document, returned as a single image. It explicitly distinguishes itself from media_overview by saying the image is meant for reading, not just navigation, and by noting storyboard cells are too small to read.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It routes the agent explicitly: use this after media_overview when one cell or page needs a closer look, with concrete questions like 'what exactly is on screen at 01:12?' and 'what does line 3 of page 7 say?'. It also warns to keep it to one frame and zoom only the parts that matter rather than re-scanning the whole file.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
video_subtitlesSubtitlesARead-only
Read subtitles for a local media file and return them as timestamped segments. (读取本地字幕文件或视频内嵌字幕轨,解析成带时间轴的段落列表。)
Two sources, in this order: an explicit transcriptFile (a local .srt, .vtt, .ass or .ssa file), or the subtitle track embedded in the container. Auto-discovery also picks up a sibling file with the same base name in the same directory (a.srt next to a.mp4) when neither of the above is available — the source and the exact path are always reported back, never silently.
No image and no pixels spent: the result is text with millisecond and second timestamps. Use media_locate to search inside the subtitles. Use media_overview if you also want to see what is on screen.
| Name | Required | Description | Default |
|---|---|---|---|
| end | No | Window end in seconds (closed interval, both ends included). Applicable to video and audio only. Defaults to the end of the media; a value past the end is clipped and reported. (终点,秒) | |
| path | Yes | Absolute path to a LOCAL file: video, audio recording, or document. URLs and relative paths are rejected — this server never downloads anything. (本地文件绝对路径) | |
| pages | No | Page range for document media: comma-separated 1-based page numbers and closed ranges, e.g. "3-10,15". Not applicable to video or audio. (页码范围) | |
| start | No | Window start in seconds (closed interval, both ends included). Applicable to video and audio only; document media uses "pages" instead. Default 0. (起点,秒) | |
| transcriptFile | No | Absolute path to a local subtitle/transcript file (.srt, .vtt, .ass, .ssa). Applicable to video and audio only. (本地字幕/转写文件) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark this read-only and closed-world, and the description adds real context beyond them: the three-tier source fallback, the guarantee that source and exact path are always reported and 'never silently' resolved, and the cost note that no images/pixels are spent. That is exactly the kind of disclosure an agent needs for a fallback-laden tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core statement, then source order, then cost, then siblings — a clean information hierarchy. Slightly longer than necessary because of the mirrored Chinese parentheticals that duplicate the English content without adding information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description picks up the slack by characterizing the return value (text with millisecond/second timestamps) and the metadata echoed back (source and exact path). Nothing an agent needs to invoke this correctly appears to be missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema carries the mechanical burden (start/end/pages/transcriptFile all documented). The description still adds semantic value by explaining that transcriptFile takes precedence over the embedded track and that format support is limited to .srt/.vtt/.ass/.ssa, though it says nothing about how pages interacts for document media.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource+output shape ('Read subtitles for a local media file and return them as timestamped segments'), which is unambiguous and clearly distinct from siblings like media_probe or media_overview. An agent knows exactly what it gets back.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly routes the agent: use media_locate to search inside subtitles, use media_overview to also see on-screen content. It also defines the source resolution order (transcriptFile > embedded track > auto-discovered sibling), which is the key invocation decision.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
5 tool updates
v0.1.0- First observed
media_locate - First observed
media_overview - First observed
media_probe - First observed
media_zoom - First observed
video_subtitles
TDQS
Scored across 5 tools
Each tool has a documented role: media_probe inspects metadata, media_overview builds the grid/transcript, media_zoom reads one frame, media_locate searches text for a timestamp/page, and video_subtitles dumps subtitle segments. The overlap is mild but real: video_subtitles and media_overview both return timestamped text, and the descriptions spend considerable effort telling the agent which one to prefer. That overlap is resolvable from the docs, so boundaries are mostly clear.
Four of five tools use a clean media_<verb> pattern (media_probe, media_overview, media_zoom, media_locate). video_subtitles breaks the prefix and is also slightly misleading since it works on any media container, not just video. Otherwise consistent snake_case with predictable verbs.
Five tools map cleanly onto a probe -> overview -> zoom/search/transcript inspection pipeline, and each earns its place against a distinct cost model (free metadata, cheap grids, expensive single frames). Nothing is redundant or missing at the count level; it is well within the 3-15 sweet spot.
The lifecycle is covered: inspect, overview, drill down, search, and read subtitles, with ASR/OCR availability surfaced by media_probe and consumed by the other tools. Minor gaps remain: there is no explicit OCR/document-text tool despite OCR being reported, and no way to discover or list local media files, so the agent must already know absolute paths. These are workable around but not fully closed.
Maintenance
Related MCP Connectors
PDF, image, video, OCR, screenshot, SQL, QR and text tools for agents. No API key, no signup.
15 media & data tools for AI agents: search, transcribe, subtitles, voiceover, translate & more.
OCR, transcription, file extraction, and image generation for AI agents via MCP.
Public web tools for agents: product extraction, claim checks, webpage QA and ranked audits.
Related MCP Servers
- AlicenseNot gradedqualityAmaintenanceProvides local, offline transcription, keyframe extraction, OCR, and pre-publish review of audio, video, and image files, enabling AI agents to see and hear media without cloud or API keys.30 npmApache 2.0
- AlicenseAqualityCmaintenanceLets AI agents inspect local or remote videos by reading metadata, detecting shots and cuts, extracting frames as images, and checking keyframes for seeking and streaming, all via local FFmpeg without an API key.51MIT
- AlicenseNot gradedqualityCmaintenanceEnables AI agents to inspect UI screen recordings as token-efficient image sheets with changed regions boxed, single-frame flicker flagged, and speech aligned to frames. Exposes watch_video and zoom_video tools so agents can pull dense keyframes across any time range or one sharp full-resolution frame, all processed locally.0MIT
- AlicenseNot gradedqualityCmaintenanceEnables coding agents to ask focused questions about local screenshots, video, audio, and music and receive compact text reports on OCR, layout, timestamps, sounds, and music description. Media stays on the machine and raw files are never loaded into the agent's context, with support for cropping, interval sampling, and durable background jobs.MIT