MultiMediaGrep
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| MMGREP_ASR_MODEL | No | ASR model name. The default is provider-independent — with `openai-compatible` you must explicitly change it to something like `whisper-1`, otherwise `scribe_v1` is sent to the OpenAI-compatible service. | scribe_v1 |
| MMGREP_CACHE_DIR | No | Cache root directory (frames, page images, transcripts, full-text dumps all live here). `~` is expanded; relative paths resolve against cwd. | ~/.cache/multimediagrep |
| MMGREP_DOC_MODEL | No | Parsing model name; empty = service default (MinerU accepts `pipeline` / `vlm` / `MinerU-HTML`). | |
| MMGREP_LOG_LEVEL | No | Log level: `error` / `warn` / `info` / `debug`. Logs always go to stderr (stdout is the protocol channel). | info |
| MMGREP_ASR_API_KEY | No | ASR API key. Required once a provider is selected — even self-hosted local services need a non-empty value (anything works, e.g. `local`). | |
| MMGREP_DOC_API_KEY | No | Document parsing service token (MinerU: https://mineru.net/apiManage/token). | |
| MMGREP_FFMPEG_PATH | No | Absolute path to the ffmpeg executable when it is outside `PATH`; defaults to lookup via `PATH`. | |
| MMGREP_ASR_BASE_URL | No | ASR API base URL. The default is provider-independent — with `openai-compatible` you must explicitly change it to your service address. | https://api.elevenlabs.io/v1 |
| MMGREP_ASR_LANGUAGE | No | Force language (BCP-47, e.g. `zh` / `en`); `auto` = let the service decide. | auto |
| MMGREP_ASR_PROVIDER | No | ASR backend: `none` (no transcription) / `elevenlabs` / `openai-compatible`. | none |
| MMGREP_DOC_BASE_URL | No | Document parsing API base URL. | https://mineru.net/api/v4 |
| MMGREP_DOC_PROVIDER | No | Document parsing backend: `none` (page images only) / `mineru`. | none |
| MMGREP_FFPROBE_PATH | No | Absolute path to the ffprobe executable when it is outside `PATH`; defaults to lookup via `PATH`. | |
| MMGREP_OCR_PROVIDER | No | Reserved for an independent local OCR channel. Not implemented in the current version: OCR is handled by the document parsing provider (MinerU); this variable only appears in the startup banner and does not affect any tool behavior. | none |
| MMGREP_ASR_TIMEOUT_MS | No | ASR single-request timeout. | 120000 |
| MMGREP_DOC_TIMEOUT_MS | No | Document parsing single-request timeout (3 minutes). MinerU is upload -> queue -> parse -> download wall-clock time, so increase it if queuing is slow. | 180000 |
| MMGREP_MAX_IMAGE_EDGE | No | Global upper limit on the longest edge of a single image (px). Requests exceeding it return `BUDGET_EXCEEDED`. | 1600 |
| MMGREP_CACHE_MAX_BYTES | No | Cache size limit, supports `512MB` / `10GB` / raw byte counts; exceeding it evicts by LRU. | 10737418240 |
| MMGREP_MAX_CONCURRENCY | No | Maximum concurrency. | 2 |
| MMGREP_MAX_IMAGE_TOKENS | No | Soft image-token budget per call. When exceeded, auto mode downshapes; explicit mode only warns. | 4000 |
| MMGREP_MAX_TEXT_INLINE_CHARS | No | Maximum number of text characters returned inline. | 24000 |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| media_probeA | Inspect a local media file — video, audio recording, or document — and report what it is: duration, resolution, page count, codec, available tracks (audio / subtitle / text layer), and whether transcription (ASR) and OCR are available for this file. (读取本地媒体的元信息:时长、分辨率、页数、编码、可用轨道(音轨 / 字幕轨 / 文本层), 以及能否做 ASR / OCR 的初判。) Call this FIRST on any file you have not inspected yet. It costs no image tokens and decodes nothing. Then pick the right next tool:
For documents it also checks whether the text layer looks unreliable — U+FFFD replacement characters, private-use characters, isolated single letters, symbol noise — and, when it does, points at media_overview({ path, ocr: "force" }) to re-recognize the pages. (文档文字层看起来不可靠时会提示,可用 ocr: "force" 强制重新识别) For audio it also reports a quick health check — average level, speech ratio and silence ratio — so you know whether the recording actually contains audible speech before you spend an ASR call on it. A recording that is effectively silent is reported as such and is never sent to ASR (no hallucinated transcript). It also shows ONE outline for the file, from the first available source: the bookmark sidecar (".mmgrep.bookmark.json", written by media_bookmark) when one exists, otherwise the file's own outline — PDF bookmarks (PHYSICAL page numbers, the same 1-based numbers media_overview accepts) or chapter markers in video/audio (title + start/end timestamps). Every line carries a bookmark id ("b3 p.12 title" for PDFs, "b3 00:12:34 title" for video/audio) — these are the ids media_bookmark accepts. When a sidecar exists, the last outline line is "full detail: ": read that file for the complete outline. The text shows at most 2 levels / 60 entries; structuredContent.outline has every entry, the source and the sidecar path. Read-only: probing never creates or modifies a sidecar. (读取侧只展示一份目录:sidecar 优先,其次文件自带目录;行首 b1…bN 就是 media_bookmark 的 id) It also reports the ANNOTATIONS of the file: how many there are and where they live — the annotation sidecar (".mmgrep.annotation.json", written by media_annotate) when one exists, otherwise the file's own annotations (PDF annotation objects). Probe reports the COUNT and the sidecar path only; the notes themselves are shown by media_overview, next to the page / time range they belong to ("n3 (p.12): note" for PDFs, "n3 (00:12:34–00:12:39): note" for video/audio). media_locate can also search the annotation text. Read-only: probing never creates or modifies a sidecar. (读取侧也展示批注:条数 + 侧车路径;note 本身在 overview 里按页 / 时间范围展示) This server never downloads anything and never calls an LLM: you always pass a local absolute path, and it only prepares images and text for your own model to look at. |
| media_overviewA | Build an overview of a local media file so you can read it without guessing. (生成媒体总览:视频出分镜宫格图与按时间对齐的文本清单;录音出带时间轴的转写;文档出页面缩略图与每页文字。) What you get depends on the media:
For DOCUMENTS, ocr: "auto" (default) uses the PDF text layer where one exists and calls the OCR provider only for pages without one; ocr: "force" re-recognizes every requested page with the OCR provider, even pages that already have a text layer, and tags them (ocr). Use force when media_probe says "text layer: looks unreliable" or when the extracted text is visibly garbled. Forced results are cached separately from auto. (文档:ocr: "force" = 强制重新 OCR / 重新识别文字层) Pass images: 0 for a text-only, zero-pixel overview: no grid is rendered and no frame is decoded (a video costs no ffmpeg work at all), so you get only the transcript — or the page text for a document — and the reported budget is all zeros. Use it when you only need the words. TEXT SOURCE ORDER — subtitles first, ASR only as a fallback:
The storyboard grid is for FINDING things, not for reading them: cells are small, so text inside a cell is usually unreadable. To read text on screen, call media_zoom on that moment and read the cropped single frame. The full text is NOT inlined. It is written to disk, and the text of the result ends with a line like "full text: /abs/path/content.md (segments.jsonl, manifest.json in the same directory)". That absolute path is what you read: content.md is the readable full text, segments.jsonl has one segment per line, manifest.json records what was produced and how. Read those files section by section and summarise them in chunks yourself — do not ask for the same range again just to get more inline text. If the export could not be written, the line reads "full text: NOT written — " instead. The inline text is a PREVIEW with three tiers. (1) WHOLE: the requested range fits, and the result says "complete text ... nothing omitted". (2) SKIM (documents only, when the document parser exposes block structure): every heading plus the first complete sentence of each paragraph, tables as "[table: caption]", with captions/footnotes/equations left out; each page is labelled like "[page 4 · skim: 12 of 210 sentences]". A skim built from the built-in heuristic blocks is best-effort — section structure may be wrong; read content.md for the authoritative text. (3) SAMPLED: complete units — document sentences, video/audio transcript segments — are sampled evenly, with a line like "preview: sampled 84 of 312 sentences across pages 1-28; not represented: 2-6, 8-12". In a skim that still does not fit, all headings are kept and sentences are sampled; only when even the headings do not fit are the headings sampled. Sampling never drops a whole page silently: pages that could not be represented are listed, and a single over-long unit is truncated with a trailing "…[+N chars]". Read content.md when you need the complete text. VIEW — the view parameter controls the inline text (default "auto"). The result carries ONE outline from the first available source: the bookmark sidecar (".mmgrep.bookmark.json", written by media_bookmark) when one exists, otherwise the file's own outline (PDF bookmarks, or chapter markers in video/audio); a document without bookmarks may get an outline GENERATED FROM ITS HEADINGS when the range has at least 5 headings, labelled "outline: generated from headings" (best-effort). (1) "auto": if the requested range fits, you get the full text; if it does not fit and there is an outline, the outline comes first (ids b1…bN with page numbers for PDFs, timestamps for chapters) and the rest of the budget goes to the text preview; if there is no outline, you get the normal skim/sampled preview. (2) "outline": return only the outline — generated from headings for a document without bookmarks; for video/audio without chapters you get an explicit "no chapters" message and NOTHING is generated, because this server never invents an outline from content. In that case read the overview (or a start/end window) and write your own outline from the time-aligned text and storyboard grids. (3) "text": ignore the outline and force the text pipeline. The full outline (entries, source, sidecar path) is always in structuredContent.outline and, for an overview that writes an export, in manifest.json. When a sidecar exists its last outline line is "full detail: " — read that file for the complete bookmark detail. (目录:sidecar 优先,其次文件自带目录,最后才是 heading 生成;view:auto 放不下时先出目录;view:outline 只出目录;view:text 只出文本) The result also carries the ANNOTATIONS of the requested range, right next to the page / time range they belong to: the annotation sidecar (".mmgrep.annotation.json", written by media_annotate) when one exists, otherwise the file's own annotations (PDF annotation objects). One annotation per line, in the same shape media_annotate writes ("n3 (p.12): note" for PDFs, "n3 (00:12:34–00:12:39): note" for video/audio). The line starts with the annotation id (n1, n2, …) — these are the ids media_annotate accepts. The author is in structuredContent.annotations only; the note text is shown as written. (批注按请求范围展示:行首 n1…nN 就是 media_annotate 的 id) Always call media_probe first: it tells you how long the file is, whether a subtitle source exists, which transcript source applies, and which outline applies (bookmark sidecar first, then the file's own PDF bookmarks / video-audio chapters, or headings for a document), for free. Long media must be narrowed with start/end (video, audio) or pages (document) — one giant grid of a two-hour film gives you illegible cells and spoils the ending. Follow the continuation hint in the result to page through the rest. |
| media_zoomA | Get a close-up of one moment in a local video, or one page of a local document, returned as a single image. (取本地视频某一时刻的特写单帧,或本地文档某一页的整页 / 局部放大图。) Use it after media_overview when one cell or page needs a closer look: "what exactly is on screen at 01:12?" "what does line 3 of page 7 say?" For video, pass at (a single timestamp) or start/end (a window — the sharpest frame inside it is returned, and the exact timestamp is reported back). For a document, pass page, optionally with crop. crop is {x, y, width, height}, each a fraction 0..1 of the frame or page, origin top-left. Keep it to one frame. It costs roughly 1,900 vision tokens — far more than a whole overview cell. Zoom the parts that matter instead of re-scanning the file. This is the ONLY tool whose image is meant for reading. A storyboard cell is a few hundred pixels wide and the text inside it is unreadable; a cropped single frame at up to 1600 px is readable. So: locate in the overview grid, then zoom to read. The returned image is always source "visual" — what you read off it is your own reading of pixels, not a transcript, so treat it as an observation rather than a quotation. |
| media_locateA | Find where something appears in a local media file and return its position: a timestamp (milliseconds and seconds) for a video or recording, a page number for a document. (定位:给一段文本或关键词,返回它在视频 / 录音里的时间戳,在文档里的页码。) Use it when you know roughly what was said but not when ("where does it mention X"), or when you need to know which page a phrase is on. Matching ignores punctuation, spacing and case, and works for both Chinese and English. Long phrases match better than single characters. For a video or recording, each match is a time window with a match score (0..1) and the matched line. For a document, each match is a page: the page number, how many times the query appears on that page (hits), and a short snippet around the first hit — never the whole page. Pages that matched but did not fit are listed as "N more pages match: ...". Read a full page with media_overview({ path, pages }). For DOCUMENTS, ocr: "auto" (default) searches the PDF text layer where one exists and OCRs only pages without one; ocr: "force" re-recognizes every requested page with the OCR provider first, even pages that already have a text layer, and tags those matches (ocr). Use force when media_probe says "text layer: looks unreliable". Forced results are cached separately from auto. (文档:ocr: "force" = 强制重新 OCR / 重新识别文字层) Text only by default — no image, no pixels spent. Add includeFrame: true to also get the picture at the best match (one image, roughly 1,900 vision tokens). A transcript is required: the embedded subtitle track, a subtitle you select explicitly, or ASR configured via MMGREP_ASR_PROVIDER. Subtitles win over ASR. If none is available this tool says so instead of guessing. Use media_overview if you only need the visuals. Each match is tagged with its source. A match whose source is "asr" came from machine transcription, so the wording may be wrong (names, numbers, jargon) — do not present it as the speaker's exact words. The result also carries the same single outline as media_probe / media_overview in structuredContent.outline: the bookmark sidecar first, then the file's own PDF bookmarks or chapter markers. Its entries carry the ids media_bookmark accepts (b1…bN), and structuredContent.outline.sidecarPath / sidecarError point at the bookmark sidecar when one exists. Locate is read-only: it never creates or modifies a sidecar. (定位结果也带同一份目录:sidecar 优先,其次文件自带目录;line 首 b1…bN 可用于 media_bookmark) It also searches the ANNOTATION text: the notes written next to the media (annotation sidecar ".mmgrep.annotation.json", or the file's own PDF annotations) are searched too, and annotation hits are reported in their own "annotation matches" section. They are marked as annotations (kind "annotation" plus the id, n1, n2, …) and are NEVER counted as media content hits — the match count above them counts transcript / page matches only. An annotation hit is a note somebody wrote; it is not something the media says. (定位也搜批注文本;批注命中单独成段,不计入媒体命中数) |
| media_bookmarkA | Write bookmarks (an outline, chapters) into a local media file so they survive across sessions and are visible to humans in the file itself: the PDF outline for a PDF, a sidecar VTT for video/audio, and a sidecar JSON for every supported media type. (写入书签 / 目录 / 章节:PDF 改写文件自带目录;视频、音频写同名 .mmgrep.bookmark.vtt; 所有类型都写 .mmgrep.bookmark.json 侧车。中文触发词:书签、目录、章节、bookmark、outline、chapter。) The sidecar ".mmgrep.bookmark.json" is the single source of truth. The first write (no sidecar yet) imports the file's own outline — PDF bookmarks, or video/audio chapter markers — as b1…bN in file order, then applies this call. New ids are max+1 and are never reused; update keeps the id. One call can mix add / remove / update; they are applied remove → update → add, all-or-nothing: if anything is invalid, nothing is written and every problem is listed at once (unknown id, bad field, page/time out of range, duplicate ids, malformed entry, …). An image has no native outline and gets the sidecar only; text and Office documents are not supported. WARNING — this tool modifies files:
After the write, media_probe / media_overview / media_locate show this outline (ids b1…bN) instead of the file's native one. The result contains the full outline, the sidecar path, the embed target, and every check result. (书签 / 目录 / 章节 / bookmark / outline / chapter) |
| media_annotateA | Write annotations (notes, comments, sticky notes, marked areas) into a local media file so they survive across sessions and are visible to humans in the file itself: real PDF annotation objects for a PDF, a sidecar VTT for video/audio, and a sidecar JSON for every supported media type. (写入批注 / 注释 / 评论 / 标注:PDF 原地写真正的批注对象;视频、音频写同名 .mmgrep.annotation.vtt;所有类型都写 .mmgrep.annotation.json 侧车。 中文触发词:批注、注释、评论、标注、annotate、comment、note。) The sidecar ".mmgrep.annotation.json" is the single source of truth. The first write (no sidecar yet) imports every note-like annotation of a PDF — Text, FreeText, Square, Circle, Line, Polygon, PolyLine, Highlight, Underline, Squiggly, StrikeOut, Stamp, Caret and Ink, in page order — as n1…nN, including its raw geometry and original author, then applies this call. Link annotations, form widgets and popups are never imported or touched. Entries are anchored per media type:
Ids are n1, n2, …; new ids are max+1 and are never reused by this call; update keeps the id. One call can mix add / remove / update; they are applied remove → update → add, all-or-nothing: if anything is invalid, nothing is written and every problem is listed at once (unknown id, bad field, page/time out of range, duplicate ids, malformed entry, …). The author is resolved once per call: the "author" argument, else the MCP client's name, else "MultiMediaGrep". The model name is never used. Imported annotations keep their original author; update never changes it. WARNING — this tool modifies files:
The result lists every written annotation in the same shape the read side uses ("n1 (p.12): note", "n1 (00:12:34–00:12:39): note"), plus the sidecar path, the embed target, and every check result. (批注 / 注释 / 评论 / 标注 / annotate / comment / note) |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 6 tools
Each tool occupies a distinct slot in the workflow: probe (metadata/health), overview (read content), locate (search text), zoom (read one frame/page), annotate (write notes), bookmark (write outline). The descriptions explicitly disambiguate the closest pair (overview grid = finding, zoom = reading) and warn against misselection. Overlap in secondary outputs (probe/overview/locate all surface the outline) does not create real selection ambiguity because primary purposes are clearly separated.
All six tools follow an identical media_<verb/noun> pattern with a shared prefix: media_probe, media_overview, media_locate, media_zoom, media_annotate, media_bookmark. No mixing of conventions or stray abbreviations.
Six tools is well-scoped for a local media inspection server: four read-side (probe, overview, locate, zoom) and two write-side (annotate, bookmark), each covering a genuinely different operation. No tool appears redundant or vestigial.
The surface covers the full lifecycle for its domain: inspect, read (text + visual), search, close-up, and persist bookmarks/annotations including add/update/remove. Minor gaps remain, such as no batch/multi-file operation and no dedicated way to discard or reset a sidecar wholesale, but these are workable via the existing remove operations.