process_media
Transcribes and indexes local video or audio files by validating the file, transcribing speech with Whisper, extracting keyframes, and OCRing text. Returns a compact summary for querying with other tools.
Instructions
Ingest a LOCAL video or audio file and make it queryable: validates the file, transcribes speech locally (whisper), extracts scene-change keyframes, OCRs on-screen text, and resolves the wall-clock start time. Returns a compact summary (job_id, media info, wall_clock, transcript preview) — full data stays on disk and is served lazily by the other tools. Idempotent by content hash: re-calling on an already-processed file returns instantly. When NOT to use: to re-fetch data you already processed (use the retrieval tools), or for URLs — local file paths only. Examples:
process_media(path="/Users/sam/Desktop/bug-repro.mov") — narrated screencast, defaults are right
process_media(path="~/Videos/demo.mp4", language="en") — pin the language, skip auto-detect
process_media(path="/rec/interview.mov", model="large-v3-turbo") — best multilingual quality (1.5 GB, one-time)
process_media(path="/tmp/standup.m4a") — audio-only: transcript tools work, frame tools will error
process_media(path="/rec/review.mov", vocabulary="OKR, PgBouncer, Kanban") — jargon survives STT
process_media(path="/rec/demo.mov", recorded_at="2026-07-10T12:03:00+02:00") — exact wall-clock anchor
process_media(path="/rec/demo.mov", recorded_at="2026-07-10T12:03:00+02:00", force=true) — re-anchor a done job
user: "I just recorded my screen, it's on my Desktop" → process_media(path="/Users//Desktop/.mov")
user drops a browser tab capture → process_media(path="~/Downloads/tab-capture.webm")
summary shows wall_clock=null → ask when recording started, re-call with recorded_at=... and force=true
transcript garbled or language_probability low → re-call with model="large-v3-turbo" (or language="ru") + force=true
30-min video is fine: progress notifications stream while whisper runs; expect minutes, not seconds
after success, do NOT dump everything — continue with get_transcript / get_moment / search on the job_id
anti-example: frames from an already-processed job → get_frames(job_id=...), never process_media again
anti-example: YouTube/URL input → unsupported in v1; have the user download the file first
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| path | Yes | ||
| force | No | ||
| model | No | ||
| language | No | ||
| vocabulary | No | ||
| recorded_at | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||