Ingest a LOCAL video or audio file and make it queryable: validates the file, transcribes speech locally (whisper), extracts scene-change keyframes, OCRs on-screen text, resolves the wall-clock start time, and (opt-in) labels who said what via local speaker diarization. Returns a compact summary (job_id, media info, wall_clock, transcript preview, speaker roster when diarized) — full data stays on disk and is served lazily by the other tools. Idempotent by content hash: re-calling on an already-processed file returns instantly. For MULTI-PERSON recordings (meetings, interviews, calls) diarize=true is part of a proper analysis — pass it even when the user only asks for a summary. When NOT to use: to re-fetch data you already processed (use the retrieval tools), or for URLs — local file paths only. Examples:

• process_media(path="/Users/sam/Desktop/bug-repro.mov") — narrated screencast, defaults are right

• meetings: model="large-v3-turbo" + vocabulary=<attendees, terms> + num_speakers=N — turbo's extra cost is trivial

• process_media(path="/tmp/standup.m4a") — audio-only: transcript tools work, frame tools will error

• process_media(path="/rec/panel.mov", diarize=true, num_speakers=4) — headcount known? ALWAYS pass it: best accuracy

• job already processed + diarize=true → amend re-runs ONLY diarization (whisper reused; long files: minutes)

• error mentions [diarization] → the extra is missing: install via uvx "talkthrough-mcp[diarization]"

• know the attendees? process_media(path=..., vocabulary="Anastasia, Evgenii, OKR") — names+jargon survive STT

• user: "analyze/summarize this meeting" → include diarize=true — speaker structure is not optional extra credit

• noisy threshold roster (clusters ≫ people)? ASK your user for the real headcount, then re-run with num_speakers=N

• cap_hit or sampling_interval_s in summary → for slide hunts raise TALKTHROUGH_MAX_FRAMES or use extract_frame

• summary shows wall_clock=null → ask when recording started, re-call with recorded_at=... and force=true

• transcript garbled or language_probability low → re-call with model="large-v3-turbo" (or language="ru") + force=true

• after success, do NOT dump everything — continue with get_transcript / get_moment / search on the job_id

• anti-example: frames from an already-processed job → get_frames(job_id=...), never process_media again

• anti-example: YouTube/URL input → unsupported in v1; have the user download the file first

Retrieve the transcript of a processed job, lazily and paginated. Formats: "segments" (default — seq, t_ms, t_wall when known, speaker when diarized, text), "text" (plain prose; "S1:" prefixes at speaker changes), "srt" (subtitles, speaker-prefixed cues). Diarized jobs also return the speaker roster (talk time, turns) in the header. Responses are capped (~8k tokens): when truncated=true, continue from the returned next_start_ms. When NOT to use: to find one keyword (use search) or to inspect one moment with visuals (use get_moment). Examples:

• get_transcript(job_id="a1b2c3d4e5f60718") — whole transcript of a short recording

• get_transcript(job_id="a1b2c3d4e5f60718", start_ms=0, end_ms=120000) — just the first two minutes

• get_transcript(job_id="...", format="text") — prose block for summarization

• get_transcript(job_id="...", format="srt") — subtitle export the user asked for

• diarized job: segments carry "speaker" + a roster header (top-12 by talk time; speakers_truncated counts the rest)

• "what did S2 say?" → format="segments", collect entries with speaker=="S2" (labels are in order of first voice)

• got truncated=true with next_start_ms=421500 → get_transcript(job_id="...", start_ms=421500)

• user: "what was said between 5:00 and 6:30?" → start_ms=300000, end_ms=390000

• meeting recording (audio-only job): this tool is the main surface — frames don't exist there

• correlate speech with logs: each segment's t_wall lines up with your log timestamps

• wall_clock=null on the job → segments carry t_ms only (relative to video start)

• 60-min video: page by ranges (e.g. 10-min windows), don't pull from 0 repeatedly

• no speaker fields on a meeting job → re-run process_media with diarize=true (adds them without re-transcribing)

• anti-example: "where did they mention checkout?" → search(job_id, "checkout"), not full paging

• anti-example: screenshots around a remark → get_moment(job_id, start_ms, end_ms)

Fetch stored keyframe images (JPEG, <=1568px wide) as MCP image content: the frames nearest to at_ms, OR unique frames across [start_ms, end_ms] evenly thinned to max_frames. Serves unique frames by default (near-duplicates from static scenes are filtered); hard cap 6 images per call. When NOT to use: exact instants between keyframes or native-resolution detail (use extract_frame), or finding on-screen text (use search — OCR text is indexed). Examples:

• get_frames(job_id="...", at_ms=83500) — what was on screen when the remark at 1:23.5 was spoken

• get_frames(job_id="...", at_ms=83500, max_frames=2) — tighter context, fewer tokens

• get_frames(job_id="...", start_ms=0, end_ms=600000, max_frames=6) — overview strip of the first 10 min

• get_frames(job_id="...", start_ms=290000, end_ms=310000, include_duplicates=true) — every capture near 5:00

• transcript hit at t_ms=421500 → get_frames(job_id, at_ms=421500) for the visual evidence

• walking a demo scene by scene → one ranged call per scene beats one giant range

• frame files are named by video-ms (t00083500.jpg ↔ t_ms 83500) — stable refs for findings

• valid_from_ms/valid_to_ms on each frame = when the screen looked like this — check the span covers your moment

• keep max_frames at 2-4 unless you are truly comparing scenes; images are token-expensive

• every frame entry carries "path" (absolute) — save/copy the image elsewhere with your own file tools

• audio-only job → this tool errors by design; use get_transcript / get_moment instead

• anti-example: need EXACTLY 12:34.500 between two keyframes → extract_frame(job_id, at_ms=754500)

• anti-example: "find the screen with the red error banner" → search(job_id, "error") first, then jump

The "one remark" evidence bundle: transcript slice + up to 3 unique frames + their OCR text + the wall-clock range for [start_ms, end_ms], in a single call. This is the workhorse for triage: one call per finding gives you the quote, the screenshot, and the on-screen text. When NOT to use: broad exploration (get_transcript / get_frames) or keyword lookup (search). Examples:

• get_moment(job_id="...", start_ms=83000, end_ms=97000) — full evidence for the remark at 1:23-1:37

• segment seq 12 spans t0_ms=83210, t1_ms=96800 → get_moment(job_id, 83210, 96800)

• pad ±2000 ms around the spoken range — narrators react to things already on screen

• triage loop: for each candidate finding, exactly one get_moment call → quote + frame + OCR

• user: "what was I showing when I said 'this button is broken'?" → search first, then get_moment at the hit

• opening context of a meeting: get_moment(job_id, 0, 15000)

• response includes the t_wall range when known → quote it in bug reports for log correlation

• diarized job → speakers_in_range + speaker on each segment: who is talking in this window, at a glance

• frame entries carry "path" (absolute) — copy the screenshot elsewhere with your own file tools

• "was X on screen at t?" → yes iff some frame's valid_from_ms <= t < valid_to_ms — no extra calls needed

• audio-only job → returns the transcript slice plus a no-frames note (that is expected)

• anti-example: whole-video summary → get_transcript(format="text"), not a chain of get_moments

• anti-example: need more than 3 frames of a range → get_frames(start_ms=..., end_ms=..., max_frames=6)

• keep ranges under ~30 s; a 5-min "moment" dilutes the bundle and wastes tokens

Case-insensitive word search across BOTH transcript segments and frame OCR text: a text hits when EVERY word of the query matches it as a substring — any order, any distance (ё and е are interchangeable). Hits carry source (transcript|ocr), t_ms, t_wall when known, the matched text, and the nearest frame position — everything needed to jump straight to evidence. Optional speaker="S2" narrows to one voice's transcript hits. No embeddings. When NOT to use: fuzzy/semantic questions ("anything about performance?") — page get_transcript and read; regex is not supported. Examples:

• search(job_id="...", query="login") — every spoken or on-screen mention of login

• user: "what did I say about the login button?" → search(job_id, "login button") → get_moment at hits

• search(job_id, "TypeError") — on-screen stack traces and error text are OCR-indexed; great for bug repros

• search(job_id, "€49") — prices, IDs, and literals on screen are findable via OCR

• take hit.t_wall and grep your server logs ±30 s around it to pair remark ↔ log line

• no hits? shorten the stem: "notif" matches notification / notifications / notify

• multi-word = ALL words as substrings, any order: "first phase" hits "the first real phase"

• stems beat inflected phrases: "кнопк отправк" finds «Кнопка отправки» and «кнопку отправки»

• every hit has nearest_frame_ms → get_frames(job_id, at_ms=) shows the moment

• diarized job: transcript hits carry "speaker" — "who mentioned the deadline?" is answered by the hit itself

• search(job_id, "deadline", speaker="S2") — only S2's mentions; OCR hits are excluded (screens have no voice)

• audio-only job → transcript hits only (there is no OCR index)

• anti-example: "summarize the pricing discussion" → get_transcript(format="text") and read it

• anti-example: finding an icon or layout glitch with no text → get_frames over the range; OCR sees text only

• anti-example: "everything S2 said" → get_transcript and collect speaker=="S2" — search always needs a query

Re-extract ONE frame at an exact timestamp from the ORIGINAL source video at native resolution, with an optional crop {x, y, w, h} in source pixels. Use when the stored keyframes miss the instant (they capture scene changes + a 1 fps floor) or when you need full-resolution detail. Slower than get_frames — it decodes the source file, which must still exist at its recorded path. When NOT to use: normal browsing — get_frames serves stored keyframes instantly without touching the source. Examples:

• keyframes sit at 12:31 and 12:38 but the flash happened at 12:34.5 → extract_frame(job_id, at_ms=754500)

• extract_frame(job_id="...", at_ms=754500, crop={"x":800,"y":40,"w":400,"h":120}) — zoom into the toast text

• tiny UI text unreadable in the 1568px keyframe → extract_frame at the same ms for native resolution

• verify a one-frame glitch: extract_frame at 12300, 12400, 12500 and compare

• OCR missed small text → extract_frame with a tight crop, then read the returned image

• crop coordinates are SOURCE pixels (a Retina screen recording may be 2940x1912) — not keyframe scale

• response JSON carries "path" (absolute) — "save this screenshot next to my docs" = copy from path yourself

• source file moved or deleted → clear error; stored keyframes via get_frames still work

• audio-only job → always errors: there is no video stream to decode

• anti-example: "show me around 5:00" → get_frames(job_id, at_ms=300000); extract_frame is for exact instants

• anti-example: scanning a range frame by frame → get_frames(start_ms, end_ms) first, refine once after

List processed recordings, newest first: job_id, source filename, duration, created, wall-clock start, segment/frame counts. The store is content-addressed — the same file maps to the same job even after renames or moves, and jobs persist across sessions and machines restarts. When NOT to use: as a health check or before every call — job_ids are stable, remember them. Examples:

• user: "triage the recording I processed this morning" → list_jobs() → pick by filename + created

• user names neither job_id nor path → list_jobs() first; only ask if still ambiguous

• resume yesterday's analysis in a fresh conversation → list_jobs() → reuse its job_id directly

• file was renamed after processing → match by duration/created; the content hash ignores names

• wall_clock.start answers "WHEN was this session?" — pick the job from "yesterday around 15:00"

• after CLI batch pre-processing (talkthrough-mcp process big.mov) the job shows up here — query it

• two jobs with the same filename → the newer created one is usually the re-recording

• diarized jobs show "speakers": N — "the 4-person meeting from Tuesday" is findable at a glance

• empty list → nothing processed on this machine yet; ask the user for a file path

• job disappeared → likely talkthrough-mcp gc cleaned it; re-run process_media on the file (same id)

• anti-example: checking whether a NEW file is processed → just call process_media, it is idempotent+instant