Ingest a LOCAL video or audio file and make it queryable: validates the file, transcribes speech locally (whisper), extracts scene-change keyframes, OCRs on-screen text, resolves the wall-clock start time, and (opt-in) labels who said what via local speaker diarization. Returns a compact summary (job_id, media info, wall_clock, transcript preview, speaker roster when diarized) — full data stays on disk and is served lazily by the other tools. Idempotent by content hash: re-calling on an already-processed file returns instantly. For MULTI-PERSON recordings (meetings, interviews, calls) diarize=true is part of a proper analysis — pass it even when the user only asks for a summary. num_speakers is a target the clusterer may not reach, not a constraint — the payload says when a re-run changed nothing (labels_changed). If an amend changes the labels, verified names become pending-review evidence rather than active identities, with old-roster anchors for re-checking. Current pending labels can be confirmed/replaced/removed; stale labels can only be removed with an explicit null patch. Full force reprocessing of a job with saved or pending identities requires diarize=true and preserves every old identity as pending review against the rebuilt roster; without diarization it refuses before changing the stored job. When NOT to use: to re-fetch data you already processed (use the retrieval tools), or for URLs — local file paths only; a public video/audio URL goes to process_url. Examples:

• process_media(path="/Users/sam/Desktop/bug-repro.mov") — narrated screencast, defaults are right

• meetings: model="large-v3-turbo" + vocabulary=<attendees, terms> + num_speakers=N — turbo's extra cost is trivial

• process_media(path="/tmp/standup.m4a") — audio-only: transcript tools work, frame tools will error

• process_media(path="/rec/panel.mov", diarize=true, num_speakers=4) — headcount known? ALWAYS pass it: best accuracy

• relabel amend → names become pending with old anchors; stale labels are removable only with null

• error mentions [diarization] → run uvx --python ">=3.11,<3.14" "talkthrough-mcp[diarization]"

• know the attendees? process_media(path=..., vocabulary="Anastasia, Evgenii, OKR") — names+jargon survive STT

• user: "analyze/summarize this meeting" → include diarize=true — speaker structure is not optional extra credit

• noisy threshold roster (clusters ≫ people)? ASK your user for the real headcount, then re-run with num_speakers=N

• cap_hit or sampling_interval_s in summary → for slide hunts raise TALKTHROUGH_MAX_FRAMES or use extract_frame

• summary shows wall_clock=null → ask when recording started, re-call with recorded_at=... and force=true

• transcript garbled or language_probability low → re-call with model="large-v3-turbo" (or language="ru") + force=true

• after success, do NOT dump everything — continue with get_transcript / get_moment / search on the job_id

• anti-example: frames from an already-processed job → get_frames(job_id=...), never process_media again

• named job + force=true → include diarize=true; old identities return as pending review, never silently vanish

Download ONE public video/audio URL once (this is the only tool that uses the network), then run the same LOCAL pipeline as process_media: transcript, keyframes, OCR, wall-clock, optional diarization. Supported: direct https:// links to a media file (mp4/mov/webm/mkv/ogv/m4a/mp3/wav/ogg/flac), one public YouTube video (watch, youtu.be, shorts, a completed live), and any public video PAGE yt-dlp can read — Instagram (public reels/posts), TikTok, Wikimedia Commons, pages with an HTML5/HLS player, other sites as far as their yt-dlp extractor works anonymously (Vimeo does not). Not supported: playlists, channels, active live streams, private/members-only/age-restricted/DRM videos, cookies/logins; sites that hide a video behind a login or a bot wall fail with a clear reason. The downloaded source is kept inside the job, so extract_frame works later without network; the raw URL is never stored (only a hash, the provider id/host and a bounded title). A repeat call on the same URL serves the stored job without touching the network unless refresh=true. Job ids stay content hashes: the same video from two URLs is one job. YouTube and other pages need the optional [url] extra. The provider's upload date is NOT the recording start: wall_clock stays null unless recorded_at is passed. When NOT to use: for local files (process_media), or to re-fetch data you already processed (use the retrieval tools). Examples:

• process_url(url="https://youtu.be/nHfGfEiVdE8") — one public YouTube video, defaults are right

• process_url(url="https://www.youtube.com/watch?v=ID&list=PL...") — the playlist part is ignored: ONE video

• process_url(url="https://cdn.example.com/recordings/standup.mp4") — direct https link to a media file

• process_url(url="https://www.tiktok.com/@nasa/video/7…") — a public video page; origin.provider names the site

• meeting from a link: process_url(url=..., diarize=true, num_speakers=3, vocabulary="Vera, Tom, OKR")

• non-English narration: process_url(url=..., model="large-v3-turbo", language="ru")

• known recording start: process_url(url=..., recorded_at="2026-09-05T14:00:00+02:00") — enables t_wall

• the video changed on the provider → process_url(url=..., refresh=true): new download, maybe a new job_id

• re-anchor or change the model on a stored URL job → process_url(url=..., recorded_at=..., force=true), no download

• error mentions [url] → run uvx --python ">=3.11,<3.14" "talkthrough-mcp[diarization,url]" and restart

• playlist / channel / live / private URL → clear error; pass a single public video URL instead

• "bot check"/"sign-in" refusal on Instagram/TikTok → the site blocked anonymous access; report it, no workaround

• origin.published_at is the provider's upload time, not when the recording was made — never use it as t_wall

• after success continue with get_transcript / search / get_moment on the job_id — never re-download

• anti-example: a file on disk → process_media(path=...); process_url is only for https URLs

Retrieve the transcript of a processed job, lazily and paginated. Formats: "segments" (default — seq, t_ms, t_wall when known, speaker when diarized, text), "text" (plain prose; "S1:" prefixes at speaker changes), "srt" (subtitles, speaker-prefixed cues). Diarized jobs also return the roster, attribution_precision, saved speaker_name values, raw OCR name_candidates, and bounded pending-review names plus old-roster context after a relabel. Pre-0.3.1 video jobs may return name_candidates_note because their flat OCR is readable but less useful for hints. Pending names are evidence to re-check, never active identities. A stale pending label can only be removed with label_speakers(..., labels={"Sx":null}). Raw S labels remain canonical. Responses are capped (~8k tokens): when truncated=true, continue from the returned next_start_ms. When NOT to use: to find one keyword (use search) or to inspect one moment with visuals (use get_moment). Examples:

• get_transcript(job_id="a1b2c3d4e5f60718") — whole transcript of a short recording

• get_transcript(job_id="a1b2c3d4e5f60718", start_ms=0, end_ms=120000) — just the first two minutes

• get_transcript(job_id="...", format="text") — prose block for summarization

• get_transcript(job_id="...", format="srt") — subtitle export the user asked for

• diarized job: segments carry "speaker" + a roster header (top-12 by talk time; speakers_truncated counts the rest)

• "what did S2 say?" → format="segments", collect entries with speaker=="S2" (labels are in order of first voice)

• got truncated=true with next_start_ms=421500 → get_transcript(job_id="...", start_ms=421500)

• user: "what was said between 5:00 and 6:30?" → start_ms=300000, end_ms=390000

• legacy video name_candidates_note → explain the limitation; safe regeneration uses force=true+diarize=true

• correlate speech with logs: each segment's t_wall lines up with your log timestamps

• no speaker fields on a meeting job → re-run process_media with diarize=true (adds them without re-transcribing)

• attribution_precision="segment" → force=true+diarize=true is required for exact word boundaries

• pending context points to old evidence; stale labels accept null removal, never a new name

• anti-example: "where did they mention checkout?" → search(job_id, "checkout"), not full paging

• anti-example: screenshots around a remark → get_moment(job_id, start_ms, end_ms)

Fetch stored keyframe images (JPEG, <=1568px wide) as MCP image content: the frames nearest to at_ms, OR unique frames across [start_ms, end_ms] evenly thinned to max_frames. Serves unique frames by default (near-duplicates from static scenes are filtered); hard cap 6 images per call. When NOT to use: exact instants between keyframes or native-resolution detail (use extract_frame), or finding on-screen text (use search — OCR text is indexed). Examples:

• get_frames(job_id="...", at_ms=83500) — what was on screen when the remark at 1:23.5 was spoken

• get_frames(job_id="...", at_ms=83500, max_frames=2) — tighter context, fewer tokens

• get_frames(job_id="...", start_ms=0, end_ms=600000, max_frames=6) — overview strip of the first 10 min

• get_frames(job_id="...", start_ms=290000, end_ms=310000, include_duplicates=true) — every capture near 5:00

• transcript hit at t_ms=421500 → get_frames(job_id, at_ms=421500) for the visual evidence

• walking a demo scene by scene → one ranged call per scene beats one giant range

• frame files are named by video-ms (t00083500.jpg ↔ t_ms 83500) — stable refs for findings

• valid_from_ms/valid_to_ms on each frame = when the screen looked like this — check the span covers your moment

• keep max_frames at 2-4 unless you are truly comparing scenes; images are token-expensive

• every frame entry carries "path" (absolute) — save/copy the image elsewhere with your own file tools

• audio-only job → this tool errors by design; use get_transcript / get_moment instead

• anti-example: need EXACTLY 12:34.500 between two keyframes → extract_frame(job_id, at_ms=754500)

• anti-example: "find the screen with the red error banner" → search(job_id, "error") first, then jump

The "one remark" evidence bundle: transcript slice + up to 3 unique frames + their OCR text + the wall-clock range for [start_ms, end_ms], in a single call. This is the workhorse for triage: one call per finding gives you the quote, the screenshot, and the on-screen text. When NOT to use: broad exploration (get_transcript / get_frames) or keyword lookup (search). Examples:

• get_moment(job_id="...", start_ms=83000, end_ms=97000) — full evidence for the remark at 1:23-1:37

• segment seq 12 spans t0_ms=83210, t1_ms=96800 → get_moment(job_id, 83210, 96800)

• pad ±2000 ms around the spoken range — narrators react to things already on screen

• triage loop: for each candidate finding, exactly one get_moment call → quote + frame + OCR

• user: "what was I showing when I said 'this button is broken'?" → search first, then get_moment at the hit

• opening context of a meeting: get_moment(job_id, 0, 15000)

• response includes the t_wall range when known → quote it in bug reports for log correlation

• diarized job → speakers_in_range + speaker on each segment: who is talking in this window, at a glance

• frame entries carry "path" (absolute) — copy the screenshot elsewhere with your own file tools

• "was X on screen at t?" → yes iff some frame's valid_from_ms <= t < valid_to_ms — no extra calls needed

• audio-only job → returns the transcript slice plus a no-frames note (that is expected)

• anti-example: whole-video summary → get_transcript(format="text"), not a chain of get_moments

• anti-example: need more than 3 frames of a range → get_frames(start_ms=..., end_ms=..., max_frames=6)

• keep ranges under ~30 s; a 5-min "moment" dilutes the bundle and wastes tokens

Case-insensitive word search across BOTH transcript segments and frame OCR text. The default match_mode="all_words" requires EVERY query word as a substring; match_mode="any_word" requires at least one (ё and е are interchangeable). Hits carry source (transcript|ocr), t_ms, t_wall when known, the matched text, and the nearest frame position — everything needed to jump straight to evidence. Optional speaker accepts a raw label ("S2") or saved name and narrows to that voice's transcript hits. Duplicate saved names search all matching labels honestly. No embeddings. When NOT to use: fuzzy/semantic questions ("anything about performance?") — page get_transcript and read; regex is not supported. Examples:

• search(job_id="...", query="login") — every spoken or on-screen mention of login

• user: "what did I say about the login button?" → search(job_id, "login button") → get_moment at hits

• search(job_id, "TypeError") — on-screen stack traces and error text are OCR-indexed; great for bug repros

• search(job_id, "€49") — prices, IDs, and literals on screen are findable via OCR

• take hit.t_wall and grep your server logs ±30 s around it to pair remark ↔ log line

• broad lexical recall: search(job_id, "timeout latency", match_mode="any_word")

• multi-word = ALL words as substrings, any order: "first phase" hits "the first real phase"

• stems beat inflected phrases: "кнопк отправк" finds «Кнопка отправки» and «кнопку отправки»

• every hit has nearest_frame_ms → get_frames(job_id, at_ms=) shows the moment

• diarized job: transcript hits carry "speaker" — "who mentioned the deadline?" is answered by the hit itself

• search(job_id, "deadline", speaker="S2") — only S2's mentions; OCR hits are excluded (screens have no voice)

• search(job_id, "deadline", speaker="Vera") — saved names are matched case-insensitively

• audio-only job → transcript hits only (there is no OCR index)

• anti-example: "summarize the pricing discussion" → get_transcript(format="text") and read it

• anti-example: "everything S2 said" → get_transcript and collect speaker=="S2" — search always needs a query

Persist VERIFIED human-readable names for anonymous S1/S2/… labels on one diarized job. labels is a patch: a 1-100 character name saves it; null or blank removes it. Optional evidence (max 500 characters per label) records why the mapping is trusted. Raw labels remain canonical in JSON; names appear separately and in text/SRT display. The write is atomic, locked, local, and idempotent. OCR name_candidates are raw hints only and are never saved automatically. A relabelled job can carry pending-review names with source-roster anchors. For a label still in the roster, an explicit patch confirms, replaces, or removes only that entry. For a stale pending label, only an explicit null removes its name, evidence, and context. When NOT to use: before diarization, or when a name is only a guess without human/screen evidence. Examples:

• label_speakers(job_id="...", labels={"S1":"Vera"}) — save one verified mapping

• label_speakers(job_id="...", labels={"S1":"Vera","S2":"Tom"}) — patch several labels

• save evidence: label_speakers(job_id="...", labels={"S1":"Vera"}, evidence={"S1":"intro at 1200ms"})

• screen name plate confirms S2 → save the name and cite its frame timestamp in evidence

• label_speakers(job_id="...", labels={"S2":null}) — remove S2's name and evidence together

• label_speakers(job_id="...", labels={"S2":" "}) — blank also removes the mapping

• named S1, new proof → label_speakers(job_id="...", labels={}, evidence={"S1":"title card at 0ms"})

• two Alexanders may map to S2 and S5 — duplicate names are allowed and search covers both

• response roster carries speaker_name beside label; raw S labels are never replaced

• name_candidates may be UI text or another person's name → inspect frames before deciding

• unknown label or a name over 100 characters → error lists the valid roster labels

• fresh session: get_transcript returns saved names; do not infer the mapping again

• stale pending S3 → labels={"S3":null}; never assign a name while S3 is outside the roster

• anti-example: uncertain identity → keep S anonymous until evidence verifies the name

Re-extract ONE frame at an exact timestamp from the ORIGINAL source video at native resolution, with an optional crop {x, y, w, h} in source pixels. Use when the stored keyframes miss the instant (they capture scene changes + a 1 fps floor) or when you need full-resolution detail. Slower than get_frames — it decodes the source file, which must still exist at its recorded path (URL jobs keep their downloaded source inside the job, no network). When NOT to use: normal browsing — get_frames serves stored keyframes instantly without touching the source. Examples:

• keyframes sit at 12:31 and 12:38 but the flash happened at 12:34.5 → extract_frame(job_id, at_ms=754500)

• extract_frame(job_id="...", at_ms=754500, crop={"x":800,"y":40,"w":400,"h":120}) — zoom into the toast text

• tiny UI text unreadable in the 1568px keyframe → extract_frame at the same ms for native resolution

• verify a one-frame glitch: extract_frame at 12300, 12400, 12500 and compare

• OCR missed small text → extract_frame with a tight crop, then read the returned image

• crop coordinates are SOURCE pixels (a Retina screen recording may be 2940x1912) — not keyframe scale

• response JSON carries "path" (absolute) — "save this screenshot next to my docs" = copy from path yourself

• source file moved or deleted → clear error; stored keyframes via get_frames still work

• audio-only job → always errors: there is no video stream to decode

• anti-example: "show me around 5:00" → get_frames(job_id, at_ms=300000); extract_frame is for exact instants

• anti-example: scanning a range frame by frame → get_frames(start_ms, end_ms) first, refine once after

List processed recordings, newest first: job_id, source filename, duration, created, wall-clock start, segment/frame counts. The store is content-addressed — the same file maps to the same job even after renames or moves, and jobs persist across sessions and machines restarts. When NOT to use: as a health check or before every call — job_ids are stable, remember them. Examples:

• user: "triage the recording I processed this morning" → list_jobs() → pick by filename + created

• user names neither job_id nor path → list_jobs() first; only ask if still ambiguous

• resume yesterday's analysis in a fresh conversation → list_jobs() → reuse its job_id directly

• file was renamed after processing → match by duration/created; the content hash ignores names

• wall_clock.start answers "WHEN was this session?" — pick the job from "yesterday around 15:00"

• after CLI batch pre-processing (talkthrough-mcp process big.mov) the job shows up here — query it

• two jobs with the same filename → the newer created one is usually the re-recording

• diarized jobs show "speakers": N — "the 4-person meeting from Tuesday" is findable at a glance

• URL jobs carry "origin" (provider, provider_id, title) — "the YouTube video from yesterday" is findable

• empty list → nothing processed on this machine yet; ask the user for a file path or a public URL

• job disappeared → likely talkthrough-mcp gc cleaned it; re-run process_media on the file (same id)

• anti-example: checking whether a NEW file is processed → just call process_media, it is idempotent+instant