Retrieves a public Instagram Reel, TikTok video, or YouTube video/Short and returns an accurate, timestamped transcript of its spoken audio — and, when visual analysis is enabled, meaningful visual information the video shows.

WHAT IT DOES

• Downloads the video/audio behind a public URL and transcribes the spoken speech using an automatic speech recognition model.

• Returns structured, timestamped segments plus a combined plain-text transcript, along with the detected spoken language and total duration.

• When visual analysis is enabled server-side, also returns "visual" — a short list of meaningful visual observations (on-screen text/slides, charts, UI, important scene context) with their own timestamps. This is never a caption for every frame; it only includes what a reader would actually need to understand the video without watching it. Absent or empty "visual" does not mean nothing was shown — it means nothing met that bar (or visual analysis wasn't enabled).

WHAT IT DOES NOT DO

• It does NOT work on private, login-gated, deleted, or otherwise inaccessible content, and it never attempts to bypass login, CAPTCHAs, or other access controls — such content returns a typed error instead.

• It does NOT currently support any platform other than Instagram Reels, TikTok videos, and YouTube videos/Shorts (no X/Twitter, etc.), and no Instagram content types other than Reels (no photo posts, carousels, Stories, or IGTV).

• Visual analysis, when it runs, does not describe trivial visual activity (a person moving, blinking, camera motion) — only information that materially helps understand the content.

SUPPORTED INPUT

• Public Instagram Reel, TikTok video, or YouTube video/Short URLs, e.g.: https://www.instagram.com/reel/ABC123xyz/ https://www.tiktok.com/@user/video/123... https://www.youtube.com/shorts/ABC123xyz

• Any other URL shape or domain returns an UNSUPPORTED_SOURCE error.

LIMITATIONS

• Videos are subject to a configured maximum duration and file size; longer/larger videos are rejected rather than partially processed.

• Background music, overlapping speakers, heavy accents, or very noisy audio can reduce accuracy; when the model itself is uncertain, "low_confidence" is set to true instead of guessing at unclear speech.

• This is a best-effort automatic transcript, not a human-verified one. Visual observations, when present, are similarly best-effort — on-screen text is preserved exactly as read, never "corrected", and a low "confidence" means treat it as uncertain rather than fact.

Use this tool when you need the words spoken in a public video from one of the supported platforms, and optionally what it visually showed.

Watches a public video end to end and returns a full multimodal understanding of it: the spoken transcript AND the meaningful visual information it shows (on-screen text, charts, products, UI, scene context) — always with visual analysis requested, unlike transcribe_video where it's opt-in.

Use this as the entry point for any question that isn't purely "what was said" — anything about what was shown, displayed, or visible. The returned "video_id" can then be passed to search_video (find every mention of a topic across both speech and on-screen content), find_moment (get the single best timestamped piece of evidence for a question), and get_video_timeline (the full chronological merge of both modalities) — without re-submitting the URL or re-processing the video.

WHAT IT DOES NOT DO

• Same platform/access limitations as transcribe_video: no private/login-gated/deleted content, no platforms beyond Instagram Reels, TikTok, and YouTube videos/Shorts.

• Does not itself answer free-form questions — call search_video or find_moment on the returned video_id for that.

Searches across every information channel of a previously-understood video — spoken transcript, on-screen text, and visual context — for a word, phrase, name, or topic. Returns every match with its own timestamp and which modality it came from ("speech" vs. an on-screen/visual type), ranked by relevance.

Requires a "video_id" from a prior understand_video or transcribe_video call. Use this instead of re-reading a whole transcript when you already know what you're looking for — e.g. "pricing", "$49", "the dashboard", "AI agents".

Returns the single best timestamped piece of evidence in a previously-understood video for a specific question — precise, verifiable, and anchored to one moment, unlike search_video which returns every match.

Requires a "video_id" from a prior understand_video or transcribe_video call. Use this when you need one authoritative answer with proof (e.g. "does the creator show pricing anywhere?", "what is he pointing at when he says 'this one'?") rather than a list of every mention. When the best match resolves to a specific visual entity (a pointing gesture, a "this"/"that" reference), the "interaction" field gives the structured target — absent when evidence was too weak to confidently resolve one, never a fabricated guess.

Returns the full chronological, merged timeline of a previously-understood video — every spoken segment interleaved with every visual observation, in time order. This is the underlying structure search_video and find_moment query; use it when you need the complete picture rather than a single answer (e.g. "walk through everything that happens in this video").

Requires a "video_id" from a prior understand_video or transcribe_video call.

Returns the Video Map: the semantic layer connecting what was SAID to what was VISIBLE and what the person DID — entities (people/products/objects that recur across the video), interactions (points_at, shows, holds, etc.), and references (what "this"/"that one" actually meant).

This is what resolves "speech → reference → visual target" — a transcript alone cannot tell you what "this one" refers to; this can, when the visual evidence was strong enough. Every interaction/reference has an "evidence_level" (observed/inferred/uncertain) and a "confidence" — absent "target_entity_id" means the evidence was too weak to confidently resolve a target, which is the CORRECT and expected answer in that case, not a failure.

Requires a "video_id" from a prior understand_video call with visual analysis enabled. Returns empty arrays (not an error) when the Video Map feature wasn't enabled or found nothing — use search_video/find_moment for the underlying speech/visual search either way.

Returns everything the Video Map knows about one specific entity (a person/product/object) from get_video_map — every interaction it was involved in and every linguistic reference resolved to it, in chronological order.

Use this after get_video_map or search_video/find_moment surfaces an entity id, to see its full history across the video (e.g. "every time the product was pointed at or referred to") rather than one isolated moment.

Looks up the full evidence behind a single Video Map fact by id — never invents detail beyond what was actually recorded (see get_video_map, get_video_entity). Use this to double-check a specific claim before treating it as authoritative, e.g. confirming a reference's evidence_level before repeating its resolved target as fact.