analyze_video
Extract transcript, key frames, OCR text, and annotated timeline from video URLs. Supports Loom, YouTube, direct files, and more.
Instructions
Analyze a video URL to extract transcript, key frames, metadata, comments, OCR text, and annotated timeline.
Returns structured data about the video content:
Transcript with timestamps and speakers
Key frames extracted via scene-change detection (deduplicated, as images). For static clips with no scene cuts (e.g. talking-head Reels/Stories where only on-screen text changes) it automatically falls back to uniform temporal sampling.
OCR text extracted from frames (code, error messages, UI text, prices/dates/CTAs visible on screen)
Annotated timeline merging transcript + frames + OCR into a unified chronological view
Metadata (title, duration, platform)
Comments from viewers (if available)
Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI).
Detail levels:
"brief": metadata + truncated transcript only (fast, no video download)
"standard": full analysis with scene-change frames (default)
"detailed": dense sampling (1 frame/sec), more frames, full OCR
Use options.fields to request only specific data (e.g., ["metadata", "transcript"]). Use options.forceRefresh to bypass the cache. Use options.model / options.language / options.initialPrompt to override Whisper transcription per call (e.g. a heavier model + a domain glossary for hard audio) without restarting the server.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file | |
| options | No | Analysis options |