analyze_video
Analyze any video by extracting frame images, audio transcript, and metadata from a local file or URL.
Instructions
Analyze a video into frames + transcript + metadata.
Provide exactly ONE source:
file_path: local path to a video file (already on disk).
url: direct/streaming URL (yt-dlp for known sites, HTTP otherwise).
frame_interval: seconds between sampled frames (default 1.0 = one per second). Denser sampling: 0.5 / 0.25 / 0.1; sparser: 2 / 5; or any custom value. The total is capped by the frame budget so long/dense videos can't flood context. Ignored by the native Gemini backend (it ingests the whole video).
The backend (local whisper.cpp / OpenAI / Groq / native Gemini) is chosen automatically from configured keys and named in the result metadata. Results are cached per (file-hash, backend, frame_interval); pass force_refresh=true to recompute.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | ||
| file_path | No | ||
| force_refresh | No | ||
| frame_interval | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |