segment_video
Detect objects in a video segment using text prompts. Get per-frame bounding boxes and confidence scores for any uploaded video without needing feature indexing.
Instructions
Detect objects in a video segment using text prompts. Describe what to look for and get per-frame detections with bounding boxes and confidence scores.
Prompt tips:
Use broad, visual categories: 'animal', 'vehicle', 'person', 'text on screen'
Specific labels ('rabbit', 'Toyota') are less reliable — the detector matches visual patterns, not semantic concepts
Best for confirming whether a category of object appears in a time window, not for precise identification
How to pick a time range:
Use search_videos to find WHEN something appears, then pass those timestamps here
Use get_scenes to scan systematically — call segment_video once per scene (scenes typically fit in the 15s window)
Or pass any range you already know
Maximum range is 15 seconds per call; for longer spans, make multiple calls with consecutive windows.
Does NOT require any feature indexing — works on any uploaded video.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| end | No | ||
| start | Yes | ||
| prompts | Yes | ||
| video_id | Yes | ||
| rationale | No | ||
| threshold | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |