Media
mediaIndex, search, and retrieve video, audio, and image assets with semantic understanding for AI agents to edit and analyze.
Instructions
Media operations for video/audio/image assets. Enables AI agents to index, search, and retrieve media with semantic understanding - solving the "LLM as video editor has no context" problem for tools like Remotion.
Actions:
index: Index a local media file or external URL. Triggers ML processing (Whisper transcription, CLIP embeddings, keyframe extraction).
status: Check indexing progress for a content_id. Returns transcript_available, keyframe_count, duration.
search: Semantic search across indexed media. Returns timestamps, transcript excerpts, keyframe URLs.
get_clip: Get clip details for a time range. Supports output_format: remotion (frame-based props), ffmpeg (timecodes), raw.
list: List indexed media assets.
delete: Remove a media asset from the index.
Example workflow:
media(action="index", file_path="/path/to/video.mp4") → get content_id
media(action="status", content_id="...") → wait for indexed
media(action="search", query="where John explains authentication") → get timestamps
media(action="get_clip", content_id="...", start="1:34", end="2:15", output_format="remotion") → get Remotion props
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | Action to perform | |
| workspace_id | No | Workspace ID (UUID). | |
| project_id | No | Project ID (UUID). | |
| target_project | No | Target child project by folder name or project name | |
| file_path | No | Local path to media file for indexing | |
| external_url | No | External URL to media file for indexing | |
| content_type | No | Type of media content. Use video, audio, image, or document; friendly aliases like photos/images and docs/PDFs/slides are accepted. | |
| content_id | No | Content ID from index operation | |
| query | No | Semantic search query for media content | |
| content_types | No | Filter search/list to content types: video, audio, image, document. Friendly aliases are accepted. | |
| start | No | Start time for clip. Formats: "1:34", "94s", or seconds as string | |
| end | No | End time for clip. Formats: "2:15", "135s", or seconds as string | |
| output_format | No | Output format: remotion (frame-based props for Video component), ffmpeg (timecodes), raw (seconds) | |
| fps | No | Frames per second for remotion format (default: 30) | |
| tags | No | Tags to associate with media | |
| limit | No | Maximum results to return |