cuttingfloor
Server Details
Hosted video editing for AI agents: cut, caption, mix and export real video. Sign in with OAuth.
- Status
- Healthy
- Uptime
- 100.0% over the last 24 hours
- OAuth
- Works in Glama
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-11-25
- URL
- Repository
- Mvp2o-ai/cuttingfloor
- GitHub Stars
- 0
- ChatGPT
- –
- Claude
- –
TDQS
Scored across 73 tools
The tools are organized into clear namespaces and most actions map to distinct editing concepts (clip.move vs clip.slide vs clip.roll, audio.set_gain vs audio.set_master). There is some overlap among analysis tools: media.analyze, media.seek, media.listen, judge.ask and qa.check all measure or judge media, though their descriptions explain when to use each.
Every tool follows a predictable namespace.action pattern using snake_case, such as clip.set_transform, media.import, timeline.create and job.get. Even the create.* and audio.* families keep the same convention.
With 73 tools, the surface is far larger than an agent can comfortably navigate, even for a broad video-editing domain. The namespaces help, but many granular clip, audio, transition, and analysis operations could be consolidated or more contextually grouped.
The surface covers the full editing lifecycle: import, analysis, timeline construction, clip manipulation, audio, captions, titles, effects, transitions, QA, rendering, export, jobs, skills, and account management. There are no obvious dead ends for core workflows.
Available Tools
72 toolsadjustment.addAdd adjustment clipAIdempotentInspect
An empty clip on a video track whose effects (filter.add with its clip_id) change everything below it while it runs: a colour grade or blur over a stretch of the edit. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| at_frame | No | Default: after the last clip on the track | |
| track_id | Yes | A video track; the effects reach every track below it | |
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| duration_frames | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotent=true, destructive=false, and readOnly=false. The description adds useful behavior beyond those annotations: the clip is empty, its effects reach every track below it while it runs, and the call returns the edit result.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single compact sentence conveys what the clip is, how its effects work, an example use, and the return behavior. There is no redundant restatement, and the core definition is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With annotations covering safety and idempotency and an output schema covering the return, the description supplies the missing conceptual model: empty clip, effect scope, and the filter.add relationship. Nothing critical for correct invocation is missing, and return details are appropriately delegated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 80%, so most parameter meaning is already provided structurally. The description reinforces that track_id is a video track whose effects reach lower tracks and links the returned clip_id to filter.add, but it does not elaborate on duration_frames or at_frame beyond the schema, fitting the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description defines an adjustment clip as an empty video-track clip whose effects change everything below it while it runs, with colour grade/blur as examples. It clearly identifies the resource and effect mechanism, though it does not open with an explicit action verb, so the title carries part of the action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives a clear use case: a colour grade or blur over a stretch of the edit, and explains that effects are added via filter.add using the returned clip_id. It does not explicitly state when not to use it or compare it with alternatives, so it provides clear context without exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
audio.fadeFade audioBIdempotentInspect
Fade a clip's sound in from its start and out to its end. null removes a fade. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| in_frames | No | ||
| out_frames | No | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotentHint=true, destructiveHint=false and readOnlyHint=false, so safety behavior is largely covered. The description usefully adds that null values remove a fade and that the edit result is returned, but says nothing about whether an existing fade is overwritten or what the fade curve looks like.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short, front-loaded sentences with no wasted lead-in. The parenthetical pointer to server instructions is the only mildly expendable element.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists and return values are correctly deferred, so that is not a gap. However, for a 5-parameter mutation tool with only 40% schema coverage, the definition leaves units, defaults, and partial-fade behavior unexplained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 40%, so the description must compensate somewhat. It clarifies the null semantics for in_frames/out_frames and ties them to start/end of the clip, but never states that values are frame counts, their defaults, or what happens if only one side is given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Names a specific verb (fade) and resource (a clip's sound), and specifies the effect direction: in from start, out to end. It is distinguishable from siblings like audio.set_gain or transition.add, but it never explicitly contrasts itself with those neighbors.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The sentence 'null removes a fade' gives one clear when-not case, which is genuinely useful. There is no guidance on when to fade instead of using audio.set_gain or a transition, and no prerequisites are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
audio.set_gainSet gainAIdempotentInspect
Volume of an audio clip (or a video clip's linked audio) in dB; keyframes ride it up and down. The preview is a waveform of the clip; with preview inline it also carries the measured loudness (preview.loudness), a link preview does not. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| clip_id | Yes | An audio clip, or a video clip whose linked audio to change | |
| gain_db | Yes | dB, -60 to 24; 0 leaves it as it is. Keyframes make a ride: the first key holds backward to the clip start, so a ride that begins part-way down must also key its starting level at frame 0 or the frames before it are changed too. A key past the clip end is stored as given, with no clamp and no warning. Two keys on one frame are refused | |
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the safety profile (readOnly=false, destructive=false, idempotent=true, openWorld=false), so the bar is lower, and the description still adds context they don't carry: that inline previews include measured loudness while link previews do not, and that the call returns an edit result. It does not cover permissions or what an undo looks like, keeping it short of a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three compact sentences, front-loaded with the core effect (dB on a clip) before preview and return details. Slightly fragmentary opening but nothing is padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so the description only needs to gesture at the return value, which it does by pointing to the edit result and server instructions. For a 4-parameter mutation with full schema coverage and annotations, the remaining gap is minor.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds meaning the schema lacks: that keyframes make the gain 'ride' over time, and that inline previews surface preview.loudness that link previews do not. That is genuine value beyond the field descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the specific resource and property (gain in dB on an audio clip, or a video clip's linked audio) and distinguishes the two clip kinds it accepts, which separates it from audio.set_master and audio.set_pan. It is phrased as a noun fragment rather than a verb, but the operation is unambiguous to an agent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the scope statement (clip-level audio, not master or pan), but the description never says when to prefer this over audio.set_master, audio.set_pan, or audio.fade, all of which live in the same namespace. No exclusions or prerequisites are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
audio.set_masterSet masterAIdempotentInspect
Gain (dB) and limiter ceiling on the whole mix. The ceiling caps sample peaks (dBFS); the true peak between samples can land above it, most on dense, bright sound, and qa.check measures it (dBTP). Audio edits with preview inline carry the measured loudness (preview.loudness); a link preview does not. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| gain_db | No | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| limit_db | No | Limiter ceiling on sample peaks, dBFS | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (idempotent, non-destructive, non-read-only), the description discloses real behavioral traits: the ceiling caps sample peaks in dBFS, true inter-sample peaks can exceed it on dense bright material, and qa.check measures that in dBTP. It also explains that inline previews carry measured loudness while link previews do not — useful context that is not derivable from structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The purpose is front-loaded in the first clause and every subsequent sentence carries distinct information (peak caveat, measurement pointer, preview caveat, return pointer). Some phrasing is terse to the point of being cryptic ("Audio edits with preview inline carry the measured loudness").
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so deferring return values to the server instructions is acceptable, and the peak/loudness caveats cover the non-obvious failure modes of mastering. Only the relationship to sibling gain tools is left unaddressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 75%, so the schema already documents limit_db and the preview enum. The description still adds meaning: it clarifies gain is in dB and that the ceiling applies to sample peaks rather than true peaks, and links preview mode to loudness reporting, which the enum description does not cover.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
"Gain (dB) and limiter ceiling on the whole mix" gives a specific resource and scope (the master mix), which implicitly distinguishes it from the per-clip audio.set_gain sibling. It never names that sibling, so the differentiation is inferred rather than stated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Scope ("whole mix") implies when to use this over clip-level gain tools, and the qa.check mention hints at a verification workflow, but there is no explicit when-to-use / when-not-to-use statement or named alternative. The agent must infer routing from the word "master".
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
audio.set_panSet panBIdempotentInspect
Place a clip's sound left (-1) to right (1). Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| pan | Yes | -1 left, 0 centre, 1 right; null resets | |
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare it is a non-read-only, idempotent, non-destructive edit, so the safety profile is covered. The description adds only that it returns the edit result and defers to server instructions, without saying what the edit result contains or any constraints (e.g., clip must have audio). Adequate but thin for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core action and range. The trailing pointer to 'server instructions' is slightly deflective but does not bloat the text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained, and annotations cover mutation safety. However, nothing explains the preview parameter's effect or whether panning applies only to clips with audio, leaving gaps for a 4-parameter edit tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 75% and the pan range is already documented in the schema; the description merely restates -1..1. The preview parameter and timeline_id handle semantics are not touched on in the description, so it adds little beyond structured fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a concrete verb and resource ('Place a clip's sound left (-1) to right (1)'), which unambiguously identifies stereo panning and separates it from gain/fade siblings. It does not explicitly contrast with audio.set_gain or audio.set_master, but the word 'pan' plus the -1..1 range does the differentiation implicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it (to position a clip's audio in the stereo field) but gives no explicit conditions, prerequisites, or alternatives such as audio.set_gain or audio.set_master. An agent must infer usage from the verb alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
caption.addAdd captionsAIdempotentInspect
Captions on a video track from segments (with optional word timings for highlights), from SRT/VTT text, or in_clip: segments timed in seconds of a clip's media, such as transcript word times, placed where they play. One clip per caption; returns their clip_ids. A style field you leave out takes the published default (video://catalog/text: bold white Inter at 6% of the frame height, near the bottom, with a black outline); timeline.get shows the full style. Lines wrap on their own at style max_width (a fraction of the frame width), balanced in length; \n forces a break. style max_lines caps the lines on screen at once: a longer caption shows a screenful at a time, each from its first word's time. A segment's spans style part of its text (words first_word to last_word, from 0: the timed words, or the text split at spaces), e.g. the key word in yellow. Burned in when exported. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| srt | No | SubRip text instead of segments | |
| vtt | No | WebVTT text instead of segments | |
| style | No | ||
| in_clip | No | Segments timed in seconds of a clip's media (e.g. transcript word times), placed where those moments play in the clip, through speed changes; what doesn't play is left out | |
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| segments | No | ||
| track_id | Yes | A video track for the captions, usually the top one | |
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| offset_frames | No | Shifts srt/vtt times |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well past the annotations: it discloses the exact default style when a style field is omitted, that captions are burned in at export, that lines wrap at max_width and balance in length, that max_lines shows a screenful at a time from each caption's first word, and that \n forces a break. It also notes clip_ids are returned. One soft caution: annotations claim idempotentHint=true while the text describes creating clips and returning new ids, which an agent may read as additive, but the description does not explicitly assert repeated calls duplicate, so this is not a hard contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the input modes, which is the right ordering, and every clause carries information. It is, however, one dense paragraph with heavy parenthetical nesting, so style behavior (defaults, wrapping, spans) blurs into the input-mode explanation rather than being separated.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, the 'Returns the edit result (see the server instructions)' pointer is sufficient and the burn-in-on-export note covers the non-obvious side effect. Coverage of the style subsystem is thorough for a 9-parameter nested schema; only preview and offset_frames go unmentioned, and both are documented in the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 78%, so the baseline would be 3, but the description adds real semantics: what in_clip timing means ('placed where those moments play ... through speed changes; what doesn't play is left out'), how spans index words ('the timed words, or the text split at spaces'), and how max_width/max_lines interact with wrapping. It leaves offset_frames and preview entirely to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific resource (captions) and the three input forms (segments with word timings, srt/vtt text, in_clip media-timed segments), plus the effect (one clip per caption, burned in on export). An agent can tell this is the captioning tool rather than a generic text operation, though it never names the nearest siblings (text.set, title.add) to draw the boundary explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The three input modes are each given a selecting condition: 'segments (with optional word timings for highlights)', 'from SRT/VTT text', and 'in_clip: segments timed in seconds of a clip's media, such as transcript word times'. It also points to timeline.get when the caller needs the full style. No explicit exclusions against text.set/title.add, which keeps it out of 5 territory.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.addAdd clipAIdempotentInspect
Place media on the timeline. A/V media lands as a linked pair on a video and an audio track. Default: the whole media, after the last clip, overwriting. A picture is scaled to fit the frame (a small logo fills it edge to edge); clip.set_transform {fit: "none"} keeps its own pixel size. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| av | No | linked: video and audio as a link group; video or audio: only that part | linked |
| mode | No | overwrite replaces what is underneath; insert pushes later clips right | overwrite |
| source | No | Source range in sequence frames, out exclusive. Default: the whole media | |
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| at_frame | No | Where the clip starts. Default: after the last clip | |
| media_id | No | Media handle from media.import or a create.* tool | |
| track_id | Yes | Track for the clip; a linked A/V clip puts its other half on link_track_id | |
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| other_tracks | No | What the other tracks do when this edit moves later clips on the clip's own tracks (linked clips always move with it). stay: they don't move, and the result warns which clips there now play early or late. sync: they get the same change, so titles, overlays and sound stay in step: time taken out here comes out of them too (like range.extract), time added here opens the same gap in them (clips crossing that point are split, as insert does). Locked tracks always stay | stay |
| after_clip_id | No | Start where this clip ends (instead of at_frame) | |
| link_track_id | No | Track for the other half of a linked clip. Default: the first track of that kind | |
| duration_frames | No | Length for stills (default 5 s); ignored when source is given | |
| nested_timeline_id | No | Instead of media: nest another timeline as one clip (a compound clip) |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare the safety profile (not read-only, idempotent, non-destructive); the description adds real behaviour: A/V media becomes a linked pair split across video and audio tracks, default placement is after the last clip, and the default mode overwrites. It also discloses the picture auto-fit side effect, which is otherwise invisible and surprising.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four short sentences, front-loaded with the action and the default behaviour, then the picture-scaling caveat and the return pointer. Each sentence contributes: placement, link semantics, defaults, fit exception, return value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 13 parameters, 100% schema coverage, annotations and an output schema, the description covers the parts an agent most needs before calling (placement default, link behaviour, overwrite default) and defers return-value detail to the server instructions. It does not mention the preview or other_tracks semantics, but those are fully handled by the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all 13 parameters (including other_tracks, mode, av, source, preview) are already documented in the schema. The description only echoes a few defaults (whole media source, placement after last clip) and adds no syntax or interaction detail beyond what the schema provides — baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource ("Place media on the timeline") and immediately specifies the two placement modes that distinguish it: linked A/V pair creation and default overwrite-after-last-clip. An agent can tell this apart from clip.replace, clip.move or clip.trim without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives concrete defaults (whole media, after the last clip, overwriting) that tell an agent what happens when it omits arguments, plus one explicit redirect: use clip.set_transform {fit: "none"} if a picture should keep its pixel size. No exclusion against siblings like clip.replace for swapping existing clips, so not a full 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.freezeFreeze frameAIdempotentInspect
Hold the frame at at_frame for duration_frames: the clip splits there, a freeze frame goes in, and later clips on its tracks move later. Freezing a clip's last frame holds it after the clip, with silence under the hold. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| at_frame | Yes | Timeline frame to hold | |
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| other_tracks | No | What the other tracks do when this edit moves later clips on the clip's own tracks (linked clips always move with it). stay: they don't move, and the result warns which clips there now play early or late. sync: they get the same change, so titles, overlays and sound stay in step: time taken out here comes out of them too (like range.extract), time added here opens the same gap in them (clips crossing that point are split, as insert does). Locked tracks always stay | stay |
| duration_frames | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare it is a non-destructive, idempotent write; the description adds real behavior the agent cannot get elsewhere: the split point, downstream ripple on the clip's own tracks, and the silence under a trailing hold. It is still silent on permissions, undo, and what's returned beyond 'the edit result', which is punted to server instructions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two front-loaded sentences with nearly zero waste; the primary effect comes first and the edge case second. Minor nits: the doubled 'at at_frame' phrasing and the deferral '(see the server instructions)' push context off-tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, return values needn't be described, and the enumerated other_tracks/preview parameters are fully documented in the schema. The description covers the operation, its ripple effects, and the trailing-frame special case — everything needed to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67%, and the description supplies the key semantics the schema lacks: that at_frame is the held frame and duration_frames is how long, plus the edge-case meaning when at_frame is the clip's last frame. other_tracks, preview and timeline_id are left entirely to the (well-written) schema fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('hold the frame at at_frame for duration_frames') and spells out the composite effect: the clip splits, a freeze frame is inserted, and later clips ripple. That composite signature separates it from clip.split and create.still without the agent having to infer anything.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a concrete usage context and an important special case ('freezing a clip's last frame holds it after the clip, with silence under the hold'), which tells the agent when the result differs from the common case. It does not name sibling alternatives such as clip.split for pure cuts, so it stops short of explicit when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.linkLink clipsCIdempotentInspect
Make clips move, trim and split together. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| clip_ids | Yes | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (idempotentHint=true, destructiveHint=false, openWorldHint=false), and the description adds the real behavioral consequence that linked clips move/trim/split as a unit. However, it is silent on what happens when clips are already linked, what happens on partial failure, or why idempotency holds, and it punts the return details to "the server instructions."
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with the behavioral effect front-loaded, which is good. The second sentence is largely a deflection to external server instructions rather than information, so it only partly earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutating edit operation with three parameters and an output schema, the description is thin: it does not explain the clip_ids semantics, the effect on already-linked clips, or the new timeline handle returned by the edit. Deferring to "server instructions" leaves the definition incomplete on its own.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67%: timeline_id and preview are documented in-schema, but clip_ids — the core parameter — has only a minItems=2 constraint and no explanation in either place. The description says nothing about any parameter, so it fails to compensate for that gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
"Make clips move, trim and split together" describes the effect of linking, which is more informative than the title alone, but it never states the action verb ("link") or names the sibling it inverts, clip.unlink. An agent can infer the purpose but must reason to it rather than read it directly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit when-to-use or when-not-to-use guidance and no mention of the obvious alternative, clip.unlink, which appears in the sibling list. Usage is only implied by the description of what linking does.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.moveMove clipBIdempotentInspect
Move a clip (and its linked clips) to a frame and optionally another track. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | overwrite replaces what is underneath; insert pushes later clips right | overwrite |
| ripple | No | Close the gap left behind | |
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| to_frame | Yes | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| to_track_id | No | ||
| other_tracks | No | What the other tracks do when this edit moves later clips on the clip's own tracks (linked clips always move with it). stay: they don't move, and the result warns which clips there now play early or late. sync: they get the same change, so titles, overlays and sound stay in step: time taken out here comes out of them too (like range.extract), time added here opens the same gap in them (clips crossing that point are split, as insert does). Locked tracks always stay | stay |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, idempotentHint=true, and destructiveHint=false, so the safety profile is partly covered. The description adds useful side-effect context that linked clips move with the clip and that the edit result is returned, but it does not elaborate on overwrite/ripple behavior beyond what the schema provides.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core action and destination, and ends with a useful pointer to the server instructions for return details. There is no wasted wording.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so the description need not explain return values in depth, and the annotations carry key safety hints. However, for an 8-parameter timeline mutation with several behavioral options, the description leaves usage against sibling tools and the semantics of required parameters largely to the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 63%, so the schema documents most optional parameters, including mode, ripple, preview, and other_tracks in detail. The description only adds destination semantics for the frame and optional track; it does not explain clip_id or add meaning beyond the schema for the required timeline handle.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: move a clip to a frame, optionally to another track, with linked clips moving too. It is clear what the tool does, but it does not explicitly distinguish this from sibling tools like clip.slide, clip.slip, or clip.roll.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no explicit guidance on when to use this tool versus alternatives such as clip.slide, clip.slip, or range.extract. It implies repositioning, but leaves the agent to infer when a move is appropriate and when another timeline-editing tool should be used instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.removeRemove clipBIdempotentInspect
Remove a clip and the clips linked to it; ripple closes the gap. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| ripple | No | Close the gap by pulling later clips left | |
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| other_tracks | No | What the other tracks do when this edit moves later clips on the clip's own tracks (linked clips always move with it). stay: they don't move, and the result warns which clips there now play early or late. sync: they get the same change, so titles, overlays and sound stay in step: time taken out here comes out of them too (like range.extract), time added here opens the same gap in them (clips crossing that point are split, as insert does). Locked tracks always stay | stay |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description says 'Remove a clip', which is a destructive operation, while annotations declare destructiveHint: false. This direct contradiction misleads the agent about the tool's safety profile. Although annotations provide other hints (readOnlyHint: false, idempotentHint: true), the contradiction is severe.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with zero waste: the first front-loads the action and scope, the second succinctly notes the return. No redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 5 parameters and an output schema, the description is minimal. The contradiction with destructiveHint undermines completeness, as an agent cannot reliably infer the tool's safety. The mention of 'Returns the edit result' is helpful but insufficient to compensate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 80%, so the schema already documents most parameters (ripple, preview, timeline_id, other_tracks). The description only hints at 'ripple' without adding syntax or format details. Baseline 3 is appropriate given the high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Remove') and resource ('clip'), and clarifies scope by including 'the clips linked to it'. It also mentions the ripple effect, making the action unambiguous and distinguishable from siblings like clip.unlink or clip.trim.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives (e.g., clip.unlink or range.extract). The description implies usage through behavior ('ripple closes the gap') but does not state conditions for selection or exclusion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.replaceReplace mediaAIdempotentInspect
Swap the media in a clip, keeping its place, length, speed and look. Starts at source_in (default 0). Its linked clips swap too (linked: false for this clip only); clip_ids lists every clip swapped. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| linked | No | Also swap its linked clips (its sound, or its picture) onto the new media, keeping their offset, where the new media has that kind; false swaps this clip only | |
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| media_id | Yes | Media handle from media.import or a create.* tool | |
| source_in | No | Source frame the clip starts at. Default 0 | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the safety profile (idempotent, non-destructive, closed-world), and the description adds real behavioral context beyond them: the source_in anchor, the fact that linked clips are also swapped by default, that linked:false scopes it to this clip only, and that clip_ids enumerates every clip affected. The remaining gap is that it defers the actual return payload to 'the server instructions' rather than describing shape.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences, front-loaded with the core operation and its invariants, with the linked-clip caveat placed after. The parenthetical '(see the server instructions)' is a small punt but compact rather than wasteful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an idempotent, non-destructive mutation with 6 params, 83% schema coverage and an output schema, the description covers the essentials: what changes, what is preserved, side effects on linked clips, and the source_in anchor. The pointer to server instructions for the return payload is acceptable given an output schema exists, though it costs a bit of self-sufficiency.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 83%, so the schema already documents most parameters well. The description largely restates schema content ('Starts at source_in (default 0)', 'linked: false for this clip only') rather than adding format or edge-case meaning; the one genuinely new semantic, clip_ids listing all swapped clips, is an output rather than an input.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The verb ('swap the media') and resource ('in a clip') are specific, and the description pins down exactly what is preserved — place, length, speed, look — which separates it from remove-then-add or clip.add workflows. It does not name a sibling tool to route away to, but the operation itself is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the description: use it when you want a different media asset in an existing slot rather than a new clip. There is no explicit 'use X instead when…' guidance versus clip.add, clip.trim or clip.remove, and no stated prerequisites (e.g. that media_id must be imported first — that only appears in the schema).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.rollRoll editAIdempotentInspect
Move the cut between a clip and the clip right after it: one gets longer, the other shorter, nothing else moves. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| clip_id | Yes | The clip before the cut | |
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| to_frame | Yes | New position of the cut | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish that this is a non-readonly, non-destructive, idempotent edit. The description adds useful behavioral context beyond that: exactly two adjacent clips are affected, one lengthens and the other shortens, and no other elements move. It does not discuss permissions or edge cases, but the added mutation footprint is meaningful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loads the core operation. The closing note about returning the edit result and referring to server instructions is slightly redundant given the output schema, but overall there is little waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a moderately complex edit tool with full schema coverage, rich annotations, and an output schema, the description gives enough core semantics to call it correctly. It could be stronger by stating prerequisites such as the need for a following clip on the same track, but the essential behavior is present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so clip_id, to_frame, preview, and timeline_id are already documented. The description clarifies the relationship between clip_id and to_frame by describing the cut between two adjacent clips, but adds no syntax, bounds, or format details beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: moving the cut between a clip and the clip immediately after it. The added detail that one clip gets longer, the other shorter, and nothing else moves distinguishes this roll edit from adjacent operations such as trim, slide, or move, even without naming those siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool by explaining that only the cut point shifts between two adjacent clips, while everything else stays fixed. However, it does not explicitly name alternatives like clip.trim or clip.slide, nor does it state exclusions or prerequisites for choosing this operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.set_animationSet animationAIdempotentInspect
Entrance, exit and loop presets layered over the clip transform (video://catalog/animations): e.g. in {name: "pop", duration_frames: 12}, loop {name: "float"}; a loop runs the whole clip, or only from_frame to to_frame of it (loop {name: "shake", from_frame: 30, to_frame: 45} for a jolt). null removes one. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| in | No | Entrance, from the clip start | |
| out | No | Exit, ending at the clip end | |
| loop | No | Repeats over the clip, or over from_frame to to_frame of it | |
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| placements | No | Where each clip the edit moved, cropped, masked or animated shows, sequence pixels |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare non-destructive, idempotent, closed-world. The description adds real behavioral content beyond them: null clears a preset, a loop can be confined to a frame range, and the call returns an edit result. It does not restate that each edit yields a new timeline_id (left to the schema).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One front-loaded sentence carries the purpose, three inline examples, the removal rule and the return note — no wasted lines. It is dense to the point of being a run-on with stacked parentheticals, which slightly hurts scannability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present and the return value deferred to server instructions, the description need only cover intent and slot semantics, which it does. Minor gaps: no mention of preview behavior or how in/out interact with loop on the same clip.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is already 83%, yet the description still earns credit by explaining loop scope semantics (whole clip vs from_frame–to_frame) with a concrete jolt example, and by naming null as the removal value for each slot. The only untouched parameter is clip_id, which is self-evident.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States exactly what is set — entrance, exit and loop animation presets — and anchors them as layered over the clip transform, which separates this from clip.set_transform and transition.add. The catalog reference (video://catalog/animations) makes the resource concrete.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Shows the calling patterns for each slot and, critically, states that passing null removes an animation — the only removal path given there is no clip.remove_animation sibling. It does not explicitly contrast with clip.set_transform or transition.add, so the routing guidance is implied rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.set_blendSet blend modeBIdempotentInspect
How a video clip mixes with the tracks below: normal, multiply, screen, overlay and more (video://catalog/blend_modes). Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| mode | Yes | ||
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly=false, idempotent=true, destructive=false, and openWorld=false, so the core safety profile is covered. The description adds that it affects how a clip mixes with lower tracks and returns an edit result, but it does not explain what changes are made to existing state, required permissions, or how the returned edit result should be interpreted beyond pointing to server instructions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two tight sentences, front-loading the blend-mode concept and then mentioning the return result. There is little wasted text, though the first sentence is a concept fragment rather than a direct action statement.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema and rich annotations, the description need not explain return values or safety in detail. It supplies the essential blend-mode context and a catalog reference, and the schema covers most parameter mechanics. The main omission is usage guidance, which is scored separately.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50%, with timeline_id and preview documented in the schema but clip_id and mode lacking schema descriptions. The description partially compensates for mode by listing example blend modes and linking to a catalog, but it says nothing about clip_id and does not clarify preview behavior beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the resource and setting: a video clip's blend mode that controls how it mixes with tracks below. It enumerates representative modes and points to a catalog, making the tool distinguishable from sibling clip setters. It is slightly oblique by describing the setting conceptually rather than explicitly saying 'sets the blend mode of clip_id', but the purpose is still clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit guidance on when to use this tool versus alternatives such as clip.set_transform, clip.set_enabled, or clip.set_mask. The blend-mode concept implies it is for visual compositing, but no conditions, prerequisites, or exclusions are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.set_enabledEnable/disable clipBIdempotentInspect
A disabled clip stays on the timeline but renders as a gap. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| clip_id | Yes | ||
| enabled | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=false and idempotentHint=true; the description usefully confirms the non-destructive effect (clip remains in place, renders as a gap) and notes that an edit result is returned. It stops short of saying the edit produces a new timeline_id, which the schema mentions instead.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with the key behavioral consequence front-loaded; the second sentence delegates return-format detail to the server instructions rather than bloating the definition. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need no explanation, but for a mutation tool with two undocumented parameters and no usage guidance the definition leaves real gaps an agent must resolve from the schema alone.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 50%: preview and timeline_id are documented in the schema, while clip_id and enabled are bare. The description only obliquely implies the meaning of enabled=false through 'a disabled clip'; it says nothing about what enabled=true does, what clip_id accepts, or format expectations, so the coverage gap is not compensated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The name and title carry the verb (enable/disable), and the description adds the crucial semantic distinction: a disabled clip is not removed but 'stays on the timeline but renders as a gap.' That implicitly separates it from clip.remove, though no sibling is named explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied — the 'stays on the timeline' framing hints you should pick this over clip.remove when you want to preserve the clip, but there is no explicit when-to-use, no prerequisite, and no mention of the alternative by name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.set_maskSet maskAIdempotentInspect
Select part of a video clip, then show, hide or fill it: rectangle (rounded), ellipse, linear, mirror, heart, star, polygon or matte (a media's alpha, e.g. a create.cutout), with feather and invert. Several shapes: add. A fill paints the selection from the picture around it and from earlier frames where that area showed, so something that never moves against its background leaves a smudge: nothing ever shows what is behind it. It is a job of its own (job.list shows its progress); a render that needs it names the job until it is done, and an export waits for it. Position, size and rotation animate like transform values. Coordinates are fractions of the clip's fitted frame: a sequence-shaped box with the picture fitted inside (bars where the shapes differ), before the clip's scale and position. A narrower picture fills k = (its aspect / the sequence's) of the box width, centred, so a picture-fraction box maps to x = 0.5 + (x - 0.5) * k, width * k (a wider picture: the same on y and height, with k = the sequence aspect / its). null removes the mask. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| mask | Yes | null removes the mask | |
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| placements | No | Where each clip the edit moved, cropped, masked or animated shows, sequence pixels |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations (which only say idempotent/non-destructive): it discloses that the operation runs as its own asynchronous job, that job.list tracks progress, that dependent renders name the job, and that exports wait on it. It also warns about the fill smudge limitation — genuinely non-obvious behavior an agent could not infer elsewhere.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the core action and shape list before the denser coordinate and job details. It is long and somewhat run-on in places, but nearly every sentence carries substantive information; trimming is possible but little is truly wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists, return values need not be explained (it still references 'the edit result'), and the async job, coordinate, and fill behaviors are all covered. A brief note on required permissions or default-mode implications would round it out, but the definition is complete enough to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 75%, so the schema already documents most parameters; the description adds real value by explaining the fitted-frame coordinate model and the k = aspect-ratio mapping formula, which clarifies how 0–1 coordinates behave on non-matching aspect ratios. It does not cover every param (e.g. preview, timeline_id), but the hard-to-grok spatial semantics are covered.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb and resource ('Select part of a video clip, then show, hide or fill it') and enumerates the exact shape vocabulary, which lets an agent distinguish it from sibling tools like clip.set_transform or create.matte without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clear about operational context: multiple shapes go through 'add', a fill vs show/hide distinction is explained, and 'null removes the mask' signals the removal path. It stops short of naming an alternative tool for related tasks (e.g. when to use create.matte vs a matte mask), so no explicit sibling routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.set_speedSet speedAIdempotentInspect
Speed up, slow down, ramp or reverse a clip and its linked clips. rate is constant; curve is a speed ramp over the clip. The clip keeps its source and its length changes; ripple (default) moves later clips. Pitch is kept unless preserve_pitch is false. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| rate | No | Constant speed: 2 = twice as fast | |
| curve | No | Speed ramp: rates at fractions of the clip as it plays after the change (first at 0, last at 1; 0.5 is halfway through the new length), linear between. timeline.get gives each point its clip frame and source frame | |
| ripple | No | Move later clips as the clip gets longer or shorter | |
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| reverse | No | ||
| interpolate | No | Video, when it shows fewer new frames a second than the sequence (slowed, or a lower frame rate; rate 1 smooths that): none repeats frames; blend crossfades neighbours; motion makes new frames from the movement (minterpolate, slow). Made once as a job of its own (job.list); a render that needs it names the job until it is done, and an export waits. Up to 6000 frames a clip | |
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| other_tracks | No | What the other tracks do when this edit moves later clips on the clip's own tracks (linked clips always move with it). stay: they don't move, and the result warns which clips there now play early or late. sync: they get the same change, so titles, overlays and sound stay in step: time taken out here comes out of them too (like range.extract), time added here opens the same gap in them (clips crossing that point are split, as insert does). Locked tracks always stay | stay |
| preserve_pitch | No | Keep voices at their pitch (default true) |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, destructiveHint=false, and idempotentHint=true, so the safety profile is covered. The description adds real behavior: the clip keeps its source, its length changes, ripple moves later clips, and pitch is preserved by default – none of which the annotations convey. It is silent on failure cases and how reverse interacts with rate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Five short sentences, front-loaded with the verbs then the mode distinction, then side effects, then the return note. Every sentence carries information an agent needs and there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists and the description correctly defers to server instructions for the returned edit result, so the return value need not be spelled out. It covers the mode choice and the ripple and pitch side effects, but leaves reverse semantics and the heavy interpolate job behavior entirely to the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 80%, so the schema already documents most parameters and the baseline is 3. The description still adds meaningful semantics by contrasting rate (constant) with curve (a ramp over the new length) and by stating that preserve_pitch defaults to true. That marginal lift over the schema earns a 4 rather than a 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States concrete verbs (speed up, slow down, ramp, reverse) against a named resource (a clip and its linked clips). That clearly separates it from clip.freeze, clip.trim, clip.slip, and clip.set_animation without the agent opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains the two mutually exclusive modes – constant rate versus a curve ramp – which is the main in-tool choice. It also tells the agent what ripple and other_tracks do by default, but never states when to prefer a sibling such as clip.freeze or clip.trim instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.set_transformSet transformAIdempotentInspect
Position, scale, rotate, fade, flip, crop or fit a video clip over the tracks below (picture-in-picture, overlays, punch-ins). fit none shows a graphic at its own pixel size, so scale 1 is its real size; the default contain scales it to touch the frame edges. Any number takes a constant or {keyframes: [{frame, value, ease}]} with frames from the clip start; a keyframe's ease shapes the move from it to the next one (sine_in_out, cubic_out, back_out, hold and more). Overlays that must follow a zoom or pan: put the clip and its overlays in a timeline of the same shape, nest it with clip.add {nested_timeline_id} and transform the nested clip, so everything moves together. Omitted fields keep their value; null resets one. To pin an overlay to a moving subject, use media.analyze track boxes as keyframes: for each box, frame = timeline frame of the box time (where the tracked clip plays that media second) minus this clip's start, x = box x + w/2 and y = box y + h/2 (add your offset). That holds when the tracked clip fills the frame as is; if it is scaled, moved or cropped, map the box through its transform first. Skip boxes with found false. The result's placements say where the clip now shows, in sequence pixels: its picture box, the part visible after crop and mask inside the frame, and the source pixels that show there. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | Centre x, 0 = left edge, 1 = right edge (default 0.5); -4 to 5 | |
| y | No | Centre y, 0 = top, 1 = bottom (default 0.5); -4 to 5 | |
| fit | No | ||
| crop | No | ||
| flip | No | ||
| scale | No | Size relative to the fit (default 1); 0 to 20 | |
| clip_id | Yes | ||
| opacity | No | Opacity (default 1); 0 to 1 | |
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| scale_x | No | Extra horizontal stretch (default 1); 0 to 20 | |
| scale_y | No | Extra vertical stretch (default 1); 0 to 20 | |
| rotation | No | Degrees clockwise (default 0); -36000 to 36000 | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| placements | No | Where each clip the edit moved, cropped, masked or animated shows, sequence pixels |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations, it discloses that omitted fields keep their value, null resets a field, keyframes are relative to clip start, and the result reports placements in sequence pixels including visible picture and source pixels. This is substantial behavioral context for a mutation tool with idempotentHint=true and destructiveHint=false.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The core transform purpose is front-loaded and the description is information-dense, but it is a long single paragraph mixing core parameters with advanced tracking and nested-timeline workflows. Those details are useful, though the structure could be easier to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex 13-parameter edit tool with an output schema and annotations, the description covers the important non-schema behavior: keyframes, fit semantics, reset behavior, tracking workflow, and placement return information. The output schema handles return details, so no critical agent-facing context is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds useful meaning for the Animatable parameters, keyframe frame/ease semantics, and scale relative to fit. However, with 13 parameters and 69% schema description coverage, some parameters such as flip and clip_id are left to the schema, and several fit/crop explanations partly repeat schema text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb set and resource: position, scale, rotate, fade, flip, crop or fit a video clip over lower tracks. That makes the tool's transform scope clear and distinguishes it from adjacent clip tools such as clip.move, clip.trim, clip.set_mask, and clip.set_blend.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit usage scenarios: fit none for graphics at native pixel size, contain as default, nested timelines for overlays that must follow zoom/pan, and media.analyze track boxes for pinning overlays to moving subjects. It also explains reset behavior and names complementary tools/approaches rather than leaving the agent to infer them.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.slideSlide editBIdempotentInspect
Move a clip between its neighbours: the clip before it and the clip after it change length so nothing else moves. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| to_frame | Yes | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already say readOnly=false, idempotent=true, destructive=false, so the safety profile is covered. The description adds the key side effect an agent would not guess: the preceding and following clips are shortened/lengthened. It does not address edge cases (first/last clip with no neighbour, insufficient adjacent media).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences, front-loaded with the operation and immediately followed by the mechanism. No filler. Only mild weakness is the somewhat loose 'between its neighbours' phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return values need no explanation, and the pointer to server instructions helps. But for a 4-parameter edit tool with two undocumented parameters, the description leaves out boundary behaviour and the to_frame target semantics, which an agent needs to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 50%; clip_id and to_frame have no schema descriptions at all, and the description does not say what to_frame means (target frame of the clip's new start?) or how frames map to the slide. It also omits the fact that timeline_id becomes stale after the edit, even though the schema hints at it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a concrete verb+resource ('move a clip') and explains the distinctive slide semantics: neighbours change length so nothing else moves. This implicitly separates it from clip.move (which shifts others) and clip.roll (which moves a boundary without moving the clip), but no sibling is named.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'nothing else moves' phrase implies the right use case, but the description never says explicitly when to pick this over clip.roll, clip.slip, or clip.move, all of which sit adjacent in a large sibling set and sound very similar.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.slipSlip editAIdempotentInspect
Show earlier or later source in a clip without moving or resizing it. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| by_frames | Yes | Positive shows later source | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already disclose that this is a mutating, non-read-only, idempotent, non-destructive edit. The description adds important behavioral context not covered there: the edit changes source exposure while preserving the clip's timeline position and size, which is central to understanding the operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the core effect, and contains no redundant or filler language. The reference to server instructions for return details is compact and appropriately defers rather than bloating the definition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With rich annotations, an output schema, and mostly described parameters, the description supplies enough core semantics to call the tool correctly. It does not explain the new timeline handle behavior, but that is already stated in the timeline_id schema description and output schema context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 75%, so most parameters are already documented in the schema, including the sign convention for by_frames and the preview enum. The description reinforces the earlier/later source direction but does not add meaningful semantics for clip_id, timeline_id, or the preview parameter beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific editing effect: showing earlier or later source in a clip without moving or resizing it. This clearly conveys the slip-edit concept and distinguishes it from clip.move and clip.trim, though it does not explicitly differentiate it from siblings such as clip.slide or clip.roll.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'without moving or resizing it' gives a useful condition for when this tool is appropriate. However, it does not name alternatives like clip.slide, clip.roll, or clip.trim, so an agent must infer when to choose this tool over those siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.splitSplit clipAIdempotentInspect
Cut a clip (and its linked clips) in two at a frame. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| at_frame | Yes | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds one genuinely useful behavioral fact beyond annotations: linked clips are split as well, which an agent could not infer from the schema. However, the return behavior is deferred to 'see the server instructions', and it does not clarify what happens at clip boundaries or on a repeated identical split, where idempotentHint=true is questionable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the action and the side effect. No filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained, but the description leans on 'see the server instructions' instead of stating the edit-result shape, and says nothing about handle invalidation or failure modes for a mutating tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50%: preview and timeline_id are documented inline, but clip_id and at_frame are not. The description only restates that a frame is the split point without saying which side the frame lands on, and adds nothing about clip_id or the timeline handle.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: cut a clip in two at a frame, and discloses that linked clips are cut too. It is distinguishable from siblings like clip.trim or clip.roll in spirit, but never names them, so differentiation is left partly to inference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by 'cut a clip in two at a frame', but there is no explicit when-to-use versus alternatives such as clip.trim, clip.slide, or clip.roll, and no preconditions (e.g. frame must lie inside the clip's range).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.trimTrim clipBIdempotentInspect
Move the start or end edge of a clip. side: video or audio trims one half of a link group (J/L cuts). Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| edge | Yes | ||
| side | No | Trim only one half of a link group (J/L cuts) | both |
| ripple | No | Shift later clips so nothing overlaps and no gap opens | |
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| to_frame | Yes | New timeline position of that edge | |
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| other_tracks | No | What the other tracks do when this edit moves later clips on the clip's own tracks (linked clips always move with it). stay: they don't move, and the result warns which clips there now play early or late. sync: they get the same change, so titles, overlays and sound stay in step: time taken out here comes out of them too (like range.extract), time added here opens the same gap in them (clips crossing that point are split, as insert does). Locked tracks always stay | stay |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, destructiveHint=false, and idempotentHint=true, so safety is covered. The description adds that one side can be trimmed for J/L cuts and that the return value is the edit result, but it does not elaborate on ripple, other_tracks, or preview behavior beyond what the schema says. With annotations present, this is an adequate but not rich behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core action and with no wasted words. The second sentence packs the J/L cut detail and return-value note efficiently, though it could be slightly clearer about what 'the edit result' entails.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 8 parameters (4 required), an output schema, and annotations, so some complexity is handled elsewhere. However, the description omits behavior of ripple, other_tracks, and preview, and gives no differentiation from closely related siblings, leaving meaningful gaps for an agent selecting and invoking this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 75%, so the schema already documents most parameters, including intricate options like other_tracks and ripple. The description only restates the edge movement and briefly mentions the side parameter for J/L cuts; it adds little beyond the schema's own field descriptions. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Move the start or end edge of a clip', which clearly identifies the trimming operation. However, it does not distinguish this from sibling tools like clip.roll, clip.slip, or clip.slide that also alter clip boundaries, leaving an agent to infer the difference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use or when-not-to-use guidance is provided, and no alternatives are named. The mention of 'side: video or audio trims one half of a link group (J/L cuts)' explains a parameter effect but does not help an agent choose between this tool and siblings such as clip.roll or range.extract.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clip.unlinkUnlink clipBIdempotentInspect
Break the link group a clip belongs to. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| clip_id | Yes | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare non-destructive, idempotent, closed-world mutation, so the safety profile is carried structurally. The description adds the key ambiguity it should resolve but doesn't: whether the whole group is dissolved or only this clip detaches, and what happens if the clip isn't linked. Pointing to 'the server instructions' for the return value is a deferral rather than disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with the action front-loaded and zero filler. The parenthetical pointer to server instructions is the only soft spot, trading brevity for a cross-reference the agent has to chase.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists and annotations cover safety, so the remaining burden is small but real: the scope of the break (single clip vs. entire group) and the no-op/error case for an unlinked clip are unaddressed. For a state-mutating timeline tool, those are the questions an agent most needs answered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
At 67% schema coverage, the schema documents preview (with enum meanings) and timeline_id (the new-handle semantics), while clip_id has no description at all. The description only implies clip_id via 'a clip belongs to' and says nothing about preview, so it neither compensates for the gap nor adds meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a concrete verb and object ('Break the link group a clip belongs to'), which an agent can map cleanly against sibling clip.link as its inverse. It stops short of naming the sibling explicitly, so it is clear but not fully differentiated in text.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use or when-not-to-use guidance. Nothing tells the agent how this differs in practice from clip.remove (which likely also detaches a clip) or that it should be used only when the clip is actually part of a link group.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create.cutoutCut out a subjectIdempotentInspect
A subject without its background, local: ProRes 4444 with alpha and the sound, or a PNG, to lay over a new background. subject any (BiRefNet) cuts out products, animals or objects; people (MODNet) is fast enough for long video. region: one item by its bounds, followed; also a clip.set_mask matte. Long-running: a task, or {job_id, media_id} for job.get.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| region | No | The item in this box (top-left x, y, size w, h; fractions) and/or at these points (keep false: not it) at at_seconds, outlined through the video (EdgeTAM, ~0.5 s a frame) | |
| smooth | No | Video: steadies the edge | |
| subject | No | people: MODNet, fast, for video of people; any: BiRefNet, any subject (products, animals, objects), about 1 s a frame and 3 GB of memory; auto: any for pictures, people for video | auto |
| media_id | Yes | Media handle from media.import or a create.* tool | |
| idempotency_key | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
create.lipsyncLip-sync a video to new speechIdempotentInspect
Make a person on camera say different words: their mouth is redrawn to match the speech you give (a re-recorded line, a create.voiceover, a word taken from elsewhere in the take). Runs at the user's lip-sync provider on their own key (sync. labs), and the provider bills them for each second sent. Only for the user's own footage, or people who agreed to it (cuttingfloor.ai/acceptable-use). To replace a line or swap a word, give range, the span of the video where it is said, and audio (with audio_range) holding just the new words. The result is that span alone, re-synced and carrying the new speech, as long as the speech with sync_mode remap; place it over the original span (clip.add on a track above, or clip.replace) and mute the old words. Send only the span: it costs less and finishes sooner. Lip sync is for when the face stays on screen while the words change. When the speaker is off screen at that moment, or a cutaway (b-roll, the screen being talked about, a reaction) can cover the splice, the cutaway hides the edit without it. Takes minutes. Long-running: a task, or {job_id, media_id} for job.get. The same recipe returns the same media. Provider limits (plan length, credits, file size) come back as limit_exceeded with the provider's error_code.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| audio | Yes | The speech they should say: audio media, or a video whose sound is used (e.g. create.voiceover, a re-recorded line) | |
| model | No | The provider's model: lipsync-2 (the cheapest, about $0.05 a second), lipsync-2-pro (finer teeth, beards and detail), sync-3 (close-ups, profiles, hands or mics in front of the mouth; about $0.13 a second). Billed to the user's own key | lipsync-2 |
| range | No | Only this span of the video, in the video's seconds (e.g. the sentence or word being replaced). The result is that span alone, re-synced; place it over the same span on the timeline. Default: the whole video | |
| video | Yes | The video of the person speaking, whose mouth is redrawn | |
| provider | No | sync: sync. labs (sync.so), with the user's SYNC_API_KEY | sync |
| sync_mode | No | When the picture and the speech differ in length. remap: the picture is sped up or slowed to the speech (result as long as the speech); cut_off: the longer one is cut (as long as the shorter); silence: the speech is padded with silence (as long as the longer); loop / bounce: the picture repeats, or plays back and forth, to cover longer speech (as long as the speech) | remap |
| audio_range | No | Only this span of the audio, in the audio's seconds. Default: all of it | |
| idempotency_key | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
create.matteCreate a matteIdempotentInspect
A solid or gradient picture (PNG, any length on a timeline): backgrounds, slugs, colour washes with clip.set_blend. Gradients work like CSS: angle 180 runs top to bottom; radial reaches the corners; colours may carry alpha (#RRGGBBAA). Long-running like every create.* tool, but quick.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| color | No | A solid colour | |
| width | No | ||
| height | No | ||
| gradient | No | ||
| idempotency_key | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
create.motionCreate a motion graphicIdempotentInspect
Render animated HTML to a video with alpha, frame-exactly: lower thirds, animated titles, callouts, logo stings, kinetic type, generated graphics. Place the media_id with clip.add on a track above the video. Up to 60 s, and width × height × fps × seconds up to 30 s of 1080p30. Runs in the sandbox: no network (inline everything, load libraries from lib/; assets are for imported files the page must show as they are, like the user's logo or a font), fixed memory and time limits. To match a reference you were shown, rebuild it in HTML/SVG; it does not need importing. The result's warnings list what went wrong in the page: uncaught errors, console errors, files blocked or not found. Long-running: returns a task, or {job_id, media_id} for job.get. The same recipe returns the same media.
| Name | Required | Description | Default |
|---|---|---|---|
| fps | No | ||
| html | Yes | An animated HTML page or fragment laid out at width × height. Anything that animates in a browser renders frame-exactly: CSS animations and transitions, the Web Animations API, SVG, requestAnimationFrame and timers (canvas, WebGL), window.__seek(seconds) for hand-driven drawing. Libraries load from lib/: anime.umd.min.js, three.module.js, lottie.min.js, d3.min.js. | |
| name | No | ||
| width | No | ||
| assets | No | ||
| height | No | ||
| transparent | No | Alpha (ProRes 4444) so it composites over video; false gives H.264 | |
| idempotency_key | No | ||
| duration_seconds | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
create.stabilizedSteady shaky videoIdempotentInspect
A steadied copy of shaky video, local (vid.stab): one pass measures the camera shake over the whole media, a second moves each frame against a smoothed camera path. New media, ProRes with the sound, to put on the timeline in place of the shaky clip (clip.replace keeps its place and timing). Long-running: a task, or {job_id, media_id} for job.get.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| edges | No | What fills the edges steadying moves into view. zoom: one zoom for the whole video, enough to hide them; adaptive_zoom: as little zoom as each moment needs; black: no zoom, black edges; keep: no zoom, edges filled from earlier frames | zoom |
| tripod | No | Hold the picture still against its first frame, as if on a tripod | |
| media_id | Yes | Video to steady | |
| shakiness | No | How shaky the footage is: 1 a little, 10 a lot | |
| smoothing | No | Frames each side the camera path is smoothed over: more is steadier but follows intended moves (pans) more slowly | |
| idempotency_key | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
create.stemsSeparate voice and musicIdempotentInspect
The sound of a video or audio file pulled apart into stems, each new audio media for clip.add: vocals (voices, spoken or sung), drums, bass, other (everything else: guitars, keys, effects, room noise) and no_vocals (all but the voice, in one file). Laid together at the same time, the four stems make the original sound again. Use it to keep only the voice (lose the music under an interview: vocals), keep only the music (take the voice out of a clip: no_vocals), turn the music down under speech (vocals and no_vocals on their own tracks, with their own gain), or take a beat or a bass line on its own. Demucs v4 (htdemucs), local, on the CPU: about a fifth of the media's length on 8 cores (a minute of sound in about 15 s). Built for music; on speech with background noise, vocals keeps the voice and other takes most of the noise. Media up to 3 hours. Each stem is a 32-bit float WAV, 44.1 kHz stereo, as long as the original; the same media and stems give the same media again while they last. Long-running: a task, or a job_id for job.get.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| stems | No | vocals: voices, spoken or sung; drums, bass, other: the rest, by kind (other holds everything else: guitars, keys, effects, room noise); no_vocals: all but the voice in one file. Each becomes its own media | |
| media_id | Yes | Video or audio with sound | |
| idempotency_key | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
create.stillCreate a stillIdempotentInspect
Render an HTML page to a PNG: title and end cards, thumbnails, overlays, designed text. Transparent where nothing is painted. Capture waits for fonts, and for window.__ready if the page sets it to a promise (up to 10 s). Runs in the sandbox: no network (inline everything, load libraries from lib/; assets are for imported files the page must show as they are, like the user's logo or a font), fixed memory and time limits. To match a reference you were shown, rebuild it in HTML/SVG; it does not need importing. The result's warnings list what went wrong in the page: uncaught errors, console errors, files blocked or not found. Long-running: returns a task, or {job_id, media_id} for job.get.
| Name | Required | Description | Default |
|---|---|---|---|
| html | Yes | An HTML page or fragment, laid out at width × height | |
| name | No | ||
| width | No | ||
| assets | No | ||
| height | No | ||
| transparent | No | Unpainted areas stay transparent | |
| idempotency_key | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
create.upscaledAdd resolutionIdempotentInspect
A larger copy of low-resolution video or an image with detail drawn in, local: Real-ESRGAN (realesr-general-x4v3, on the CPU) makes each frame four times the size, then it is resized to scale. Slow: on 8 cores about 1 s a frame for 320x180, 2.5 s for 640x360. Sources up to 1280x720, results up to 3840 px on the long edge. New media (ProRes with the sound, or a PNG) to put in place of the original (clip.replace keeps its place and timing). Long-running: a task, or {job_id, media_id} for job.get.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| scale | No | How many times wider and taller the result is | |
| denoise | No | How much noise and compression blocking is cleaned while detail is drawn in: light keeps grain and fine texture, strong gives the smoothest picture | medium |
| media_id | Yes | Video or image to enlarge | |
| idempotency_key | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
create.voiceoverCreate a voiceoverIdempotentInspect
English text spoken by a natural voice (Kokoro, local; the model downloads once) as WAV media. words (default) times each word into the media transcript (media.get include transcript). Long-running: a task, or {job_id, media_id} for job.get.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| text | Yes | What to say, in English | |
| speed | No | ||
| voice | No | a/b: American/British; f/m: female/male | af_heart |
| words | No | Time each word | |
| idempotency_key | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
edit.batchBatch editAIdempotentInspect
Apply many edits as one change: one new timeline_id, one preview. All apply or none do. Use {"$ref":"ops[N].clip_id"} (or track_id, clip_ids[M]) to use an id an earlier op made. Returns the edit result plus results: [{tool, clip_id?, clip_ids?, track_id?}] per op.
| Name | Required | Description | Default |
|---|---|---|---|
| ops | Yes | Edits applied in order as one change. An arg may be {"$ref":"ops[2].clip_id"} to use an id made by an earlier op | |
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| results | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| placements | No | Where each clip the edit moved, cropped, masked or animated shows, sequence pixels |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare non-read-only, idempotent, non-destructive, but the description adds the key behavioral fact they do not: transactional semantics ('all apply or none do') and that each call yields one new timeline_id plus one preview. It also discloses the return envelope (results per op), going beyond the annotation set.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three dense sentences, front-loaded with the core contract (atomic batch, one timeline_id, one preview) before the $ref mechanics and return shape. No filler or restatement of the name/title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers atomicity, cross-op referencing, and return shape; an output schema exists so return value detail is not required. Minor gap: no mention of the 200-op cap or ordering-failure diagnostics, though the schema carries maxItems and minItems.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description earns extra credit by generalizing the $ref mechanism beyond the schema's single example: it shows forms for clip_id, track_id, and clip_ids[M], clarifying that any arg may reference an earlier op's created id.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Apply many edits as one change') and immediately names the distinguishing property: a single new timeline_id, a single preview, and all-or-nothing atomicity. An agent can tell this apart from the ~40 single-operation siblings without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The atomicity framing ('All apply or none do') implicitly tells the agent when to reach for this instead of chaining individual sibling tools, and the $ref note points to ordered multi-op workflows. It stops short of explicitly naming alternatives or stating when not to batch.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
export.timelineExport timelineADestructiveIdempotentInspect
Write the edit for another editor: fcpxml (Final Cut Pro, DaVinci Resolve), otio (Premiere, Resolve, OpenTimelineIO apps), edl (CMX 3600, one video track), or srt/vtt (the captions). Best effort: warnings lists what the format could not carry (masks, blend modes, effects, animated transforms, ramps, text styling). timeline.get is the lossless copy. media reference (default) points at the media where this server keeps it, which another machine cannot open and which goes 7 days after last use (the result warns); media collect makes a .zip with the media files next to the timeline file, to hand the edit on. Long-running: a task, or {job_id} for job.get; the result has a signed download URL.
| Name | Required | Description | Default |
|---|---|---|---|
| media | No | reference: point at the media where it is; collect: a .zip with the media in it | reference |
| format | Yes | fcpxml: Final Cut Pro, Resolve; otio: Premiere, Resolve, any OTIO app; edl: CMX 3600; srt/vtt: captions | |
| filename | No | Output file name without extension. Default: timeline name and id | |
| overwrite | No | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| idempotency_key | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations it discloses that this is best-effort with a warnings list naming what could not carry (masks, blend modes, effects, ramps, text styling), that reference media expires 7 days after last use, that the call is long-running via task or job_id, and that the result carries a signed download URL. The one gap is that destructiveHint=true is never explained — nothing says what gets destroyed or how overwrite relates to it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the purpose, then the lossless alternative, then caveats, then media modes, then the async mechanics — a sensible order with no filler sentences. The telegraphic, parenthetical style ("(masks, blend modes...)", "(the result warns)") is dense but occasionally reads as notes rather than prose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a long-running export with an output schema and a warnings channel, the description covers what an agent must decide before calling: format, media mode, and how to handle the async result. Return format is legitimately delegated to the output schema, and the lossless-vs-lossy fork is resolved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67%, so the schema already documents the enums. The description restates format and media semantics rather than adding to them, though it does add the 7-day expiry rationale for reference media. It says nothing about filename, overwrite, or idempotency_key, which is the un-covered third.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
"Write the edit for another editor" states a specific verb and outcome, then enumerates the concrete artifacts (fcpxml, otio, edl, srt/vtt). It explicitly distinguishes itself from timeline.get ("the lossless copy") and implicitly from export.video by naming the interchange formats, so an agent can pick it out of a 60+ sibling list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives the selection rule for every format by naming the target application (FCP/Resolve, Premiere/OTIO, CMX 3600, captions) and routes lossless copying to timeline.get. It also states when to choose media collect over the default reference, with the reason (handing the edit to another machine).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
export.videoExport videoADestructiveIdempotentInspect
Render the timeline to a file. Presets: web_1080p_h264 (MP4, fits 1920x1080), master_prores_422hq (MOV, sequence size), social_9x16_h264 (9:16 sequences, 1080x1920), web_1080p_av1 (MP4, AV1/Opus), gif (640 px, 15 fps), review_h264 (MP4, fits 854x854 with sound: small and quick, to watch a cut), audio_wav_48k, audio_mp3. overrides changes size, fps, bitrate or crf (video://catalog/presets). Long-running: returns a task, or {job_id} for job.get. The result links the file (a resource_link to video://jobs/{job_id}/output, which reads as the file, or its signed URL when large), has a signed download URL and a page_url that plays it in any browser (the link to give a person to watch or share), both working for 24 h (job.get gives fresh ones) . The server keeps the file 7 days: to keep it longer, save it from the url, and media.import it to use it again. qa.check measures a render (gaps, frames, loudness) before you export. sha256 and fingerprint tell two renders apart without downloading them (fingerprint matches media.get of the file once imported).
| Name | Required | Description | Default |
|---|---|---|---|
| range | No | Frames, end exclusive | |
| preset | Yes | video://catalog/presets lists each with what it takes | |
| filename | No | Output file name without extension. Default: timeline name and id | |
| overrides | No | Change the preset: a size box (aspect kept), frame rate, bitrate or quality | |
| overwrite | No | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| idempotency_key | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations: discloses that the call is long-running and returns a task or {job_id}, that download/page URLs expire in 24 h, and that the server retains the file only 7 days. It also describes the return payload (resource_link, signed URL, page_url, sha256, fingerprint), which the destructive/idempotent hints alone would never convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action and preset list, then lifecycle details. It is dense and a few sentences run long (the URL/retention sentence is overloaded), but nearly every clause carries operational information an agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 7-parameter, nested-object, long-running export tool, the description covers presets, overrides, async behavior, output shape, URL expiry and retention. Even with an output schema present, the extra lifecycle context makes the definition self-sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds meaning beyond the 71%-covered schema: each preset is described with container, resolution and use case (e.g. review_h264 as a small quick cut to watch), overrides are explained as a size box/fps/bitrate/quality change, and filename's default is stated. The idempotency_key and overwrite parameters are left unexplained, keeping it from a 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource: 'Render the timeline to a file.' That is unambiguous about what the tool produces. It does not, however, distinguish itself from the sibling export.timeline, which an agent would reasonably confuse with this one.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives real routing guidance: run qa.check to measure a render before exporting, use job.get for fresh URLs, and media.import to reuse a file after the 7-day window. It stops short of stating when to prefer export.timeline or render.frames instead, so no explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
filter.addAdd effectAIdempotentInspect
Apply an effect to a clip, every clip on a track, or the whole timeline: picture effects (colour, LUT, blur, vignette, chroma key, glitch, denoise and more) on video, sound effects (EQ, compressor, noise_reduction, voice_denoise, pitch and more) on audio. Order matters: effects run in the order added (a track's before a clip's own). Picture numbers take keyframes, except glitch, denoise and chroma_key tolerance; sound params are constants. A wrong name or param is refused with the full list, ranges, units and defaults (also at video://catalog/effects); timeline.get shows every param applied. Returns its id as clip_id. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| params | No | The effect's own params; any left out take their defaults. A wrong one is refused with the effect's params, ranges, units and defaults | |
| clip_id | No | Apply to one clip (or an adjustment clip) | |
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline | No | Or true for the whole picture or mix | |
| track_id | No | Or to every clip on a track | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds a lot beyond the annotations: ordering semantics, that picture params accept keyframes except glitch/denoise/chroma_key tolerance, that sound params are constants, and that invalid names/params are rejected with the full list, ranges, units and defaults (also at video://catalog/effects). One soft tension with idempotentHint=true: 'effects run in the order added' implies calling twice stacks two effects, so repeated invocation is not state-neutral, and the description never reconciles this.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense paragraph, front-loaded with what gets the effect and where; every clause carries information. It is long, but the length is earned by the number of distinct behavioral rules packed in, with few wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers targeting, ordering, keyframe limits, error recovery via the effects catalog, the returned id, and defers return-shape detail to the server instructions (an output schema exists). The only gap is the idempotency/stacking question an agent would face when re-applying an effect.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 86%, so the schema already documents nearly all parameters; the description mostly reinforces it. It does add genuinely extra meaning (which effect families take keyframes vs. constant params, that params omitted take defaults, the refusal-with-details error contract), but it adds no format/syntax detail for the params object itself.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Apply an effect') plus the three possible targets (clip, all clips on a track, whole timeline) and enumerates the effect families (colour, LUT, blur, EQ, compressor...). The purpose is unambiguous, but it never names the adjacent filter.set / filter.remove siblings, so the agent must infer that this tool only creates effects and does not modify them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives concrete selection rules for each targeting parameter ('Apply to one clip', 'or true for the whole picture or mix', 'or to every clip on a track'), plus an evaluation-order rule (a track's effects run before a clip's own). It does not say when to use filter.set or filter.remove instead, which is the main missing piece.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
filter.removeRemove effectCIdempotentInspect
Take an effect off. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| filter_id | Yes | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the operation is a non-read-only but non-destructive, idempotent local mutation, so safety is covered. The description adds that an edit result is returned and defers its shape to the server instructions, which is useful. It does not say what happens if filter_id is unknown, whether the removal fails silently, or how the remaining effect stack is affected.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with the action front-loaded and a pointer for return values; no filler. The brevity is efficient, though it shades into under-specification rather than verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return values needn't be spelled out, but for a mutation tool this leaves real gaps: no source for filter_id, no behavior on missing/invalid ids, and no note that the timeline handle changes after the edit (only visible in the schema). Not enough for an agent to call it confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 67% and filter_id has no description anywhere; the description contributes zero parameter meaning, notably not telling the agent where a filter_id comes from (filter.add / timeline.get). preview and timeline_id are already documented in the schema, so only the undocumented id remains, and it is unaddressed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
"Take an effect off" names a verb and a resource, so the basic action is inferable, but it is close to a restatement of the title "Remove effect" and uses "effect" rather than the schema's "filter". Nothing distinguishes it from siblings like transition.remove, clip.remove or filter.set.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to use this versus filter.set (modify) or transition.remove, and no prerequisites such as the filter existing on the timeline. The agent must infer the routing decision entirely from the tool name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
filter.setSet effectAIdempotentInspect
Change an effect's params (merged) or turn it off and on. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| params | No | Merged into its params | |
| enabled | No | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| filter_id | Yes | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare mutation, idempotency, and non-destructiveness. The description adds useful detail that params are merged and that the effect can be enabled/disabled, plus a pointer to return behavior, but does not explain permission needs or full side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The definition is two short sentences with no filler, and the main action is front-loaded before the return-value note. Every phrase contributes to understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists and annotations cover the safety profile, so the description need not explain return values or mutation risk. However, the description lacks context around preview behavior, required identifiers, and sibling differentiation, leaving gaps for a five-parameter mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 60%, so some parameters are already documented. The description clarifies that params are merged and that the tool can turn the effect on/off, but it does not explain filter_id, preview modes, or the required timeline handle beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action: change an effect's params or toggle it on/off. It clearly distinguishes mutation of an existing filter from creation or removal, though it does not explicitly name sibling tools like filter.add or filter.remove.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the described operations, but there is no explicit guidance on when to use filter.set versus filter.add, filter.remove, or text.set. The agent must infer the appropriate context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
job.cancelCancel jobADestructiveIdempotentInspect
Stop a job, queued or running, within seconds: its engines are killed and its slot goes to the next job. Finished jobs are left as they are.
| Name | Required | Description | Default |
|---|---|---|---|
| job_id | Yes | Job handle; the same value is the MCP task id |
Output Schema
| Name | Required | Description |
|---|---|---|
| for | No | What it is for, known from the start |
| tool | Yes | |
| error | No | |
| job_id | Yes | Job handle; the same value is the MCP task id |
| result | No | The tool's structured result once completed |
| status | Yes | |
| progress | No | |
| created_at | Yes | |
| updated_at | Yes | |
| seconds_left | No | Running: seconds left at its recent rate of progress, once it has moved steadily (later steps may differ) |
| result_summary | No | result "summary": the result in brief; result "full" has all of it |
| status_message | No | What it is doing; while it waits, its place in the queue and what it waits for (a job slot, a render slot or the sandbox) |
| starts_in_seconds | No | Queued next in line: seconds until a slot should free, when every job holding one has seconds_left |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well past the destructiveHint/idempotentHint annotations by stating the actual mechanics: engines are killed, the slot is released to the next job, and terminal jobs are unaffected. That last clause effectively documents no-op semantics for already-finished jobs, which the annotations cannot express.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, zero filler, with the scope constraint and the effect front-loaded before the edge-case clause. Every clause carries information an agent would otherwise have to guess.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained, and the description covers scope, timing, and side effects. It leaves open whether cancellation is reversible and what error or result occurs when the job is already terminal.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter and the schema already documents it fully at 100% coverage, including the pattern and the note that it doubles as the MCP task id. The description adds no additional meaning about job_id, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb plus resource ("Stop a job") with an explicit scope qualifier ("queued or running") and a concrete timing claim ("within seconds"). An agent can immediately tell this apart from job.get or job.list, which read state rather than terminate work.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Finished jobs are left as they are" is an implicit when-not condition that prevents pointless calls on completed jobs. It does not name sibling alternatives (job.get/job.list for status checks), so it stops short of full routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
job.getGet jobARead-onlyIdempotentInspect
Status, progress and, once completed, the result of long-running calls (media.import, media.analyze, create.*, export.video, qa.check). Same record as tasks/get. status_message says what it is doing, or where it stands in the queue; for names the media or timeline. wait_seconds (up to 50) answers when any job asked for finishes, fails or is cancelled, or when the wait runs out (then with its progress), so one call waits instead of many polls. seconds_left and starts_in_seconds come only when progress so far allows. job_ids reads many and answers {jobs, not_found}, each result in brief (result_summary: lists cut to their first items, with what was cut) unless result is full; result summary does the same for one job. A job whose progress stops for minutes ends with a stalled error, freeing its slot.
| Name | Required | Description | Default |
|---|---|---|---|
| job_id | No | One job: the answer is that job | |
| result | No | Default full for one job_id; summary for job_ids, so a check on many stays small (full gives them whole) | |
| job_ids | No | Many jobs at once (up to 100): the answer is {jobs, not_found} | |
| wait_seconds | No | Wait up to this long (at most 50 s) for any job asked for to finish (its status changes); progress along the way does not end the wait; 0 answers at once |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the read-only and idempotent annotations, the description discloses important behavior: wait_seconds can block until any requested job finishes, fails, or is cancelled; progress does not end the wait; stalled jobs eventually error out; and result summaries truncate lists. These details materially help an agent predict what a call will do.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose, but it is a dense run-on paragraph with semicolon-separated clauses and some repetition of schema parameter details. It is informative but could be structured more cleanly for faster scanning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema, rich annotations, and full schema coverage, the description is complete enough for an agent to call the tool correctly. It covers waiting behavior, batch reads, result summarization, and stalled-job handling without needing to explain return values in detail.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema already documents job_id, result defaults, job_ids batch behavior, and wait_seconds semantics. The description largely restates those parameter meanings rather than adding substantial new parameter-level guidance, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: it returns status, progress, and completed results for long-running jobs, with concrete examples like media.import and export.video. It is clear what the tool does, though it does not explicitly contrast itself with job.list or job.cancel.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explains the main usage context: checking long-running calls, waiting instead of polling repeatedly, and reading many jobs at once via job_ids. It does not explicitly state when to use job.list or job.cancel instead, so the alternative-selection guidance is not complete.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
job.listList jobsARead-onlyIdempotentInspect
Your recent jobs, newest first, each with its status, progress and for (the media or timeline it is for), filtered by status, tool or media_id. Results are left out unless you ask (result summary or full); job.get reads one.
| Name | Required | Description | Default |
|---|---|---|---|
| tool | No | Only jobs of this tool, e.g. media.import | |
| limit | No | ||
| result | No | none (default): each job without its result, to find one; summary: lists cut short; full: every result whole | none |
| status | No | ||
| media_id | No | Only jobs for this media (importing, making or reading it) |
Output Schema
| Name | Required | Description |
|---|---|---|
| jobs | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover readOnly/idempotent/non-destructive, so the bar is lower, and the description still adds real behavior: results are omitted by default to keep the listing light, with opt-in 'summary' or 'full'. It doesn't mention the limit cap or what happens when more jobs exist than the limit, leaving a pagination gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One dense sentence that front-loads ordering and returned fields before the filters, with the default-exclusion rule and the sibling pointer at the end. Every clause carries information, though the phrase 'and for (the media or timeline it is for)' is awkward and slightly costly to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values needn't be described, and the description covers filtering, verbosity control, ordering and the alternative tool. The only meaningful omission is how limit interacts with result size and whether results are truncated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 60% and the description reinforces the filter semantics of tool, status and media_id ('importing, making or reading it') and the three result modes. However, 'limit'—the one undocumented parameter in the schema—is never addressed in the description, so the coverage gap remains unclosed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (list) and resource (jobs) plus ordering ('newest first') and the per-job fields returned (status, progress, target), then names the sibling 'job.get' that reads a single job. An agent can distinguish this from job.get and job.cancel without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains the workflow split: this tool finds a job, and 'job.get reads one' once you know which. It also states results are excluded by default unless requested, which shapes how to call it. It stops short of explicit when-not-to-use guidance (e.g. preferring job.get for a known id).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
judge.askAsk questions about text or picturesARead-onlyIdempotentInspect
Answer your questions about text and pictures you give (yes or no, a choice, a score, or where something is in a picture), as typed answers in about a second. It asks TypeSafe Jev, a fast judgment model (preview).
Jev sees exactly the state: each entry under its name, and your questions refer to entries by name in backticks ("in passage"). An entry is text (a string, or a JSON object or array of related facts) or a picture: {media_id} for an imported image, {media_id, at_seconds} for the frame of a video at that time, {timeline_id, at_seconds} for a frame of an edit as it renders (to judge a grade or a layout you made). Mix them freely. Each call stands alone, so put everything a judgment needs into its state.
Choose the type by the answer you need. noul: yes or no, answered as noul, the probability of yes from 0 to 1 (0.5 or more reads as yes). choice: one of options you name, answered as choice plus probabilities and confidence. score: a degree on levels you order, answered as score (0 is the first level) plus confidence. box: where one thing is in a picture, answered as box {x, y, w, h} in fractions of that picture plus confidence.
Write each question as one thing, worded so yes means it is there. Open with what the state is ("passage is a transcript of someone speaking off the cuff"), and give a noul criteria {true, false} with an example of each: Jev answers most accurately from examples. To locate, name the one thing so only it fits ("the yellow headline 'Get started' at the top of shot") and ask a noul whether it is there; trust a box with high confidence.
One call answers many questions about one state at once, so judge many items together: put the shared text in state once, and give each item its own question carrying its own marked copy. Filler words: state {passage: "So basically the way this works is"}, and per word a noul "passage is a transcript of someone speaking off the cuff. With one word marked: "So [[basically]] the way this works is". Is the marked word filler, so the sentence says the same without it?" with criteria {true: "the [[the]] report is late", false: "the report is [[late]]"}. A shot: state {shot: {media_id, at_seconds: 12.5}}, choice "What fills shot?" with options {speaker: "A person talking to camera", screen: "A screen recording", none: "Anything else"}. Both: state {line: "Click Get started", screen: {media_id, at_seconds: 4}}, noul "Does screen show the button line talks about?".
Up to 300 questions, 8 pictures and 200k characters of text a call. To find where something shows in a video or an edit, media.seek takes it in plain words; to measure every frame, media.analyze look asks your questions of frames it samples. Guide to writing questions: https://docs.typesafe.ai/primitives and https://docs.typesafe.ai/images.
| Name | Required | Description | Default |
|---|---|---|---|
| state | Yes | What Jev sees, entry by entry: name to a string, a JSON object or array, or a picture: {media_id} (an image), {media_id, at_seconds} (a video frame) or {timeline_id, at_seconds} (a frame of an edit as it renders). Questions refer to entries by name in backticks | |
| questions | Yes | id (yours, not shown to the model) to question; up to 300, all answered about the same state |
Output Schema
| Name | Required | Description |
|---|---|---|
| model | Yes | |
| answers | Yes | Per question id: noul {noul}; choice {choice, probabilities, confidence}; score {score, probabilities, confidence}; box {box: {x, y, w, h} fractions of the picture, confidence} |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/openWorld/idempotent/non-destructive, and the description adds context beyond them: statelessness ('Each call stands alone, so put everything a judgment needs into its state'), capacity limits (300 questions, 8 pictures, 200k characters), latency ('about a second'), and that the model is a preview.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The core purpose and limits are front-loaded, but the body is a dense run-on of inline examples with no headings or lists despite the highly structured content. The filler-word example in particular requires re-reading, and the paired criteria example does not clearly map to the filler question.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a nested two-parameter tool with an output schema, the description covers everything an agent needs: entry formats, question types and criteria shapes, per-call limits, statelessness, and links to fuller guides. Return values need not be described because the output schema exists.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline would be 3, but the description adds real meaning: backtick entry references ('in `passage`'), the interpretation rule '0.5 or more reads as yes' for noul, 'score (0 is the first level)', and 'box {x, y, w, h} in fractions of that picture'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening states a specific verb and resource ('Answer your questions about text and pictures you give'), enumerates the four answer shapes (yes/no, choice, score, box), and names the underlying model (TypeSafe Jev). It also distinguishes the tool from media.seek and media.analyze by describing what each does instead.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It routes the agent explicitly: 'To find where something shows in a video or an edit, media.seek takes it in plain words; to measure every frame, media.analyze look asks your questions of frames it samples.' It also gives selection guidance for the question subtypes ('Choose the type by the answer you need') and a batching rule ('judge many items together').
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
media.analyzeAnalyze mediaADestructiveIdempotentInspect
Find what is in imported media, or in the sound of an edit, as data for edits. transcript: every word with its time (whisper.cpp on the CPU; the model downloads once). For English the word edges are then aligned to the sound (wav2vec2 forced alignment): a word starts and ends within about 0.05 s of where its letters sound, and an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more; timing_confidence says how clearly each word was heard. Other languages keep whisper's own times, up to about a second off. Short ums and uhs are often left out of the words; kind speech still hears them. from_seconds and until_seconds transcribe just that stretch. beats: tempo, beat and downbeat times, and strengths: each beat's onset strength, 0–1 of the strongest (the hardest hits, the drop). faces: face boxes over time, e.g. to reframe to 9:16 with clip.set_transform keyframes. track: follow a region you give from a moment on, a box for each sampled time (found false where the subject was lost); to make a title, sticker, arrow or mask follow the subject, turn the boxes into clip.set_transform (or clip.set_mask) keyframes, no tracking of your own needed. look: your own typed questions about what is on screen, answered on every sampled frame (to find where something happens from plain words, media.seek). silence: stretches quieter than threshold_db for at least min_seconds. scenes: times the picture changes by more than threshold (0–100; hard cuts between ordinary shots score about 10 to 30, see threshold), with each cut's score. speech: the probability that someone is speaking, every 32 ms (Silero VAD), and the spans where it is at least your min_probability; it hears speech, not noise or music, so a cough or a door is not speech. loudness: integrated loudness (LUFS), true peak (dBTP), loudness range (LU), RMS level and noise floor (dBFS) over the whole media or from_seconds to until_seconds. To line up two recordings, media.correlate. sounds: what can be heard and when, every 40 ms, for 447 named sounds (video://catalog/sounds: Laughter, Applause, Cough, Sneeze, Music, Speech, Door, Footsteps, Beep…): give labels for the ones you want, or none for every sound that reaches your min_probability; you get each sound's spans and its probability in every frame (a run over the whole media is kept with it; one over from_seconds to until_seconds comes back only). To find a sound you describe in your own words, media.listen. speakers: who speaks when (pyannote and ERes2Net on the CPU, about 10 to 20 s a minute): turns numbered in the order voices are first heard; give speakers (how many) or cluster_threshold. Transcript words then carry their speaker. voice: pitch (YIN, Hz) and level (RMS dBFS) every 0.1 s, and a summary (pitch median and range, level, voiced seconds, words per second from the transcript), e.g. as judge.ask state. colour: per-channel mean and percentiles, luma, white point estimate and clipping (RGB 0–255) of the frame and regions you name, at sampled times. picture: FFmpeg signalstats of every frame over a stretch: luma, chroma, saturation, hue, and ydif, the change from the frame before (a flash or frozen frame shows there). With timeline_id instead of media_id, speech, loudness, silence, sounds, speakers, voice measure the edit's mix and faces, track, look, scenes, colour, picture its picture as it renders (titles, effects and grades included), over from_seconds to until_seconds (timeline seconds); they come back in the result. A source or range with no sound fails with no_audio, naming it. Times are media seconds (timeline seconds for a timeline); boxes are fractions of the picture, and a box you give takes units "pixels" too (picture_size gives the size). Results stay with the media (media.get include; the latest run of each kind) except track, silence, scenes and loudness, which come back in the result; speech over the whole media is kept (media.get include speech), so asking again with another min_probability or range answers from it. A result too long to return (a long transcript) comes back as its summary and a note: read it in pieces with media.get include and a time range or page. Long-running: returns a task, or a job_id for job.get.
HOW LOOK WORKS. TypeSafe Jev, a fast judgment model (preview), answers your questions about frames sampled from the media. Each sampled frame is its own request: Jev sees that one picture, named frame, and with with_previous also the frame sampled just before it, named previous, plus any state you give: text or JSON entries by name, the same on every frame (the script, what a term means). Every question is answered on every frame, independently and in parallel (about 0.15 s a frame), as typed data you then combine in your own code.
Choose the type by the answer you need. noul: yes or no, answered as noul, the probability of yes from 0 to 1. choice: one of options you name, answered as choice plus probabilities and confidence. score: a degree on levels you order, answered as score (0 is the first level; it can land between levels) plus confidence. box: where one thing is, answered as box {x, y, w, h} in fractions of the picture plus confidence.
Write each question about the picture as it looks. Name the picture in backticks ("in frame"), ask about one thing, word it so yes means the thing is there, and describe it as it appears: "Does frame show a human being, even if only their face fills the picture?", "Is one large word laid over frame, much bigger than any other text?". To compare, name both pictures and the thing that differs: "Is there a button in frame that is not in previous?". To locate, name the one thing so only it fits ("the 'Get started for free' button under the headline in frame") and, in the same call, a noul asking whether it is there; use the box on frames where that noul is high. Give a choice a none option for frames where nothing fits, and each score level a concrete situation.
Read the answers this way: a noul of 0.5 or more reads as yes, and you can set your own cut-off; confidence tells you how far to trust a choice, a score or a box, and a loose box shows as low confidence. Work in passes: every_seconds 1 over the whole media, then at_seconds around the times where answers change. media.get scenes give when the picture cuts; look tells you what is on it.
The result gives each question's spans (when a noul reads yes, which choice holds when); every frame's answers stay with the media (media.get include ['look'], the last 10 runs). Up to 600 frames and 32 questions a call. To find the moments that match a description, media.seek takes it in plain words and returns where it holds. For one judgment about text, a still or a single frame, judge.ask answers at once. Full guide to writing questions: https://docs.typesafe.ai/primitives and https://docs.typesafe.ai/images.
| Name | Required | Description | Default |
|---|---|---|---|
| kind | Yes | transcript: words with times (whisper.cpp, then aligned to the sound); beats: tempo, beat times and the onset strength at each; faces: face boxes over time; track: follow a region; look: your own questions answered on sampled frames; silence: stretches quieter than threshold_db for min_seconds; scenes: times the picture changes by more than threshold; speech: where someone is speaking (Silero VAD probability every 32 ms, spans at or above min_probability); loudness: integrated loudness, true peak, loudness range, RMS and noise floor; sounds: when each of 447 named sounds (laughter, applause, cough, music, speech, door…) can be heard, every 40 ms; speakers: who speaks when; voice: pitch, level and pace over time; colour: colour statistics of frames and regions; picture: every frame's brightness, colour and change from the frame before | |
| model | No | transcript: whisper size, on the server CPU. Bigger is more accurate and slower: small takes about 3 times as long as base, tiny about half | base |
| state | No | look: text or JSON entries shown beside `frame` on every frame, by name, for questions to refer to (a script line, a definition) | |
| labels | No | sounds: the sounds to report, named as in video://catalog/sounds (e.g. ["Laughter", "Applause", "Cough"]). Default: every sound that reaches min_probability somewhere | |
| region | No | track: the box to follow (top-left x, y and size) and when | |
| regions | No | colour: named boxes to measure besides the whole frame (frame), up to 16 | |
| language | No | transcript: e.g. 'en', or 'auto' | auto |
| media_id | No | Analyze imported media (media seconds) | |
| speakers | No | speakers: how many people speak, when you know (or give cluster_threshold) | |
| questions | No | look: id (yours, not shown to the model) to question; up to 32, all asked of every frame | |
| threshold | No | scenes (required): how much the picture must change to count as a cut, 0–100 (FFmpeg scdet's score): the change from the frame before, in percent of full range, less the change just before it (so steady motion stays low). Hard cuts between ordinary shots score about 10 to 30, a cut between opposite pictures up to 100: start at FFmpeg's own default, 10, and raise it if you get too many | |
| at_seconds | No | look, colour: sample exactly these times instead | |
| min_seconds | No | silence (required): only stretches at least this long | |
| percentiles | No | colour: the percentiles to give of each channel | |
| timeline_id | No | Analyze an edit as it plays (timeline seconds): its mix (speech, loudness, silence, sounds, speakers, voice) or its picture (faces, track, look, scenes, colour, picture) | |
| from_seconds | No | Start sampling or measuring here (any kind but beats, and faces and scenes on media). Default 0 | |
| threshold_db | No | silence (required): quieter than this counts as silence, dBFS | |
| every_seconds | No | look, colour: sample a frame this often. Default 1 | |
| until_seconds | No | Stop sampling or measuring here. Default: the end | |
| with_previous | No | look: also show the frame sampled before, as `previous` | |
| idempotency_key | No | ||
| min_probability | No | speech (required): a 32 ms frame with at least this speech probability is speech; sounds (required): a 40 ms frame has the sound when its probability is at least this | |
| cluster_threshold | No | speakers: when the count is unknown, how different two voices must be to count as two people; lower finds more speakers (try 0.5 to 0.9) |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the supplied annotations the description discloses persistence rules (results kept with the media via media.get include, except track, silence, scenes and loudness which return inline), truncation behavior (long transcripts return as summary plus a note), failure modes ("fails with no_audio, naming it"), execution model (returns a task or job_id for job.get), cost (speakers ~10-20 s per minute of media), and hard limits (600 frames, 32 questions, last 10 look runs). These are exactly the behavioral traits the annotations cannot express.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The initial purpose sentence is well front-loaded, but the body is a single semicolon-chained run-on enumerating 14 kinds, followed by a very long "HOW LOOK WORKS" tutorial with external doc links. Given the 14-kind surface area most content earns its place, but the presentation is hard to scan and it repeats the enum descriptions verbatim from the schema instead of trusting them.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, 96% schema coverage and rich annotations, the description fills the remaining gaps an agent needs: which kinds persist vs return, which parameters are required per kind, timeline vs media semantics, and the sampling/limit constraints. Nothing material is missing for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 96%, so the schema already carries most parameter documentation, and much of the description's per-kind parameter prose duplicates the enum and property descriptions. It still adds real interpretive value: calibrating threshold against FFmpeg scdet scores ("hard cuts between ordinary shots score about 10 to 30… start at 10"), guidance on cluster_threshold ranges, and the fractional-vs-pixel box units convention.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a concrete verb-and-resource statement ("Find what is in imported media, or in the sound of an edit, as data for edits") and then enumerates all 14 `kind` values with what each returns. It actively distinguishes itself from media.seek, media.listen, media.correlate, judge.ask and media.get, so an agent can route without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit routing is given per sibling: "to find where something happens from plain words, media.seek"; "To find a sound you describe in your own words, media.listen"; "To line up two recordings, media.correlate"; "For one judgment about text, a still or a single frame, judge.ask answers at once." It also states media_id vs timeline_id selection and which kinds work on which, plus mode-specific exclusions (from_seconds not valid for beats).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
media.correlateSync offsetAIdempotentInspect
Find how far one recording sits from another: the sync offset of b against a, by cross-correlating their sound, with how strongly they match. Use it to line up a camera with a separate microphone, two cameras of the same moment, or a re-recorded line against the original. Each side is imported media or an edit's mix (timeline_id), over its own from_seconds to until_seconds, so you can match a short stretch of b against a longer stretch of a. offset_seconds: b's moment t lines up with a's moment t + offset_seconds, each in its own seconds. To line them up on a timeline, place b's clip so it starts offset_seconds later than a's (earlier when negative), source starts being equal. strength is the correlation of the overlapping sound at that offset, -1 to 1: near 1 the same sound, near 0 unrelated; next_best is the best offset outside the winning peak, and a strength close to it means the match is unclear (repetitive sound, or not the same moment). Offsets are searched up to max_offset_seconds either way, where at least half of the shorter sound overlaps the other. Up to 600 s read from each side. A side with no sound in its range (no audio stream, a range past its end, or digital silence) fails with no_audio, naming which side. Short reads answer at once; longer ones return a task or job.
| Name | Required | Description | Default |
|---|---|---|---|
| a | Yes | The sound to line up against: the reference | |
| b | Yes | The sound whose offset you want | |
| idempotency_key | No | ||
| max_offset_seconds | Yes | Search offsets up to this far either way |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds substantial behavior beyond the annotations: the search constraint that at least half of the shorter sound must overlap, the 600 s read cap per side, the no_audio failure mode with the failing side named, and the short-read-vs-task/job response split. It also explains how to interpret strength vs next_best to detect ambiguous matches, which the annotations cannot convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the purpose and use cases, then moves through the offset semantics, strength interpretation, search limits, and failure modes in a logical order. It is dense and slightly long, but nearly every clause carries operational information; only the placement note about nudging b's clip is arguably extra.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a nested-object, async-capable tool with an output schema, the description covers inputs, sign conventions, search bounds, ambiguity detection, error behavior, and sync-vs-async returns. An agent has everything needed to invoke it and interpret the outcome.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 75%, and the description fills the important gaps: it defines offset_seconds sign convention (b's t aligns with a's t + offset_seconds), explains that each side may be media or timeline_id with its own from/until range, and clarifies what max_offset_seconds actually constrains. The idempotency_key parameter is left unexplained, but it is a conventional field.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: cross-correlate two recordings' sound to find the sync offset of b against a, plus a match strength. No sibling tool does this, and the description names the exact operation (cross-correlation) rather than a vague 'compare'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives three concrete use cases (camera with a separate mic, two cameras of the same moment, a re-recorded line against the original), which tells the agent when this is the right call. It stops short of naming alternatives or exclusion conditions, so it is clear context rather than full when/when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
media.deleteDelete mediaDestructiveIdempotentInspect
Delete a media for good, so an old or wrong version stops turning up: its file here, proxy, mezzanine and media.analyze results go, and its room comes back to this account’s storage. A path import leaves the original in its folder; exports made from it stay. Timelines that use it keep their ids and edits but no longer render (media_offline names it, as for expired media): replace those clips with clip.replace. Importing the same file again makes a new media_id.
| Name | Required | Description | Default |
|---|---|---|---|
| media_id | Yes | Media handle from media.import or a create.* tool |
Output Schema
| Name | Required | Description |
|---|---|---|
| deleted | Yes | |
| storage | Yes | |
| media_id | Yes | Media handle from media.import or a create.* tool |
| freed_bytes | Yes | What it held here: the file, its proxy and mezzanine, and its analyses |
media.getGet mediaRead-onlyIdempotentInspect
Status, where it came from (origin: the import source, the export of which timeline_id and preset, or the create.* tool and the media it was made from), a signed download url for the file (media is kept 7 days after last use: save the file from it to keep it, and media.import it again later) and a page_url that plays or shows it in any browser (to give a person), its format (extension and mime_type, read from the file, not its name), everything the probe read (probe: the format and every stream, e.g. pix_fmt, colour space, primaries, transfer), duration, frame rate, conform reasons, integrated loudness and summaries of media.analyze results under analyses (counts only: analyses.transcript.words is how many words). While it is importing or being made, job names the job doing it and its progress. include returns those results at the top level, e.g. include ["transcript"] gives transcript.words, the list of words with times. For long media, read them in pieces: from_seconds and to_seconds keep what overlaps that stretch of the media, offset and limit page through what is left, and pages says, per analysis, the index of the first item returned in the whole list (for transcript words, the index clip.add source {words} takes), how many matched and next_offset for the next page (null at the end).
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | At most this many items of each included list; pages says where the next starts | |
| offset | No | Skip this many items of each included list (after the time range) | |
| include | No | Also return these analyses (otherwise only their summaries): in full, or the piece from_seconds, to_seconds, offset and limit choose. One never run on this media is absent, and warnings says so | |
| media_id | Yes | Media handle from media.import or a create.* tool | |
| to_seconds | No | Only what starts before this media time | |
| from_seconds | No | Only what ends after this media time: transcript words, beats, speaker turns, frames |
Output Schema
| Name | Required | Description |
|---|---|---|
| fps | Yes | |
| job | No | media.get, while importing or being made: the job doing it |
| url | No | media.get: a signed download of the file itself, once it has one |
| kind | Yes | |
| look | No | |
| name | Yes | |
| beats | No | |
| error | No | |
| faces | No | |
| pages | No | media.get with a range, offset or limit: per included analysis, which piece of its list came back |
| probe | No | media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs |
| voice | No | |
| width | Yes | |
| codecs | Yes | |
| colour | No | |
| height | Yes | |
| origin | No | Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from |
| sounds | No | With include ["sounds"]: the latest media.analyze sounds run, every probability included |
| speech | No | With include ["speech"]: the speech probability of every 32 ms frame |
| status | Yes | importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed |
| upload | No | media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands |
| conform | Yes | Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed |
| picture | No | |
| analyses | No | Summaries of media.analyze results (counts, not data); include puts each in full at the top level |
| analysis | Yes | |
| media_id | Yes | Media handle from media.import or a create.* tool |
| page_url | No | media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at |
| speakers | No | |
| warnings | No | create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway |
| extension | No | The extension its format takes (.png, .mp4), from the file itself, not its name |
| has_audio | Yes | |
| has_video | Yes | |
| mime_type | No | Its MIME type, likewise |
| created_at | No | When it was imported or made |
| expires_at | No | When url and page_url stop working |
| transcript | No | With include ["transcript"]: every word as {text, start, end, confidence} |
| fingerprint | No | Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives |
| font_family | No | Fonts: the family; use font_media_id in a text style to set text in it |
| sound_seconds | No | How long the sound runs; under duration_seconds, the end is soundless |
| duration_seconds | Yes |
media.importImport mediaIdempotentInspect
Import the user's own video, audio, image or font to edit with or place as is: probe, fingerprint, conform if needed (VFR, rotation, HDR → BT.709), 540p proxy and integrated loudness. A reference image to recreate needs no import: rebuild it with create.motion or create.still. Long-running: returns a task, or {job_id, media_id} for job.get. The media_id is usable in edits as soon as the job reports processing, so with many files, start every import first and edit while they finish. For a file on the user's phone or computer, the easiest way in is media.upload: give the user its link; they open it on their phone or computer and pick the file, then import its upload_id. Sources: an http(s) URL (Dropbox and Google Drive share links work as they are), {upload: {name, bytes}} for a file you send yourself (the answer is the new media with upload.url: PUT the file there, curl --fail -T file '', and it imports once the last byte arrives, with no base64 in a message; a broken PUT resumes as media.upload says), the upload_id of a file sent through media.upload, a small file as a data: URI (up to 8 MB, with its sha256 so a garbled copy is refused), the job_id of a completed export.video (its render becomes media), or still: one frame of a video (media_id) or of an edit (timeline_id) as a PNG still at full size, to place as a picture, cut out or reuse. A URL is fetched once, when the import starts: it only has to work until then (a short-lived signed link from the user's own storage is enough), and the copy stays here 7 days after last use. A file the user attached in the chat comes as file, when the client passes it. Importing the same file again returns the same media_id.
| Name | Required | Description | Default |
|---|---|---|---|
| file | No | A file the user attached in the chat, as the client passes it (instead of source) | |
| name | No | ||
| source | No | Where the file is. Give this or file | |
| idempotency_key | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
media.listList mediaRead-onlyIdempotentInspect
What the user has here: imported and created media, and files they sent that aren't imported yet (through media.upload's link, or dropped on the cuttingfloor site), with names, sizes and when each expires, plus this account's storage used and how much it holds. When the user mentions a file they sent or dropped ("the interview I uploaded"), find it here by name and media.import {source: {upload_id}} it; complete says it is all in. Listing does not count as a use, so nothing here lasts longer for being listed.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| media | Yes | Imported and created media, most recently used first (up to 200) |
| storage | Yes | |
| uploads | Yes | Files sent to the server and not imported yet, newest first (up to 200): through media.upload, or dropped on the cuttingfloor site by the user |
| media_count | Yes | All media held, including any not listed |
| upload_count | Yes |
media.listenFind sounds by descriptionAIdempotentInspect
Find when something can be heard in a video or audio file, or in an edit as it plays, from what you describe in plain words, so you go straight to it instead of listening to it all: laughter, applause, a cough, a door slamming, a phone ringing, a dog barking, music starting, someone typing, a car passing. Name each sound and say what it is; you get back the stretches of time where each one can be heard, with the moment it is clearest. Use it to:
Find reactions and moments: where the crew or the audience laughs, where people clap or cheer, a gasp, a scream.
Find trouble to cut or cover: a cough, a sneeze, a phone going off, a door, a siren, wind on the microphone, a click or a bump.
Find sound to cut on: a clap, a bang, a hit, the music starting or stopping.
HOW IT WORKS. FlexSED, an open-vocabulary sound detection model (local, on the CPU; about 15 to 20 times faster than real time), listens to every 40 ms of the sound and gives each description the probability that it can be heard there. Describe the sound itself, briefly, as you would caption it ("laughter", "a crowd cheering", "glass breaking"), not the scene or a judgment; it is asked as "The sound of ". Short, common sound names work best; put each sound you want in its own entry, up to 16 a call, all listened for at once. LIMITS. Inside music it hears "music", not drums, instruments or single hits (for hits in music, media.analyze beats). A short sound can score higher over a short from_seconds to until_seconds around it. READING THE RESULT. matches gives, per sound, each run of frames at or above min_probability (default 0.5) with its start, end, peak_seconds and frame (for render.frames; media frames for video, timeline frames for an edit). peaks gives the highest probability each sound reaches, and scores every probability (step_seconds apart), for your own cut-off. A sound whose peak stays well under your cut-off is most likely not there; one hovering near it is worth a listen or a lower cut-off. Media or an edit: with media_id, times are the file's own seconds; with timeline_id, it listens to the edit's mix as it plays (cuts, gaps, levels and effects included), over from_seconds to until_seconds, and times are timeline seconds, so a match is where that sound now sits in the edit. A source or range with no sound fails with no_audio, naming it. For the 447 sounds with names (Laughter, Applause, Cough, Speech, Music…) as stored data, media.analyze kind sounds. Long-running: returns a task, or a job_id for job.get.
| Name | Required | Description | Default |
|---|---|---|---|
| sounds | Yes | Name (yours) to the sound in plain words: what it is, as you would caption it ("laughter", "a crowd cheering", "a door slams", "a phone ringing", "a dog barking", "keyboard typing"). Up to 16, each listened for in every 40 ms frame | |
| media_id | No | Listen to imported media (media seconds) | |
| timeline_id | No | Listen to an edit as it plays (timeline seconds): its mix, with cuts, gaps, levels and effects | |
| from_seconds | No | Start listening here (media or timeline seconds). Default 0 | |
| step_seconds | No | scores: one value per this many seconds (the highest in each). Default 0.04, every frame, coarser when that would be too many | |
| until_seconds | No | Stop here. Default: the end | |
| idempotency_key | No | ||
| min_probability | No | A frame matches a sound when its probability is at least this |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare non-readonly, idempotent, non-destructive; the description goes well beyond that with model identity, ~15-20x real-time CPU cost, 40 ms frame granularity, a 16-sound cap, long-running task/job_id behavior, and the no_audio failure mode. It does not mention the idempotency_key parameter or what a repeated call does, which is the one behavioral trait left uncovered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with a one-sentence purpose, then structured under Use it / HOW IT WORKS / LIMITS / READING THE RESULT / media-vs-edit headings, so an agent can skim to the section it needs. It is on the long side and the prose is dense (the examples list inside the first sentence runs nine items), but almost every sentence carries operational information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the hard parts an agent would otherwise get wrong: what units the returned times are in, how to interpret peaks versus matches, what a sub-threshold peak implies, and the failure case. With an output schema present it correctly does not re-document the return shape, but the unmentioned idempotency_key leaves a small hole for a long-running tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds real meaning beyond the 88%-covered schema: sounds must be short caption-style names in their own entries, up to 16, prompted as 'The sound of <your words>'; media_id versus timeline_id is explained in terms of which clock the returned times are on and that the timeline path listens to the rendered mix. min_probability and step_seconds are explained operationally, though idempotency_key gets nothing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource (find when a described sound can be heard in media or a timeline edit) and concretely enumerates the kind of targets (laughter, applause, a door slamming). It explicitly separates itself from media.analyze (beats inside music, the 447 named sounds) and points at render.frames for frame numbers.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives three explicit use-case families (reactions/moments, trouble to cut or cover, sound to cut on) and names the alternatives with the condition that selects them ('for hits in music, media.analyze beats'; 'for the 447 sounds with names… media.analyze kind sounds'). It also states how to phrase a query versus how not to ('not the scene or a judgment').
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
media.seekSeek by natural languageAIdempotentInspect
Find the moments in a video, or in an edit as it renders, that match what you describe in plain words, so you go straight to them instead of watching it all. Name each criterion and say what a matching frame shows; you get back the stretches of time where each one holds, with the time and picture of the best frame in each. Use it to:
Check quality: find frames that are too dark or blown out, out of focus, black or blank (a gap in an edit), with a microphone or crew in shot, with text cut off at the edge, or with a caption over a face. With timeline_id it checks the edit as it renders, titles, effects and gaps included.
Watch efficiently: looking through a whole video costs you a picture for every moment. Seek judges the frames for you and says where things are, so you look only there: where the demo starts, where the speaker holds up the product, where the second location begins.
Pick shots: B-roll or a thumbnail moment that fits a description ("a wide shot of a city street at night", "she smiles straight at the camera with her eyes open").
Find changes: with with_previous, where the slide changes, someone walks in or the camera angle switches ("
frameshows a different slide fromprevious").Find room or trouble for overlays: where the lower third stays empty for a title, or where a logo, a licence plate or a face appears that needs a blur.
HOW TO WRITE CRITERIA. TypeSafe Jev, a fast judgment model (preview), judges each sampled frame on its own, as a picture named frame (about 512 px on its long edge; with with_previous also previous, the frame sampled before), and gives each criterion the probability that it is true. A criterion is a statement or a question about how the picture looks. Keep one judgment to a criterion, and say what shows rather than a verdict: not "bad lighting" but "the faces are too dark to make out their eyes"; not "a good thumbnail" but "a person looks straight at the camera with eyes open and mouth closed". Criteria that name no picture or state entry in backticks are asked about frame ("In frame: "); name frame, previous or your state entries yourself to judge them together ("frame shows the button line talks about"). Jev answers most accurately from examples: give {instructions, criteria: {true: "…", false: "…"}} with a case of each ({true: "the face is a dark silhouette against a bright window", false: "the face is evenly lit, a little dim"}). Detail too small to see at that size (fine print, slight softness) answers poorly; check it with render.frames at full size.
READING THE RESULT. matches gives, per criterion, each run of consecutive samples at or above min_probability (default 0.5), with its start, end and best sample: peak_seconds, frame (for render.frames) and uri (a resource to read the picture). samples has every probability, for your own cut-off. A probability near 0.5 means Jev cannot tell, not that the thing is half there: reword a criterion that keeps landing there. asked shows the question each criterion became.
COST. Each frame is one judgment however many criteria it carries, so ask everything you are looking for in one call (up to 32 criteria). Work in passes: every_seconds 2 to 5 over the whole video, then at_seconds around a match to find more exactly where it starts. Sampling sees only the frames it samples, so no match can't prove a one-frame defect (a flash, a black or dropped frame) absent: read every frame of a suspect stretch with render.frames, or measure black frames with qa.check. Up to 600 frames a call. Long-running: returns a task, or a job_id for job.get.
media.analyze look asks typed questions (a choice, a score, a box) of every frame when you need measurements rather than moments; judge.ask judges one picture at once.
| Name | Required | Description | Default |
|---|---|---|---|
| state | No | Text or JSON entries shown beside `frame` on every frame, by name, for criteria to refer to (a script line, what a term means) | |
| criteria | Yes | Name (yours, not shown to the model) to what a matching frame shows: a statement or question about the picture, or {instructions, criteria: {true, false}} to say what counts as yes and as no. Up to 32, each judged on every frame | |
| media_id | No | Seek in imported media (media seconds) | |
| at_seconds | No | Sample exactly these times instead | |
| timeline_id | No | Seek in an edit as it renders (timeline seconds): gaps, cuts, titles and effects included | |
| from_seconds | No | Start sampling here. Default 0 | |
| every_seconds | No | Sample a frame this often. Default 1 | |
| until_seconds | No | Stop here. Default: the end | |
| with_previous | No | Also show the frame sampled before, as `previous`, for criteria about change. The first sample is only shown as `previous` | |
| idempotency_key | No | ||
| min_probability | No | A frame matches a criterion when its probability is at least this |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well past the annotations: discloses that each frame is one judgment regardless of criterion count, caps of 32 criteria and 600 frames, the recommended two-pass sampling strategy, and that the call is long-running and returns a task or job_id. It also warns that ~0.5 means the model cannot tell rather than half-true, which is behavior an agent cannot infer from any structured field.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose is front-loaded and the HOW TO WRITE CRITERIA / READING THE RESULT / COST headings make it scannable. The body is very long, though, with some lower-value sentences (the 'asked' field note, the model branding) that could be trimmed without loss.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite an output schema existing, the definition still explains matches, samples and asked, and covers cost, limits, sampling blind spots and failure modes for a complex nested-parameter tool. An agent has everything needed to write criteria and interpret a null result correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is already 91%, but the description still adds real meaning: the `frame`/`previous`/state-entry addressing convention, the {instructions, criteria:{true,false}} example form, how min_probability is applied, and how every_seconds versus at_seconds should be used across passes. That is interpretation guidance the schema does not carry.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb and resource: find moments in a video or a rendering edit that match a plain-language description, returning time ranges and the best frame. It explicitly separates itself from media.analyze look, judge.ask, render.frames and qa.check, so an agent can route without opening sibling schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
A bulleted 'Use it to' list gives five concrete situations (quality check, efficient review, shot picking, change detection, overlay clearance), each with example phrasing. It also states when NOT to rely on it (sampling cannot prove a one-frame defect; use render.frames or qa.check) and when a sibling is the better fit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
media.uploadUpload mediaInspect
A link to send one of the user's files to this server, for files on their phone or computer that the server can't read, including a file attached in the chat that the chat app doesn't hand you. Give the user the url: they open it on their phone or computer and pick the file; the page sends it and resumes on its own. Then media.import {source: {upload_id}}; media.import says whether the file is all in yet. The URL needs no other credentials and works for a day. If you can send the file yourself, PUT it there instead (curl --fail -T interview.mp4 ''). A broken upload resumes: HEAD the url for Upload-Offset (the bytes received), then PUT the rest with Content-Range: bytes -/. Large files can go in parts the same way.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | The file name with its extension, e.g. interview.mp4 | |
| bytes | No | The file size, to be refused now rather than part way if it is too big |
Output Schema
| Name | Required | Description |
|---|---|---|
| how | Yes | |
| url | Yes | Signed: PUT the file here; no other credentials needed |
| max_bytes | Yes | |
| upload_id | Yes | Upload handle from media.upload |
| expires_at | Yes |
qa.checkQA checkAIdempotentInspect
Measure the rendered timeline: gaps in the picture, offline media, rendered against expected frame count, integrated loudness (LUFS) and true peak (dBTP), plus black and silent stretches when you give their settings. With speech {min_probability}, every cut is listed with speech facts: the speech probability of the mix just before and after it (Silero VAD), whether it lands while someone is speaking, the seconds without speech each side, and, per clip edge, the transcript word it cuts through and how much of that word is lost (from the media's transcript, when it has one). Facts only, no verdicts. A timeline up to 20 s is checked within the call (which waits its turn for a render when the server is busy); longer ones return a task or job.
| Name | Required | Description | Default |
|---|---|---|---|
| black | No | Find black stretches with these settings; left out, none are looked for | |
| range | No | ||
| speech | No | List every cut with speech facts (Silero VAD on the mix, and the transcript word each clip edge cuts through); left out, cuts are not listed. Only junctions between two clips are listed, so a lone clip reports none and its own head and tail are never examined. A null word means the edge falls between words: that is a clean cut, not a missing reading | |
| silence | No | Find silent stretches in the mix with these settings; left out, none are looked for | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotent, non-destructive, non-read-only behavior, and the description adds genuinely new operational context: timelines up to 20 s are checked inline and may wait for a render when the server is busy, while longer ones return a task or job. It also clarifies that only junctions between two clips are examined and that a null word means a clean cut, not a missing reading. It does not discuss cost, permissions, or failure modes, so a 4 rather than 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core measurement purpose and then layers the optional analyses, with no filler sentences. It is dense and long-sentenced, which slightly hurts scannability, but every clause carries information an agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, the description need not explain return values, and it correctly focuses on inputs, opt-in behavior, and the sync-vs-async execution model. The main remaining gap is that the 'range' parameter's effect on measurement scope is never described, which matters for a measurement tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 80%, so the schema already documents most parameters, and the description largely restates the opt-in semantics of black/silence/speech rather than adding syntax or format detail. It does add real meaning for the speech block (Silero VAD on the mix, per-clip-edge transcript word, null-word interpretation), but the range and timeline_id parameters get nothing beyond the schema. Baseline 3 is appropriate given the high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource ('Measure the rendered timeline') and enumerates the exact metrics produced: gaps, offline media, frame count, LUFS, true peak, black and silent stretches, and per-cut speech facts. This is clearly distinguishable from siblings like media.analyze or timeline.diff, which do not measure rendered output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It states the conditional activation of each optional analysis ('when you give their settings', 'left out, none are looked for', 'left out, cuts are not listed'), which tells the agent exactly how to opt in. It also explains the short-vs-long timeline execution path. It does not name an alternative tool or say when not to use this one, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
range.extractExtract rangeAIdempotentInspect
Remove everything in a frame range on the given tracks (default all unlocked, so a picture range takes the sound under it too: lock a track with track.set or name track_ids to keep it) and close the gap. For cutting out silences, filler words or mistakes across all tracks at once. changed lists each clip removed or cut, with its track; a warning says when sound went with the picture. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| range | Yes | Timeline frames, end exclusive | |
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| track_ids | No | Tracks to act on. Default: every unlocked track | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With annotations already covering the safety profile, the description still adds real behavioral detail: the surprising default that all unlocked tracks are affected (so picture edits take sound with them), the warning emitted when sound went with picture, and the contents of `changed`. That is exactly the kind of side-effect disclosure annotations cannot carry.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the action and its scope before the parenthetical caveat, and every clause carries information. The long parenthetical is dense but earns its place by documenting the tracking default.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers scope, default behavior, side effects, and return contents, and defers correctly to the server instructions for the edit result since an output schema exists. Nothing needed to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description goes beyond it by explaining the operational consequence of track_ids' default (locking a track keeps it intact), which the schema only states as 'every unlocked track'. Range and preview semantics are left to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Remove everything in a frame range on the given tracks ... and close the gap'), with the gap-closing behavior making it distinguishable from a lift-style sibling. It never names range.lift explicitly, so the differentiation is implied rather than stated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a concrete use case ('cutting out silences, filler words or mistakes across all tracks at once') and explains the default scope plus how to narrow it (lock a track with track.set or name track_ids). It stops short of naming the alternative tool or stating when not to use this one.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
range.liftLift rangeAIdempotentInspect
Remove everything in a frame range on the given tracks (default all unlocked, so a picture range takes the sound under it too: lock a track with track.set or name track_ids to keep it), leaving a gap. changed lists each clip removed or cut, with its track; a warning says when sound went with the picture. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| range | Yes | Timeline frames, end exclusive | |
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| track_ids | No | Tracks to act on. Default: every unlocked track | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds real context beyond annotations: the gap is left (not closed), clips may be cut as well as removed, changed enumerates each clip with its track, and a warning fires when audio went with the picture. That is useful disclosure. However, the annotation destructiveHint=false is arguably in tension with a tool that permanently removes clips – the description doesn't clarify whether this is undoable, which is exactly the question an agent would have.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action and scope, then default behavior, then return/warning behavior. It is one dense sentence chain with parenthetical nesting; slightly awkward to parse but every clause carries information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the default track scope, the locking alternative, what gets reported in changed, and the audio-with-picture warning; output schema handles return shape. What's missing is any statement about reversibility/undo, which matters most for a destructive-feeling range operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so range, track_ids, preview and timeline_id are all documented in the schema. The description adds the meaningful default semantics for track_ids (all unlocked) and the interaction with track locking, but says nothing extra about preview or the end-exclusive range. Baseline 3 with a modest lift for the default-scope explanation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource: remove everything in a frame range on given tracks, leaving a gap. Distinguishes clearly from siblings clip.remove (single clip) and range.extract (which closes the gap), and the 'leaving a gap' phrase pins down the semantic difference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains the default scope (all unlocked tracks), why that matters (a picture range also takes the sound under it), and how to narrow it (lock a track with track.set, or name track_ids). It doesn't explicitly name range.extract or clip.remove as alternatives, but the when-to-use guidance for track scoping is concrete.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
render.framesRender framesARead-onlyIdempotentInspect
Look at a timeline (or a media file): up to 16 frames as one contact-sheet image (layout grid) or one image each (single). Media frames are numbered at the media frame rate; a picture that can be see-through (alpha or a palette) is shown over a light checkerboard, so transparency shows as checks and a background baked into the file as itself. size full: up to 4 frames at the sequence (or media) size from the original media, for thumbnails and stills to keep. format webp (default), png or jpeg, with quality for webp and jpeg. Each image comes inline and with a signed download url.
| Name | Required | Description | Default |
|---|---|---|---|
| size | No | full: the sequence size from original media (long_edge ignored) | preview |
| format | No | jpeg: for places that take nothing else; png: lossless | webp |
| frames | Yes | Timeline frames, or media frames at the media rate | |
| layout | No | grid: one contact sheet | grid |
| quality | No | WebP and JPEG, 1–100 (default 82 WebP, 85 JPEG) | |
| media_id | No | Media handle from media.import or a create.* tool | |
| long_edge | No | ||
| timeline_id | No | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| images | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With annotations already declaring read-only, idempotent, non-destructive behavior, the description adds useful rendering behavior beyond the annotations: media frame numbering, transparency checkerboard display, full-size limits, and inline images plus signed download URLs. It does not discuss authentication, errors, or rate limits, but it covers the core behavioral traits an agent needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single dense paragraph with the main action front-loaded and no obvious filler. It is appropriately sized for the tool’s complexity, though some semicolon-heavy clauses reduce readability slightly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema and safety annotations, the description covers purpose, frame limits, layout behavior, transparency handling, output formats, and delivery mechanism. The main remaining gap is the absence of any explanation for long_edge, and the timeline/media exclusivity is only implied.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is high at 88%, establishing a baseline of 3, and the description adds meaningful semantics: the 16-frame cap, grid versus single layout, full-size limited to 4 frames at original media size, format/quality behavior, and media-frame numbering. It does not address the long_edge parameter, which appears to be the schema’s undocumented parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that it renders up to 16 frame images from a timeline or media file, either as one contact-sheet grid or as individual images. It is specific about the output form and limits, but it does not explicitly name or differentiate itself from sibling tools such as export.video or create.still.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear usage context: look at a timeline or media file, use grid for a contact sheet, single for individual images, and size full for thumbnails and stills to keep. It does not state when not to use it or name alternative tools, so it stops short of explicit routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
skills.getRead a skillARead-onlyIdempotentInspect
The whole skill as Markdown: when to use it, what it needs, numbered Steps naming the tools to call, a decision table and the rules to keep. Follow the Steps in order.
| Name | Required | Description | Default |
|---|---|---|---|
| slug | Yes | From skills.search, e.g. "cut-pauses-ums-and-restarts" |
Output Schema
| Name | Required | Description |
|---|---|---|
| name | Yes | |
| slug | Yes | |
| author | Yes | Who wrote the skill; credit them when you name it |
| related | Yes | |
| version | Yes | |
| markdown | Yes | The whole skill, frontmatter included: follow its Steps |
| author_url | Yes | A link to the author or the work the skill came from; null when there is none |
| description | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive and closed-world, so the safety profile is covered. The description adds valuable behavior beyond that: the payload is a full Markdown procedure document with an ordered Steps list intended to be executed sequentially, which is not derivable from the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the return payload and the document structure, ending with the imperative instruction. No filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, the description does not need to restate return values, and it still usefully outlines the document's anatomy and the sequential-execution expectation. The only gap is any mention of what happens when the slug does not match an existing skill.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the slug property already documents its format and its origin ('From skills.search, e.g. ...'). The description adds no further meaning about slug resolution or behavior on an unknown slug, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific resource and what it returns: the whole skill as Markdown, enumerated as when-to-use, requirements, numbered Steps, decision table and rules. That is clearly distinct from siblings like skills.search, although the contrast is only implied by the schema's 'From skills.search' note rather than stated in the description itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Follow the Steps in order' tells the agent how to act on the result, which is useful usage guidance for a procedure document. It does not say when to call this versus skills.search, nor any precondition such as resolving a slug first — that routing only appears inside the parameter schema.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
skills.searchFind a skillARead-onlyIdempotentInspect
Find step-by-step instructions for a common job, written for you to follow with these tools: cutting the pauses, ums and restarts out of a talking video, for one. Search in the user's words; every skill comes back best first with its score (roughly 0 to 1). Below 0.1 is unrelated; a real fit usually scores 0.15 to 0.45, lower for a short or general ask, and an unrelated skill that shares words can score as high, so read the description of each one above 0.1 to judge the fit. Read one with skills.get, then follow its Steps.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Most matches to return, best first | |
| query | Yes | What the user wants done, in their words, e.g. "remove the ums from my video" |
Output Schema
| Name | Required | Description |
|---|---|---|
| skills | Yes | Best first; empty when nothing fits |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnly, idempotent, closed-world), so the remaining burden is output semantics — and the description does add real value there: relevance scores on roughly a 0–1 scale, best-first ordering, and a 0.1 unrelated threshold. What it omits is the practical side of the tool: whether search is exhaustive, how the index is maintained, or what happens with a 500-char query, though the score calibration note partially compensates.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The core guidance is front-loaded and every clause is functional, but the opening sentence is a run-on whose illustrative rider ('cutting the pauses, ums and restarts out of a talking video, for one') reads as filler. The score-calibration sentence is long and comma-chained, making the thresholds harder to scan than they need to be.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return-shape explanation is rightly omitted, and the description covers the one thing the schema can't: how to interpret the relevance score and what to do next. The only gap is that it never tells the agent when to raise or lower the limit, which is the sole non-obvious parameter choice left.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and both parameter descriptions already state 'in their words' phrasing and 'most matches to return, best first', so the description largely restates the schema. It adds no format, syntax, or limit-tuning guidance beyond what the schema provides; baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a concrete verb and resource ('Find step-by-step instructions for a common job') and anchors it with a concrete worked example (removing ums/pauses from a talking video). It also implicitly separates itself from skills.get, which it names as the follow-up read step, so an agent can tell search from fetch without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says to search in the user's words and prescribes the full workflow: search, triage results by score, read candidates with skills.get, then follow the skill's Steps. It even warns against a specific failure mode (keyword-sharing false positives scoring as high as real fits), which is genuine when/when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
text.setSet textAIdempotentInspect
Change a title's or caption's text, or restyle many at once (clip_ids, or every one on track_id). The style is merged; null turns an outline, shadow, box or highlight off (and max_lines back to all lines). spans (one clip) restyle part of its text. Fonts: video://catalog/fonts, or an unknown font is refused with the close matches. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| text | No | Only with a single clip | |
| spans | No | Only with a single clip: replaces its spans (null removes them). New text drops spans it doesn't give | |
| style | No | Merged into each clip's style; null fields reset | |
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| clip_ids | No | Titles or captions | |
| track_id | No | Instead of clip_ids: every title and caption on a track | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotent and non-destructive, and the description adds the merge semantics that make idempotency meaningful, plus the negative-path behavior: an unknown font is refused with close matches. It omits destructive-scope detail (e.g., what a new text string does to existing spans beyond the schema note) but overall adds real behavioral signal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the primary action, then progressively narrower modes. Dense with parenthetical clauses, but each clause carries a distinct constraint; no filler sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists and the description defers detail to server instructions for the return shape, so return values needn't be restated. For a 7-param, nested-style mutation the coverage is solid, though the relationship to title.add/caption.add remains unstated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3; the description exceeds it by explaining null-as-reset for style fields and max_lines, the single-clip restriction on text and spans, and the font catalog reference. It leaves other TextStyle fields (reveal, anchor, max_width) to the schema, which already documents them.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb+resource: 'Change a title's or caption's text, or restyle many at once', with the batch vs. single distinction made explicit via clip_ids/track_id. It also carves out the spans sub-case. It stops short of naming a sibling (e.g., title.add/caption.add for creation), so an agent must infer that boundary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear operating conditions: style is merged rather than replaced, null is the removal mechanism for outline/shadow/box/highlight and max_lines, and spans applies only with a single clip. No explicit when-not or alternative-tool routing, but the mode selection (single text vs. clip_ids vs. track_id) is spelled out.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
timeline.createCreate timelineAIdempotentInspect
Start an empty timeline. Returns its timeline_id and its tracks (ids for track_id). Timelines are immutable: every edit returns a new timeline_id, and older ids stay valid for 24 hours after last use. For work longer than a day, keep the timeline.get document on your side and restore it with timeline.load.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Untitled | |
| tracks | No | How many empty tracks to start with | |
| sequence | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| tracks | Yes | The new tracks; edits take the id (track_id), not the name |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well past the annotations by disclosing immutability (every edit yields a new timeline_id), the 24-hour retention window after last use, and what identifiers the call returns. These are exactly the lifecycle facts an agent needs and none are in the structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four short sentences, each doing distinct work: purpose, return values, lifecycle semantics, and long-running-work guidance. Purpose is front-loaded with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lifecycle and return behavior are well covered, and the output schema handles return shape, but for a tool whose only required argument is a nested sequence spec the description leaves that side entirely unexplained. Adequate for calling it, thin for calling it correctly on first try.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 33%, and the description says nothing about any parameter — not the required nested `sequence` (width/height/fps), nor `name`, nor how `tracks` counts map to the returned track ids. The heaviest documentation burden falls on the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Start an empty timeline") and immediately names the sibling interactions it pairs with (timeline.get, timeline.load), so an agent can distinguish it from the many clip/track tools in the sibling list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a concrete routing rule: for work longer than a day, persist the timeline.get document and restore via timeline.load. That effectively covers when to prefer an alternative workflow, though it does not say when not to create a fresh timeline at all.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
timeline.diffDiff timelinesBRead-onlyIdempotentInspect
Which clips, tracks and settings differ between two timelines.
| Name | Required | Description | Default |
|---|---|---|---|
| to | Yes | Timeline handle; every edit returns a new one | |
| from | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| changed | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive and closed-world, so the safety profile is fully covered. The description adds the scope of the comparison (clips, tracks, settings) but says nothing about directionality from→to, ordering of results, or how large diffs are represented.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tight fragment with no filler. It would be marginally stronger if it led with a verb, but every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need no explanation, and annotations cover the safety profile. Still, for a diff tool the description leaves the comparison semantics (direction, what counts as a 'settings' difference) implicit, which is the minimum viable rather than complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%: both from and to are documented with the tl_<32 hex> pattern and the note that every edit yields a new handle. The description's "between two timelines" implies the pairing but adds no syntax or ordering semantics beyond the schema, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States the resource pair (two timelines) and the exact output dimension (clips, tracks, settings that differ), which cleanly separates it from timeline.get and timeline.history. It is a sentence fragment with no explicit verb, but the meaning is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to use this versus timeline.get, timeline.history, or edit.batch. An agent must infer the comparison scenario entirely from the one-line description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
timeline.getGet timelineARead-onlyIdempotentInspect
The full timeline: tracks, clips (frames, end exclusive), link groups and the media it uses. The server keeps a timeline 24 hours after last use: for work longer than a day, save this document on your side (it is the edit) and restore it with timeline.load.
| Name | Required | Description | Default |
|---|---|---|---|
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| name | Yes | |
| media | Yes | |
| master | No | |
| nested | No | Timelines nested as compound clips, at any depth: each has this shape without timeline_id, parent or nested |
| parent | Yes | |
| tracks | Yes | Video tracks bottom to top, then audio tracks |
| effects | No | Effects on the whole picture and mix |
| duration | Yes | Timeline length in frames |
| sequence | Yes | |
| timeline_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations that already mark this as a safe read, the description discloses a critical lifecycle behavior: timelines are retained for 24 hours after last use and must be saved externally for longer work. It also clarifies that frame end is exclusive, adding useful semantic detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two tight sentences with no wasted language. It front-loads what the timeline document contains and then adds the retention/restoration warning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema, full parameter schema coverage, and annotations covering safety, the description is nearly complete. Its main omission is explicit differentiation from sibling timeline inspection tools, though the retrieval scope is still clear enough to call correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the sole timeline_id parameter is fully documented in the schema as a handle returned by edits. The description adds no parameter-specific detail beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that this retrieves the full timeline, including tracks, clips, link groups, and media. It is specific about the returned document, but it does not explicitly distinguish itself from sibling timeline tools such as timeline.diff or timeline.history.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies retrieval use and gives important persistence guidance, including saving externally and restoring with timeline.load. However, it does not explicitly say when to use timeline.get instead of other timeline inspection tools like timeline.diff or timeline.history.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
timeline.historyTimeline historyARead-onlyIdempotentInspect
The chain of edits that made this timeline, newest first. To go back, keep using an older timeline_id.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| entries | Yes | |
| truncated | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive and closed-world, so the safety profile is covered. The description adds real context beyond them: results come newest first, and each entry corresponds to an earlier timeline_id, which explains the versioning model an agent needs to paginate backwards. Pagination limits are left to the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, ordering stated up front and the traversal tip second. No filler; every clause carries information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return shape need not be explained, and annotations cover safety. Still missing are the limit semantics and any guidance about how deep the chain can go or when this is preferable to timeline.diff — gaps that matter for an agent deciding between several timeline-reading siblings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50%: timeline_id is documented ('every edit returns a new one') while limit is not. The description reinforces the timeline_id semantics ('older timeline_id' walks back in time), but says nothing about the limit parameter's role or cap, so it only marginally exceeds the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific resource and verb-ish action: the edit chain of a timeline, returned newest first. That distinguishes it from timeline.get (current state) implicitly, though it never names a sibling to sharpen the contrast.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The sentence 'To go back, keep using an older timeline_id' hints at the traversal workflow, but there is no explicit guidance on when to call this versus timeline.get, timeline.diff, or timeline.load, and no exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
timeline.loadLoad timelineAIdempotentInspect
Restore a timeline from the document timeline.get returned, e.g. after it expired (24 h after last use). Its media must still be here: media expires 7 days after last use, so import saved files again with media.import and swap them in with clip.replace.
| Name | Required | Description | Default |
|---|---|---|---|
| timeline | Yes | A timeline as returned by timeline.get |
Output Schema
| Name | Required | Description |
|---|---|---|
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare it non-read-only, idempotent, non-destructive, and closed-world. The description adds important expiration semantics for both timelines and media, though it does not explain whether loading replaces the active timeline or what happens on conflicts.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tightly written sentences with the restore purpose and expiration caveat front-loaded. Every sentence adds operational value with no wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the rich input schema, output schema, and annotations, the description is complete enough to call correctly. It covers the key operational failure mode: expired media requiring re-import and clip replacement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The sole parameter has 100% schema coverage, so the baseline is 3. The description adds meaningful provenance and a precondition beyond the schema by saying it must be a document previously returned by timeline.get and that referenced media must still exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb, 'Restore', and resource, 'timeline', and identifies the source document as the one returned by timeline.get. This clearly distinguishes it from timeline.create and timeline.get without needing the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use it: after a timeline expires 24 hours after last use. It also gives the fallback workflow when media has expired, naming media.import and clip.replace as alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
timeline.set_sequenceSet sequenceAIdempotentInspect
Change resolution, frame rate or audio format. Clip times are kept in time, so they survive a rate change. Each picture is refitted to the new shape by its fit (contain by default, which leaves bars where the shapes differ); the result warns how many clips no longer fill the frame, and clip.set_transform {fit: "cover"} fills it. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| sequence | Yes | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, it discloses important side effects: clip times survive rate changes, pictures are refitted according to their fit with contain as the default, and the result warns how many clips no longer fill the frame. It also explains the relevant follow-up action and points to server instructions for the edit result.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the main action, then adds only necessary behavioral details and a concrete alternative. Every sentence contributes information needed to invoke the tool correctly, with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the nested schema, annotations, and existing output schema, the description supplies the key behavioral context an agent needs: what changes, how clip timing and fitting are affected, what warning to expect, and how to resolve framing issues. The return value is appropriately delegated to the server instructions and output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description maps the core sequence object to resolution, frame rate, and audio format, which adds useful meaning beyond the schema. It does not explain the preview parameter, the fps numerator/denominator structure, or optional fields like channels and color, leaving some parameter semantics to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: change resolution, frame rate, or audio format. It is clearly distinguishable from sibling tools such as clip.set_transform, which is explicitly named as the alternative for filling the frame after a sequence change.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear context for when to use the tool and names clip.set_transform {fit: "cover"} as the follow-up alternative when clips no longer fill the frame. It does not explicitly state when not to use it or contrast it with timeline.create or timeline.load, so it falls short of full when/when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
title.addAdd titleAIdempotentInspect
Text on a video track, over the tracks below: styled with font, size, colour, outline, shadow and box, placed by x/y, revealed with a typewriter or word animation. A style field you leave out takes the published default (video://catalog/text: bold white Inter at 9% of the frame height, centred, with a soft shadow); timeline.get shows the full style. spans style part of the text (words first_word to last_word, from 0: one word in yellow or bold). It is a clip, so transforms, masks, animations and transitions apply. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Line breaks with \n | |
| spans | No | Style part of the text: each span restyles words first_word to last_word (from 0: the timed words, or the text split at spaces), e.g. one word in yellow | |
| style | No | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| at_frame | No | Default: after the last clip on the track | |
| track_id | Yes | A video track; text composites over the tracks below | |
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| duration_frames | No | Default 3 s |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare this is a non-read-only, non-destructive, idempotent mutation. The description adds useful behavioral context beyond the annotations: omitted style fields use published defaults, timeline.get reveals the full style, spans can restyle word ranges, and the tool returns an edit result. No annotation contradiction is present.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the purpose and core behavior, and most clauses carry useful information for a complex 8-parameter tool. It is dense and somewhat run-on, but not bloated with repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex nested-style clip creation tool with an output schema and detailed annotations, the description covers the important operational context: defaults, span styling, clip semantics, and the edit result. The main remaining gap is explicit sibling routing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is high at 88%, so the baseline is 3. The description adds meaning beyond the schema by explaining default style behavior, span word indexing/ranges, text positioning, reveal animations, and that style fields default when omitted.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb and resource: adding text/title as a clip on a video track. It explains compositing over lower tracks and that transforms, masks, animations, and transitions apply. However, it does not explicitly distinguish itself from relevant siblings such as text.set or caption.add.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by phrases like 'Text on a video track' and the explanation of default placement, but there is no explicit guidance on when to use this tool versus text.set, caption.add, or clip.add. The description provides context but leaves alternative selection to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
track.addAdd trackAIdempotentInspect
Add an empty video or audio track. New video tracks go on top. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| kind | Yes | ||
| name | No | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish that this is a non-destructive, idempotent mutation, so the description needn't repeat that. It adds genuinely useful behavior beyond annotations: new video tracks go on top, and the result is an edit result documented in server instructions. It leaves open what happens with audio track ordering and whether an identical track is deduplicated despite idempotentHint=true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, front-loaded with the core action and with zero filler. The placement detail and return-value pointer each earn their place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists and the description appropriately points to server instructions for the edit result rather than duplicating it. For a simple add-track mutation with annotations covering safety and idempotency, the main remaining gap is track ordering or naming behavior for audio tracks, which is not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50%, with the schema itself documenting preview and timeline_id. The description adds meaning for kind (video or audio) and placement of video tracks, but does not explain the name parameter, defaults, or how preview affects the returned edit result.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ("Add an empty video or audio track"), and the word "empty" distinguishes it from clip-adding siblings like clip.add. It does not explicitly contrast with track.remove or track.set, but the action is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by "empty" and the video/audio kind, suggesting this is for creating a track before adding clips. However, it never explicitly says when to choose this over related tools such as timeline.create or clip.add, nor does it state prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
track.removeRemove trackBIdempotentInspect
Remove a track and its clips. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| track_id | Yes | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare destructiveHint=false and idempotentHint=true, so the description's statement that clips are removed is genuinely useful context that the annotations do not convey. However, it does not explain why a clip-deleting cascade is flagged non-destructive (presumably because every edit returns a new timeline handle and is undoable) or whether the removal is reversible — the single most important thing an agent needs before calling this.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the action and its blast radius. "(see the server instructions)" is a deferral that forces an extra lookup, which is the only real waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema and annotations exist, so return values need not be spelled out. Still missing for a mutation tool: reversibility/undo semantics, the expected track_id format, and whether track_id must belong to the given timeline_id.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67%; timeline_id and preview are documented in the schema, but track_id has no description anywhere. The description adds nothing about parameters, so it neither compensates for the track_id gap nor explains the preview option's relevance to a removal call (baseline 3 at this coverage level).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Remove a track") and adds the important scope detail that its clips go with it, which distinguishes it from clip.remove and clip.link. No explicit sibling routing, but the resource is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this versus track.set, clip.remove, or timeline-level edits. The agent must infer that this is the whole-track deletion path from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
track.setSet trackBIdempotentInspect
Rename, mute (audio), hide (video) or lock a track. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| muted | No | ||
| hidden | No | ||
| locked | No | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| track_id | Yes | ||
| timeline_id | Yes | Timeline handle; every edit returns a new one |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds the useful domain nuance that muted applies to audio and hidden to video tracks, but it says nothing about what the preview default produces or what changes on each edit. The pointer to 'server instructions' is a hand-wave rather than disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the capability list, no padding. The second sentence is low-value since the output schema already covers the return value and the server-instructions pointer is vague, but the overall size is appropriate.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return values need not be spelled out, and annotations cover safety. Still, for a 7-parameter mutation with 29% schema coverage and no usage routing, the description should say more about required identifiers and the preview behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 29%, and the description names four of the seven parameters (name, muted, hidden, locked) with the audio/video qualifier that gives them extra meaning. However, it says nothing about track_id, timeline_id, or the preview enum, leaving a meaningful gap the schema only partly fills.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Names concrete verbs on a specific resource: rename, mute (audio), hide (video), lock a track. That is far more informative than the title 'Set track' and lets an agent distinguish it from track.add and track.remove. It stops short of an explicit statement that only track attributes are modified, not the track's position or existence.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this versus siblings like track.add, track.remove, timeline.set_sequence, or text.set. The only usage-adjacent text is 'Returns the edit result (see the server instructions)', which defers rather than instructs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
transition.addAdd transitionAIdempotentInspect
Transition at the cut after a clip, into the clip that touches it (video://catalog/transitions). Linked audio gets a crossfade at the same cut. kind crossfade is sound only: on a video clip it crossfades the linked audio and the picture cuts hard (use mode handles for that). mode overlap (default) overlaps the clips and moves later clips earlier; handles uses source past the cut and moves nothing. changed lists each transition added, with its track and kind. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| kind | Yes | video://catalog/transitions; crossfade on audio tracks | |
| mode | No | overlap: the clips overlap by the duration and later clips move earlier; handles: use source past the cut, nothing moves | overlap |
| clip_id | Yes | The clip before the cut; the clip after it must touch it | |
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| other_tracks | No | overlap only: what the other tracks do. stay: they don't move, so what sits after the cut there plays later against the picture (the result warns which clips); extract: the same span comes out of them too, like range.extract, so everything stays in sync | stay |
| duration_frames | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare non-readOnly, non-destructive, idempotent, and closed-world behavior. The description adds substantial context beyond that: overlap moves later clips earlier, handles moves nothing, other_tracks stay can create timing drift and warnings, extract keeps tracks in sync, and changed/return behavior is described. This is rich behavioral disclosure for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but front-loaded, starting with the core operation and then moving into mode and audio behavior. Most sentences earn their place by clarifying non-obvious effects. There is some repetition of schema-provided enum and URI details, which keeps it from being maximally concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex timeline-editing tool, the description covers purpose, mode effects, linked-audio behavior, changed-output semantics, and a pointer to the server instructions for the edit result. An output schema exists, and the input schema covers most parameters, so the remaining gaps are minor. The main missing element is explicit sibling-tool routing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is high at 86%, so the schema already documents most parameters. The description still adds cross-parameter meaning, especially that kind crossfade is sound-only on video clips and requires mode handles for a visual transition. It does not cover every parameter in prose, but the added interaction guidance goes beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: it adds a transition at the cut after a clip and into the touching clip. It also explains linked-audio behavior, making the operation clear. It does not explicitly distinguish itself from sibling tools such as transition.set or transition.remove.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives useful internal guidance for choosing kind crossfade versus mode handles, and explains overlap versus handles. However, it does not say when to use transition.add instead of transition.set, transition.remove, or audio.fade. Usage is implied by the cut/clip context rather than explicitly routed against alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
transition.removeRemove transitionBIdempotentInspect
Remove a transition; the cut becomes a hard cut. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| transition_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, destructiveHint=false and idempotentHint=true, so much of the safety profile is covered; the description adds real value by disclosing the outcome (the cut becomes a hard cut) and pointing at the server instructions for the edit result. It does not say what happens when the transition_id is absent or already removed, which is the key ambiguity for an idempotent remove.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short clauses, front-loaded with the action and its consequence, with no filler. Every phrase carries information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, the description correctly defers return-value detail to the server instructions rather than duplicating it, and annotations cover the safety profile. The only gap is edge-case behavior (no-op vs error on a missing transition) for what is otherwise a simple, single-purpose tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67%: timeline_id and preview are documented in the schema, but transition_id has no description anywhere, and the description only implies that it selects the transition to remove. Baseline 3 applies since the schema does most of the work and the description adds nothing parameter-specific.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Remove a transition") plus the resulting state ("; the cut becomes a hard cut"), which pins down exactly what disappears versus transition.set/transition.add. It stops short of naming a sibling alternative, so it is clear but not fully differentiated by text alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit when-to-use or when-not-to-use guidance, and the description never contrasts this with transition.set (modify a transition) or transition.add. The agent must infer from the tool name that this is the undo path for an existing transition.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
transition.setSet transitionBIdempotentInspect
Change a transition's kind or duration. Returns the edit result (see the server instructions).
| Name | Required | Description | Default |
|---|---|---|---|
| kind | No | ||
| preview | No | inline: image in the result; link: URI that renders on read; none | link |
| timeline_id | Yes | Timeline handle; every edit returns a new one | |
| transition_id | Yes | ||
| duration_frames | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| parent | Yes | Timeline handle; every edit returns a new one |
| changed | Yes | |
| clip_id | No | |
| preview | Yes | |
| clip_ids | No | |
| track_id | No | |
| warnings | Yes | |
| timeline_id | Yes | Timeline handle; every edit returns a new one |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the mutation profile (readOnlyHint=false, idempotentHint=true, destructiveHint=false), so the safety bar is lower. The description adds only the pointer to the server instructions for the edit result and says nothing about what happens if kind or duration is omitted, or about the non-obvious fact that every edit mints a new timeline handle (which sits in the schema, not the description).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with the action front-loaded and no padding. The second sentence is efficient but defers real information to an external document.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained, and annotations cover the safety profile of this mutation. For a five-parameter field-update tool with a defined result schema, the description supplies what is minimally required to invoke it correctly, though it is thin on optional-parameter behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 40%, so the schema leaves gaps. The description names two of the five parameters (kind, duration) but never mentions preview, transition_id, or timeline_id semantics, and adds no format or constraint detail beyond the field names already implied by the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: change a transition's kind or duration. An agent can tell this edits an existing transition rather than adding (transition.add) or removing (transition.remove) one, though the distinction is inferred from the verb rather than stated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not guidance and no named alternative. The only signal is the implied notion that the transition must already exist, since the operation is a 'change' keyed on transition_id.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
13 tool updates
- Changed
create.cutout1 field changed- changed
Output schema / anyOfPrevious value: -[ - { - "properties": { - "analyses": { - "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", - "properties": { - "beats": { - "properties": { - "beats": { - "type": "number" - }, - "bpm": { - "type": "number" - } - }, - "required": [ - "bpm", - "beats" - ], - "type": "object" - }, - "colour": { - "properties": { - "frames": { - "type": "number" - }, - "regions": { - "items": { - "type": "string" - }, - "type": "array" - } - }, - "required": [ - "frames", - "regions" - ], - "type": "object" - }, - "faces": { - "properties": { - "frames_with_faces": { - "type": "number" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames_with_faces" - ], - "type": "object" - }, - "look": { - "description": "frames and questions: the latest run", - "properties": { - "frames": { - "type": "number" - }, - "questions": { - "items": { - "type": "string" - }, - "type": "array" - }, - "runs": { - "type": "number" - } - }, - "required": [ - "runs", - "frames", - "questions" - ], - "type": "object" - }, - "picture": { - "properties": { - "frames": { - "type": "number" - }, - "from": { - "type": "number" - } - }, - "required": [ - "frames", - "from" - ], - "type": "object" - }, - "sounds": { - "description": "heard: the sounds the latest run found at or above its min_probability", - "properties": { - "heard": { - "items": { - "type": "string" - }, - "type": "array" - }, - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "min_probability", - "heard" - ], - "type": "object" - }, - "speakers": { - "properties": { - "model": { - "type": "string" - }, - "speakers": { - "type": "number" - }, - "turns": { - "type": "number" - } - }, - "required": [ - "model", - "speakers", - "turns" - ], - "type": "object" - }, - "speech": { - "properties": { - "frame_seconds": { - "type": "number" - }, - "frames": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "frames", - "frame_seconds" - ], - "type": "object" - }, - "transcript": { - "properties": { - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "words": { - "description": "How many words; the words themselves are in transcript.words", - "type": "number" - } - }, - "required": [ - "model", - "language", - "words" - ], - "type": "object" - }, - "voice": { - "additionalProperties": {}, - "description": "frames, and the summary media.analyze voice gives", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "type": "object" - }, - "analysis": { - "anyOf": [ - { - "properties": { - "channels": { - "description": "Each channel at import", - "items": { - "properties": { - "channel": { - "exclusiveMinimum": 0, - "type": "integer" - }, - "noise_floor_dbfs": { - "type": [ - "number", - "null" - ] - }, - "peak_dbfs": { - "type": [ - "number", - "null" - ] - }, - "rms_dbfs": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "channel", - "rms_dbfs", - "peak_dbfs", - "noise_floor_dbfs" - ], - "type": "object" - }, - "type": "array" - }, - "loudness_lufs": { - "description": "Integrated loudness at import", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "loudness_lufs" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "beats": { - "properties": { - "beats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "bpm": { - "type": "number" - }, - "downbeats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "strengths": { - "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "bpm", - "beats", - "downbeats" - ], - "type": "object" - }, - "codecs": { - "properties": { - "audio": { - "type": [ - "string", - "null" - ] - }, - "video": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "video", - "audio" - ], - "type": "object" - }, - "conform": { - "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", - "items": { - "type": "string" - }, - "type": "array" - }, - "created_at": { - "description": "When it was imported or made", - "type": "string" - }, - "duration_seconds": { - "type": [ - "number", - "null" - ] - }, - "error": { - "properties": { - "code": { - "enum": [ - "bad_request", - "no_audio", - "not_found", - "expired", - "media_not_ready", - "media_offline", - "engine_error", - "render_failed", - "interrupted", - "stalled", - "cancelled", - "credential_required", - "credential_invalid", - "output_exists", - "path_not_allowed", - "limit_exceeded", - "quota_exceeded", - "subscription_required", - "email_unverified", - "service_unavailable", - "server_busy" - ], - "type": "string" - }, - "detail": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "hint": { - "type": "string" - }, - "message": { - "type": "string" - } - }, - "required": [ - "code", - "message" - ], - "type": "object" - }, - "expires_at": { - "description": "When url and page_url stop working", - "type": "string" - }, - "extension": { - "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", - "type": "string" - }, - "faces": { - "properties": { - "frames": { - "items": { - "properties": { - "faces": { - "items": { - "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", - "properties": { - "h": { - "type": "number" - }, - "score": { - "type": "number" - }, - "w": { - "type": "number" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "x", - "y", - "w", - "h", - "score" - ], - "type": "object" - }, - "type": "array" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "faces" - ], - "type": "object" - }, - "type": "array" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames" - ], - "type": "object" - }, - "fingerprint": { - "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", - "type": [ - "string", - "null" - ] - }, - "font_family": { - "description": "Fonts: the family; use font_media_id in a text style to set text in it", - "type": "string" - }, - "fps": { - "anyOf": [ - { - "properties": { - "den": { - "type": "number" - }, - "num": { - "type": "number" - } - }, - "required": [ - "num", - "den" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "has_audio": { - "type": "boolean" - }, - "has_video": { - "type": "boolean" - }, - "height": { - "type": [ - "number", - "null" - ] - }, - "job": { - "description": "media.get, while importing or being made: the job doing it", - "properties": { - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "progress": { - "type": "number" - }, - "seconds_left": { - "type": "number" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_message": { - "type": "string" - }, - "tool": { - "type": "string" - } - }, - "required": [ - "job_id", - "tool", - "status" - ], - "type": "object" - }, - "kind": { - "anyOf": [ - { - "enum": [ - "video", - "audio", - "image", - "font", - "lut" - ], - "type": "string" - }, - { - "type": "null" - } - ] - }, - "look": { - "properties": { - "runs": { - "items": { - "properties": { - "frames": { - "items": { - "properties": { - "answers": { - "additionalProperties": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "answers" - ], - "type": "object" - }, - "type": "array" - }, - "id": { - "type": "string" - }, - "model": { - "type": "string" - }, - "questions": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "state": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "with_previous": { - "type": "boolean" - } - }, - "required": [ - "id", - "model", - "questions", - "with_previous", - "frames" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "runs" - ], - "type": "object" - }, - "media_id": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "mime_type": { - "description": "Its MIME type, likewise", - "type": "string" - }, - "name": { - "type": "string" - }, - "origin": { - "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", - "properties": { - "attached": { - "description": "url: the file the user attached in the chat, passed as media.import file", - "type": "boolean" - }, - "file_name": { - "description": "upload: the name the file was sent with", - "type": "string" - }, - "frame": { - "description": "still: the frame", - "type": "number" - }, - "from": { - "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", - "enum": [ - "upload", - "url", - "path", - "render", - "created", - "still" - ], - "type": "string" - }, - "job_id": { - "description": "render: the export.video job; created: the create.* job", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "still: the video the frame is from", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "media_ids": { - "description": "created: the media it was made from (a cutout's source, a page's assets)", - "items": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "type": "array" - }, - "path": { - "description": "path: the file it was imported from", - "type": "string" - }, - "preset": { - "description": "render: the export preset", - "type": "string" - }, - "range": { - "description": "render: the frames exported, when not the whole timeline", - "properties": { - "end": { - "type": "number" - }, - "start": { - "type": "number" - } - }, - "required": [ - "start", - "end" - ], - "type": "object" - }, - "recipe": { - "description": "created: a hash of what it was made from; the same recipe makes the same media", - "type": "string" - }, - "sources": { - "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", - "items": { - "properties": { - "end": { - "type": "number" - }, - "kind": { - "enum": [ - "video", - "audio" - ], - "type": "string" - }, - "media_id": { - "description": "The media this stretch plays", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "source_in": { - "description": "Seconds into that media", - "type": "number" - }, - "source_out": { - "type": "number" - }, - "speed": { - "additionalProperties": {}, - "description": "The clip's speed change, as clip.set_speed set it", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "start": { - "description": "Seconds into this file", - "type": "number" - }, - "track_id": { - "description": "Track of the timeline; outer/inner inside a nested one", - "type": "string" - } - }, - "required": [ - "track_id", - "kind", - "start", - "end", - "media_id", - "source_in", - "source_out" - ], - "type": "object" - }, - "type": "array" - }, - "timeline_id": { - "description": "render: the timeline that was exported; still: the edit the frame is from", - "pattern": "^tl_[0-9a-f]{32}$", - "type": "string" - }, - "tool": { - "description": "created: the create.* tool that made it", - "type": "string" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "url: the address it was downloaded from, without its query string", - "type": "string" - }, - "via": { - "description": "upload: sent through a media.upload link, dropped on the account page, or a data: URI", - "enum": [ - "media.upload", - "account_page", - "data_uri" - ], - "type": "string" - } - }, - "required": [ - "from" - ], - "type": "object" - }, - "page_url": { - "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", - "type": "string" - }, - "pages": { - "additionalProperties": { - "properties": { - "first": { - "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", - "type": "integer" - }, - "list": { - "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", - "type": "string" - }, - "matching": { - "description": "Items in the time range (the whole list without one)", - "type": "integer" - }, - "next_offset": { - "anyOf": [ - { - "type": "integer" - }, - { - "type": "null" - } - ], - "description": "The offset that gives the next page of this range, or null at its end" - }, - "returned": { - "type": "integer" - }, - "total": { - "description": "Items in the whole list", - "type": "integer" - } - }, - "required": [ - "list", - "first", - "returned", - "matching", - "total", - "next_offset" - ], - "type": "object" - }, - "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", - "propertyNames": { - "enum": [ - "transcript", - "beats", - "faces", - "look", - "speech", - "sounds", - "speakers", - "voice", - "colour", - "picture" - ], - "type": "string" - }, - "type": "object" - }, - "probe": { - "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", - "properties": { - "format": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "streams": { - "items": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "format", - "streams" - ], - "type": "object" - }, - "sound_seconds": { - "description": "How long the sound runs; under duration_seconds, the end is soundless", - "type": "number" - }, - "sounds": { - "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", - "properties": { - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - }, - "peaks": { - "additionalProperties": { - "type": "number" - }, - "description": "Per sound reported: the highest probability it reaches anywhere", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "scores": { - "description": "Every probability, for your own cut-off or arithmetic", - "properties": { - "start_seconds": { - "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", - "type": "number" - }, - "step_seconds": { - "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", - "type": "number" - }, - "values": { - "additionalProperties": { - "items": { - "type": "number" - }, - "type": "array" - }, - "description": "Per sound, the highest probability in each step, 0 to 1", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "start_seconds", - "step_seconds", - "values" - ], - "type": "object" - }, - "spans": { - "additionalProperties": { - "items": { - "properties": { - "end": { - "description": "Where it ends, on the same clock", - "type": "number" - }, - "peak_probability": { - "type": "number" - }, - "peak_seconds": { - "description": "The middle of the most likely frame", - "type": "number" - }, - "start": { - "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", - "type": "number" - } - }, - "required": [ - "start", - "end", - "peak_seconds", - "peak_probability" - ], - "type": "object" - }, - "type": "array" - }, - "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "model", - "min_probability", - "spans", - "peaks" - ], - "type": "object" - }, - "speech": { - "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", - "properties": { - "frame_seconds": { - "description": "Each probability covers this long", - "type": "number" - }, - "from": { - "description": "Seconds where the first probability starts", - "type": "number" - }, - "model": { - "type": "string" - }, - "probabilities": { - "description": "Speech probability 0–1, one per frame, in order", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "model", - "frame_seconds", - "from", - "probabilities" - ], - "type": "object" - }, - "status": { - "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", - "enum": [ - "importing", - "processing", - "ready", - "failed" - ], - "type": "string" - }, - "transcript": { - "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", - "properties": { - "aligned_by": { - "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", - "type": [ - "string", - "null" - ] - }, - "from": { - "description": "The stretch transcribed, if not all", - "type": "number" - }, - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "text": { - "type": "string" - }, - "until": { - "type": [ - "number", - "null" - ] - }, - "words": { - "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", - "items": { - "properties": { - "confidence": { - "description": "0–1: how sure whisper is of the word itself", - "type": "number" - }, - "end": { - "type": "number" - }, - "speaker": { - "description": "Who says it, from the latest speakers run", - "type": [ - "number", - "null" - ] - }, - "start": { - "type": "number" - }, - "text": { - "type": "string" - }, - "timing_confidence": { - "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "text", - "start", - "end", - "confidence" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "model", - "language", - "text", - "words" - ], - "type": "object" - }, - "upload": { - "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", - "properties": { - "expires_at": { - "type": "string" - }, - "max_bytes": { - "type": "number" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "Signed: PUT the file here; no other credentials needed", - "type": "string" - } - }, - "required": [ - "upload_id", - "url", - "expires_at", - "max_bytes" - ], - "type": "object" - }, - "url": { - "description": "media.get: a signed download of the file itself, once it has one", - "type": "string" - }, - "warnings": { - "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", - "items": { - "type": "string" - }, - "type": "array" - }, - "width": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "media_id", - "name", - "status", - "kind", - "duration_seconds", - "fps", - "width", - "height", - "has_video", - "has_audio", - "codecs", - "conform", - "analysis" - ], - "type": "object" - }, - { - "properties": { - "hint": { - "type": "string" - }, - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "The media it is making; usable in edits once processing", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_uri": { - "type": "string" - } - }, - "required": [ - "job_id", - "status", - "status_uri", - "hint" - ], - "type": "object" - } -]New value: +[ + { + "properties": { + "analyses": { + "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", + "properties": { + "beats": { + "properties": { + "beats": { + "type": "number" + }, + "bpm": { + "type": "number" + } + }, + "required": [ + "bpm", + "beats" + ], + "type": "object" + }, + "colour": { + "properties": { + "frames": { + "type": "number" + }, + "regions": { + "items": { + "type": "string" + }, + "type": "array" + } + }, + "required": [ + "frames", + "regions" + ], + "type": "object" + }, + "faces": { + "properties": { + "frames_with_faces": { + "type": "number" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames_with_faces" + ], + "type": "object" + }, + "look": { + "description": "frames and questions: the latest run", + "properties": { + "frames": { + "type": "number" + }, + "questions": { + "items": { + "type": "string" + }, + "type": "array" + }, + "runs": { + "type": "number" + } + }, + "required": [ + "runs", + "frames", + "questions" + ], + "type": "object" + }, + "picture": { + "properties": { + "frames": { + "type": "number" + }, + "from": { + "type": "number" + } + }, + "required": [ + "frames", + "from" + ], + "type": "object" + }, + "sounds": { + "description": "heard: the sounds the latest run found at or above its min_probability", + "properties": { + "heard": { + "items": { + "type": "string" + }, + "type": "array" + }, + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "min_probability", + "heard" + ], + "type": "object" + }, + "speakers": { + "properties": { + "model": { + "type": "string" + }, + "speakers": { + "type": "number" + }, + "turns": { + "type": "number" + } + }, + "required": [ + "model", + "speakers", + "turns" + ], + "type": "object" + }, + "speech": { + "properties": { + "frame_seconds": { + "type": "number" + }, + "frames": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "frames", + "frame_seconds" + ], + "type": "object" + }, + "transcript": { + "properties": { + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "words": { + "description": "How many words; the words themselves are in transcript.words", + "type": "number" + } + }, + "required": [ + "model", + "language", + "words" + ], + "type": "object" + }, + "voice": { + "additionalProperties": {}, + "description": "frames, and the summary media.analyze voice gives", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "type": "object" + }, + "analysis": { + "anyOf": [ + { + "properties": { + "channels": { + "description": "Each channel at import", + "items": { + "properties": { + "channel": { + "exclusiveMinimum": 0, + "type": "integer" + }, + "noise_floor_dbfs": { + "type": [ + "number", + "null" + ] + }, + "peak_dbfs": { + "type": [ + "number", + "null" + ] + }, + "rms_dbfs": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "channel", + "rms_dbfs", + "peak_dbfs", + "noise_floor_dbfs" + ], + "type": "object" + }, + "type": "array" + }, + "loudness_lufs": { + "description": "Integrated loudness at import", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "loudness_lufs" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "beats": { + "properties": { + "beats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "bpm": { + "type": "number" + }, + "downbeats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "strengths": { + "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "bpm", + "beats", + "downbeats" + ], + "type": "object" + }, + "codecs": { + "properties": { + "audio": { + "type": [ + "string", + "null" + ] + }, + "video": { + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "video", + "audio" + ], + "type": "object" + }, + "conform": { + "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", + "items": { + "type": "string" + }, + "type": "array" + }, + "created_at": { + "description": "When it was imported or made", + "type": "string" + }, + "duration_seconds": { + "type": [ + "number", + "null" + ] + }, + "error": { + "properties": { + "code": { + "enum": [ + "bad_request", + "no_audio", + "not_found", + "expired", + "media_not_ready", + "media_offline", + "engine_error", + "render_failed", + "interrupted", + "stalled", + "cancelled", + "credential_required", + "credential_invalid", + "output_exists", + "path_not_allowed", + "limit_exceeded", + "quota_exceeded", + "subscription_required", + "email_unverified", + "service_unavailable", + "server_busy" + ], + "type": "string" + }, + "detail": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "hint": { + "type": "string" + }, + "message": { + "type": "string" + } + }, + "required": [ + "code", + "message" + ], + "type": "object" + }, + "expires_at": { + "description": "When url and page_url stop working", + "type": "string" + }, + "extension": { + "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", + "type": "string" + }, + "faces": { + "properties": { + "frames": { + "items": { + "properties": { + "faces": { + "items": { + "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", + "properties": { + "h": { + "type": "number" + }, + "score": { + "type": "number" + }, + "w": { + "type": "number" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "x", + "y", + "w", + "h", + "score" + ], + "type": "object" + }, + "type": "array" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "faces" + ], + "type": "object" + }, + "type": "array" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames" + ], + "type": "object" + }, + "fingerprint": { + "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", + "type": [ + "string", + "null" + ] + }, + "font_family": { + "description": "Fonts: the family; use font_media_id in a text style to set text in it", + "type": "string" + }, + "fps": { + "anyOf": [ + { + "properties": { + "den": { + "type": "number" + }, + "num": { + "type": "number" + } + }, + "required": [ + "num", + "den" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "has_audio": { + "type": "boolean" + }, + "has_video": { + "type": "boolean" + }, + "height": { + "type": [ + "number", + "null" + ] + }, + "job": { + "description": "media.get, while importing or being made: the job doing it", + "properties": { + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "progress": { + "type": "number" + }, + "seconds_left": { + "type": "number" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_message": { + "type": "string" + }, + "tool": { + "type": "string" + } + }, + "required": [ + "job_id", + "tool", + "status" + ], + "type": "object" + }, + "kind": { + "anyOf": [ + { + "enum": [ + "video", + "audio", + "image", + "font", + "lut" + ], + "type": "string" + }, + { + "type": "null" + } + ] + }, + "look": { + "properties": { + "runs": { + "items": { + "properties": { + "frames": { + "items": { + "properties": { + "answers": { + "additionalProperties": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "answers" + ], + "type": "object" + }, + "type": "array" + }, + "id": { + "type": "string" + }, + "model": { + "type": "string" + }, + "questions": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "state": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "with_previous": { + "type": "boolean" + } + }, + "required": [ + "id", + "model", + "questions", + "with_previous", + "frames" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "runs" + ], + "type": "object" + }, + "media_id": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "mime_type": { + "description": "Its MIME type, likewise", + "type": "string" + }, + "name": { + "type": "string" + }, + "origin": { + "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", + "properties": { + "attached": { + "description": "url: the file the user attached in the chat, passed as media.import file", + "type": "boolean" + }, + "file_name": { + "description": "upload: the name the file was sent with", + "type": "string" + }, + "frame": { + "description": "still: the frame", + "type": "number" + }, + "from": { + "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", + "enum": [ + "upload", + "url", + "path", + "render", + "created", + "still" + ], + "type": "string" + }, + "job_id": { + "description": "render: the export.video job; created: the create.* job", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "still: the video the frame is from", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "media_ids": { + "description": "created: the media it was made from (a cutout's source, a page's assets)", + "items": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "type": "array" + }, + "path": { + "description": "path: the file it was imported from", + "type": "string" + }, + "preset": { + "description": "render: the export preset", + "type": "string" + }, + "range": { + "description": "render: the frames exported, when not the whole timeline", + "properties": { + "end": { + "type": "number" + }, + "start": { + "type": "number" + } + }, + "required": [ + "start", + "end" + ], + "type": "object" + }, + "recipe": { + "description": "created: a hash of what it was made from; the same recipe makes the same media", + "type": "string" + }, + "sources": { + "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", + "items": { + "properties": { + "end": { + "type": "number" + }, + "kind": { + "enum": [ + "video", + "audio" + ], + "type": "string" + }, + "media_id": { + "description": "The media this stretch plays", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "source_in": { + "description": "Seconds into that media", + "type": "number" + }, + "source_out": { + "type": "number" + }, + "speed": { + "additionalProperties": {}, + "description": "The clip's speed change, as clip.set_speed set it", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "start": { + "description": "Seconds into this file", + "type": "number" + }, + "track_id": { + "description": "Track of the timeline; outer/inner inside a nested one", + "type": "string" + } + }, + "required": [ + "track_id", + "kind", + "start", + "end", + "media_id", + "source_in", + "source_out" + ], + "type": "object" + }, + "type": "array" + }, + "timeline_id": { + "description": "render: the timeline that was exported; still: the edit the frame is from", + "pattern": "^tl_[0-9a-f]{32}$", + "type": "string" + }, + "tool": { + "description": "created: the create.* tool that made it", + "type": "string" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "url: the address it was downloaded from, without its query string", + "type": "string" + }, + "via": { + "description": "upload: sent through a media.upload link, dropped on the cuttingfloor site, or a data: URI", + "enum": [ + "media.upload", + "account_page", + "data_uri" + ], + "type": "string" + } + }, + "required": [ + "from" + ], + "type": "object" + }, + "page_url": { + "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", + "type": "string" + }, + "pages": { + "additionalProperties": { + "properties": { + "first": { + "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", + "type": "integer" + }, + "list": { + "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", + "type": "string" + }, + "matching": { + "description": "Items in the time range (the whole list without one)", + "type": "integer" + }, + "next_offset": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "The offset that gives the next page of this range, or null at its end" + }, + "returned": { + "type": "integer" + }, + "total": { + "description": "Items in the whole list", + "type": "integer" + } + }, + "required": [ + "list", + "first", + "returned", + "matching", + "total", + "next_offset" + ], + "type": "object" + }, + "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", + "propertyNames": { + "enum": [ + "transcript", + "beats", + "faces", + "look", + "speech", + "sounds", + "speakers", + "voice", + "colour", + "picture" + ], + "type": "string" + }, + "type": "object" + }, + "probe": { + "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", + "properties": { + "format": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "streams": { + "items": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "format", + "streams" + ], + "type": "object" + }, + "sound_seconds": { + "description": "How long the sound runs; under duration_seconds, the end is soundless", + "type": "number" + }, + "sounds": { + "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", + "properties": { + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + }, + "peaks": { + "additionalProperties": { + "type": "number" + }, + "description": "Per sound reported: the highest probability it reaches anywhere", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "scores": { + "description": "Every probability, for your own cut-off or arithmetic", + "properties": { + "start_seconds": { + "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", + "type": "number" + }, + "step_seconds": { + "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", + "type": "number" + }, + "values": { + "additionalProperties": { + "items": { + "type": "number" + }, + "type": "array" + }, + "description": "Per sound, the highest probability in each step, 0 to 1", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "start_seconds", + "step_seconds", + "values" + ], + "type": "object" + }, + "spans": { + "additionalProperties": { + "items": { + "properties": { + "end": { + "description": "Where it ends, on the same clock", + "type": "number" + }, + "peak_probability": { + "type": "number" + }, + "peak_seconds": { + "description": "The middle of the most likely frame", + "type": "number" + }, + "start": { + "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", + "type": "number" + } + }, + "required": [ + "start", + "end", + "peak_seconds", + "peak_probability" + ], + "type": "object" + }, + "type": "array" + }, + "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "model", + "min_probability", + "spans", + "peaks" + ], + "type": "object" + }, + "speech": { + "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", + "properties": { + "frame_seconds": { + "description": "Each probability covers this long", + "type": "number" + }, + "from": { + "description": "Seconds where the first probability starts", + "type": "number" + }, + "model": { + "type": "string" + }, + "probabilities": { + "description": "Speech probability 0–1, one per frame, in order", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "model", + "frame_seconds", + "from", + "probabilities" + ], + "type": "object" + }, + "status": { + "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", + "enum": [ + "importing", + "processing", + "ready", + "failed" + ], + "type": "string" + }, + "transcript": { + "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", + "properties": { + "aligned_by": { + "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", + "type": [ + "string", + "null" + ] + }, + "from": { + "description": "The stretch transcribed, if not all", + "type": "number" + }, + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "text": { + "type": "string" + }, + "until": { + "type": [ + "number", + "null" + ] + }, + "words": { + "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", + "items": { + "properties": { + "confidence": { + "description": "0–1: how sure whisper is of the word itself", + "type": "number" + }, + "end": { + "type": "number" + }, + "speaker": { + "description": "Who says it, from the latest speakers run", + "type": [ + "number", + "null" + ] + }, + "start": { + "type": "number" + }, + "text": { + "type": "string" + }, + "timing_confidence": { + "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "text", + "start", + "end", + "confidence" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "model", + "language", + "text", + "words" + ], + "type": "object" + }, + "upload": { + "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", + "properties": { + "expires_at": { + "type": "string" + }, + "max_bytes": { + "type": "number" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "Signed: PUT the file here; no other credentials needed", + "type": "string" + } + }, + "required": [ + "upload_id", + "url", + "expires_at", + "max_bytes" + ], + "type": "object" + }, + "url": { + "description": "media.get: a signed download of the file itself, once it has one", + "type": "string" + }, + "warnings": { + "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", + "items": { + "type": "string" + }, + "type": "array" + }, + "width": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "media_id", + "name", + "status", + "kind", + "duration_seconds", + "fps", + "width", + "height", + "has_video", + "has_audio", + "codecs", + "conform", + "analysis" + ], + "type": "object" + }, + { + "properties": { + "hint": { + "type": "string" + }, + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "The media it is making; usable in edits once processing", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_uri": { + "type": "string" + } + }, + "required": [ + "job_id", + "status", + "status_uri", + "hint" + ], + "type": "object" + } +]
- Changed
create.lipsync1 field changed- changed
Output schema / anyOfPrevious value: -[ - { - "properties": { - "analyses": { - "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", - "properties": { - "beats": { - "properties": { - "beats": { - "type": "number" - }, - "bpm": { - "type": "number" - } - }, - "required": [ - "bpm", - "beats" - ], - "type": "object" - }, - "colour": { - "properties": { - "frames": { - "type": "number" - }, - "regions": { - "items": { - "type": "string" - }, - "type": "array" - } - }, - "required": [ - "frames", - "regions" - ], - "type": "object" - }, - "faces": { - "properties": { - "frames_with_faces": { - "type": "number" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames_with_faces" - ], - "type": "object" - }, - "look": { - "description": "frames and questions: the latest run", - "properties": { - "frames": { - "type": "number" - }, - "questions": { - "items": { - "type": "string" - }, - "type": "array" - }, - "runs": { - "type": "number" - } - }, - "required": [ - "runs", - "frames", - "questions" - ], - "type": "object" - }, - "picture": { - "properties": { - "frames": { - "type": "number" - }, - "from": { - "type": "number" - } - }, - "required": [ - "frames", - "from" - ], - "type": "object" - }, - "sounds": { - "description": "heard: the sounds the latest run found at or above its min_probability", - "properties": { - "heard": { - "items": { - "type": "string" - }, - "type": "array" - }, - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "min_probability", - "heard" - ], - "type": "object" - }, - "speakers": { - "properties": { - "model": { - "type": "string" - }, - "speakers": { - "type": "number" - }, - "turns": { - "type": "number" - } - }, - "required": [ - "model", - "speakers", - "turns" - ], - "type": "object" - }, - "speech": { - "properties": { - "frame_seconds": { - "type": "number" - }, - "frames": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "frames", - "frame_seconds" - ], - "type": "object" - }, - "transcript": { - "properties": { - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "words": { - "description": "How many words; the words themselves are in transcript.words", - "type": "number" - } - }, - "required": [ - "model", - "language", - "words" - ], - "type": "object" - }, - "voice": { - "additionalProperties": {}, - "description": "frames, and the summary media.analyze voice gives", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "type": "object" - }, - "analysis": { - "anyOf": [ - { - "properties": { - "channels": { - "description": "Each channel at import", - "items": { - "properties": { - "channel": { - "exclusiveMinimum": 0, - "type": "integer" - }, - "noise_floor_dbfs": { - "type": [ - "number", - "null" - ] - }, - "peak_dbfs": { - "type": [ - "number", - "null" - ] - }, - "rms_dbfs": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "channel", - "rms_dbfs", - "peak_dbfs", - "noise_floor_dbfs" - ], - "type": "object" - }, - "type": "array" - }, - "loudness_lufs": { - "description": "Integrated loudness at import", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "loudness_lufs" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "beats": { - "properties": { - "beats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "bpm": { - "type": "number" - }, - "downbeats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "strengths": { - "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "bpm", - "beats", - "downbeats" - ], - "type": "object" - }, - "codecs": { - "properties": { - "audio": { - "type": [ - "string", - "null" - ] - }, - "video": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "video", - "audio" - ], - "type": "object" - }, - "conform": { - "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", - "items": { - "type": "string" - }, - "type": "array" - }, - "created_at": { - "description": "When it was imported or made", - "type": "string" - }, - "duration_seconds": { - "type": [ - "number", - "null" - ] - }, - "error": { - "properties": { - "code": { - "enum": [ - "bad_request", - "no_audio", - "not_found", - "expired", - "media_not_ready", - "media_offline", - "engine_error", - "render_failed", - "interrupted", - "stalled", - "cancelled", - "credential_required", - "credential_invalid", - "output_exists", - "path_not_allowed", - "limit_exceeded", - "quota_exceeded", - "subscription_required", - "email_unverified", - "service_unavailable", - "server_busy" - ], - "type": "string" - }, - "detail": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "hint": { - "type": "string" - }, - "message": { - "type": "string" - } - }, - "required": [ - "code", - "message" - ], - "type": "object" - }, - "expires_at": { - "description": "When url and page_url stop working", - "type": "string" - }, - "extension": { - "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", - "type": "string" - }, - "faces": { - "properties": { - "frames": { - "items": { - "properties": { - "faces": { - "items": { - "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", - "properties": { - "h": { - "type": "number" - }, - "score": { - "type": "number" - }, - "w": { - "type": "number" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "x", - "y", - "w", - "h", - "score" - ], - "type": "object" - }, - "type": "array" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "faces" - ], - "type": "object" - }, - "type": "array" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames" - ], - "type": "object" - }, - "fingerprint": { - "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", - "type": [ - "string", - "null" - ] - }, - "font_family": { - "description": "Fonts: the family; use font_media_id in a text style to set text in it", - "type": "string" - }, - "fps": { - "anyOf": [ - { - "properties": { - "den": { - "type": "number" - }, - "num": { - "type": "number" - } - }, - "required": [ - "num", - "den" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "has_audio": { - "type": "boolean" - }, - "has_video": { - "type": "boolean" - }, - "height": { - "type": [ - "number", - "null" - ] - }, - "job": { - "description": "media.get, while importing or being made: the job doing it", - "properties": { - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "progress": { - "type": "number" - }, - "seconds_left": { - "type": "number" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_message": { - "type": "string" - }, - "tool": { - "type": "string" - } - }, - "required": [ - "job_id", - "tool", - "status" - ], - "type": "object" - }, - "kind": { - "anyOf": [ - { - "enum": [ - "video", - "audio", - "image", - "font", - "lut" - ], - "type": "string" - }, - { - "type": "null" - } - ] - }, - "look": { - "properties": { - "runs": { - "items": { - "properties": { - "frames": { - "items": { - "properties": { - "answers": { - "additionalProperties": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "answers" - ], - "type": "object" - }, - "type": "array" - }, - "id": { - "type": "string" - }, - "model": { - "type": "string" - }, - "questions": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "state": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "with_previous": { - "type": "boolean" - } - }, - "required": [ - "id", - "model", - "questions", - "with_previous", - "frames" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "runs" - ], - "type": "object" - }, - "media_id": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "mime_type": { - "description": "Its MIME type, likewise", - "type": "string" - }, - "name": { - "type": "string" - }, - "origin": { - "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", - "properties": { - "attached": { - "description": "url: the file the user attached in the chat, passed as media.import file", - "type": "boolean" - }, - "file_name": { - "description": "upload: the name the file was sent with", - "type": "string" - }, - "frame": { - "description": "still: the frame", - "type": "number" - }, - "from": { - "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", - "enum": [ - "upload", - "url", - "path", - "render", - "created", - "still" - ], - "type": "string" - }, - "job_id": { - "description": "render: the export.video job; created: the create.* job", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "still: the video the frame is from", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "media_ids": { - "description": "created: the media it was made from (a cutout's source, a page's assets)", - "items": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "type": "array" - }, - "path": { - "description": "path: the file it was imported from", - "type": "string" - }, - "preset": { - "description": "render: the export preset", - "type": "string" - }, - "range": { - "description": "render: the frames exported, when not the whole timeline", - "properties": { - "end": { - "type": "number" - }, - "start": { - "type": "number" - } - }, - "required": [ - "start", - "end" - ], - "type": "object" - }, - "recipe": { - "description": "created: a hash of what it was made from; the same recipe makes the same media", - "type": "string" - }, - "sources": { - "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", - "items": { - "properties": { - "end": { - "type": "number" - }, - "kind": { - "enum": [ - "video", - "audio" - ], - "type": "string" - }, - "media_id": { - "description": "The media this stretch plays", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "source_in": { - "description": "Seconds into that media", - "type": "number" - }, - "source_out": { - "type": "number" - }, - "speed": { - "additionalProperties": {}, - "description": "The clip's speed change, as clip.set_speed set it", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "start": { - "description": "Seconds into this file", - "type": "number" - }, - "track_id": { - "description": "Track of the timeline; outer/inner inside a nested one", - "type": "string" - } - }, - "required": [ - "track_id", - "kind", - "start", - "end", - "media_id", - "source_in", - "source_out" - ], - "type": "object" - }, - "type": "array" - }, - "timeline_id": { - "description": "render: the timeline that was exported; still: the edit the frame is from", - "pattern": "^tl_[0-9a-f]{32}$", - "type": "string" - }, - "tool": { - "description": "created: the create.* tool that made it", - "type": "string" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "url: the address it was downloaded from, without its query string", - "type": "string" - }, - "via": { - "description": "upload: sent through a media.upload link, dropped on the account page, or a data: URI", - "enum": [ - "media.upload", - "account_page", - "data_uri" - ], - "type": "string" - } - }, - "required": [ - "from" - ], - "type": "object" - }, - "page_url": { - "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", - "type": "string" - }, - "pages": { - "additionalProperties": { - "properties": { - "first": { - "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", - "type": "integer" - }, - "list": { - "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", - "type": "string" - }, - "matching": { - "description": "Items in the time range (the whole list without one)", - "type": "integer" - }, - "next_offset": { - "anyOf": [ - { - "type": "integer" - }, - { - "type": "null" - } - ], - "description": "The offset that gives the next page of this range, or null at its end" - }, - "returned": { - "type": "integer" - }, - "total": { - "description": "Items in the whole list", - "type": "integer" - } - }, - "required": [ - "list", - "first", - "returned", - "matching", - "total", - "next_offset" - ], - "type": "object" - }, - "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", - "propertyNames": { - "enum": [ - "transcript", - "beats", - "faces", - "look", - "speech", - "sounds", - "speakers", - "voice", - "colour", - "picture" - ], - "type": "string" - }, - "type": "object" - }, - "probe": { - "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", - "properties": { - "format": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "streams": { - "items": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "format", - "streams" - ], - "type": "object" - }, - "sound_seconds": { - "description": "How long the sound runs; under duration_seconds, the end is soundless", - "type": "number" - }, - "sounds": { - "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", - "properties": { - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - }, - "peaks": { - "additionalProperties": { - "type": "number" - }, - "description": "Per sound reported: the highest probability it reaches anywhere", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "scores": { - "description": "Every probability, for your own cut-off or arithmetic", - "properties": { - "start_seconds": { - "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", - "type": "number" - }, - "step_seconds": { - "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", - "type": "number" - }, - "values": { - "additionalProperties": { - "items": { - "type": "number" - }, - "type": "array" - }, - "description": "Per sound, the highest probability in each step, 0 to 1", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "start_seconds", - "step_seconds", - "values" - ], - "type": "object" - }, - "spans": { - "additionalProperties": { - "items": { - "properties": { - "end": { - "description": "Where it ends, on the same clock", - "type": "number" - }, - "peak_probability": { - "type": "number" - }, - "peak_seconds": { - "description": "The middle of the most likely frame", - "type": "number" - }, - "start": { - "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", - "type": "number" - } - }, - "required": [ - "start", - "end", - "peak_seconds", - "peak_probability" - ], - "type": "object" - }, - "type": "array" - }, - "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "model", - "min_probability", - "spans", - "peaks" - ], - "type": "object" - }, - "speech": { - "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", - "properties": { - "frame_seconds": { - "description": "Each probability covers this long", - "type": "number" - }, - "from": { - "description": "Seconds where the first probability starts", - "type": "number" - }, - "model": { - "type": "string" - }, - "probabilities": { - "description": "Speech probability 0–1, one per frame, in order", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "model", - "frame_seconds", - "from", - "probabilities" - ], - "type": "object" - }, - "status": { - "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", - "enum": [ - "importing", - "processing", - "ready", - "failed" - ], - "type": "string" - }, - "transcript": { - "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", - "properties": { - "aligned_by": { - "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", - "type": [ - "string", - "null" - ] - }, - "from": { - "description": "The stretch transcribed, if not all", - "type": "number" - }, - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "text": { - "type": "string" - }, - "until": { - "type": [ - "number", - "null" - ] - }, - "words": { - "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", - "items": { - "properties": { - "confidence": { - "description": "0–1: how sure whisper is of the word itself", - "type": "number" - }, - "end": { - "type": "number" - }, - "speaker": { - "description": "Who says it, from the latest speakers run", - "type": [ - "number", - "null" - ] - }, - "start": { - "type": "number" - }, - "text": { - "type": "string" - }, - "timing_confidence": { - "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "text", - "start", - "end", - "confidence" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "model", - "language", - "text", - "words" - ], - "type": "object" - }, - "upload": { - "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", - "properties": { - "expires_at": { - "type": "string" - }, - "max_bytes": { - "type": "number" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "Signed: PUT the file here; no other credentials needed", - "type": "string" - } - }, - "required": [ - "upload_id", - "url", - "expires_at", - "max_bytes" - ], - "type": "object" - }, - "url": { - "description": "media.get: a signed download of the file itself, once it has one", - "type": "string" - }, - "warnings": { - "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", - "items": { - "type": "string" - }, - "type": "array" - }, - "width": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "media_id", - "name", - "status", - "kind", - "duration_seconds", - "fps", - "width", - "height", - "has_video", - "has_audio", - "codecs", - "conform", - "analysis" - ], - "type": "object" - }, - { - "properties": { - "hint": { - "type": "string" - }, - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "The media it is making; usable in edits once processing", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_uri": { - "type": "string" - } - }, - "required": [ - "job_id", - "status", - "status_uri", - "hint" - ], - "type": "object" - } -]New value: +[ + { + "properties": { + "analyses": { + "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", + "properties": { + "beats": { + "properties": { + "beats": { + "type": "number" + }, + "bpm": { + "type": "number" + } + }, + "required": [ + "bpm", + "beats" + ], + "type": "object" + }, + "colour": { + "properties": { + "frames": { + "type": "number" + }, + "regions": { + "items": { + "type": "string" + }, + "type": "array" + } + }, + "required": [ + "frames", + "regions" + ], + "type": "object" + }, + "faces": { + "properties": { + "frames_with_faces": { + "type": "number" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames_with_faces" + ], + "type": "object" + }, + "look": { + "description": "frames and questions: the latest run", + "properties": { + "frames": { + "type": "number" + }, + "questions": { + "items": { + "type": "string" + }, + "type": "array" + }, + "runs": { + "type": "number" + } + }, + "required": [ + "runs", + "frames", + "questions" + ], + "type": "object" + }, + "picture": { + "properties": { + "frames": { + "type": "number" + }, + "from": { + "type": "number" + } + }, + "required": [ + "frames", + "from" + ], + "type": "object" + }, + "sounds": { + "description": "heard: the sounds the latest run found at or above its min_probability", + "properties": { + "heard": { + "items": { + "type": "string" + }, + "type": "array" + }, + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "min_probability", + "heard" + ], + "type": "object" + }, + "speakers": { + "properties": { + "model": { + "type": "string" + }, + "speakers": { + "type": "number" + }, + "turns": { + "type": "number" + } + }, + "required": [ + "model", + "speakers", + "turns" + ], + "type": "object" + }, + "speech": { + "properties": { + "frame_seconds": { + "type": "number" + }, + "frames": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "frames", + "frame_seconds" + ], + "type": "object" + }, + "transcript": { + "properties": { + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "words": { + "description": "How many words; the words themselves are in transcript.words", + "type": "number" + } + }, + "required": [ + "model", + "language", + "words" + ], + "type": "object" + }, + "voice": { + "additionalProperties": {}, + "description": "frames, and the summary media.analyze voice gives", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "type": "object" + }, + "analysis": { + "anyOf": [ + { + "properties": { + "channels": { + "description": "Each channel at import", + "items": { + "properties": { + "channel": { + "exclusiveMinimum": 0, + "type": "integer" + }, + "noise_floor_dbfs": { + "type": [ + "number", + "null" + ] + }, + "peak_dbfs": { + "type": [ + "number", + "null" + ] + }, + "rms_dbfs": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "channel", + "rms_dbfs", + "peak_dbfs", + "noise_floor_dbfs" + ], + "type": "object" + }, + "type": "array" + }, + "loudness_lufs": { + "description": "Integrated loudness at import", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "loudness_lufs" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "beats": { + "properties": { + "beats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "bpm": { + "type": "number" + }, + "downbeats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "strengths": { + "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "bpm", + "beats", + "downbeats" + ], + "type": "object" + }, + "codecs": { + "properties": { + "audio": { + "type": [ + "string", + "null" + ] + }, + "video": { + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "video", + "audio" + ], + "type": "object" + }, + "conform": { + "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", + "items": { + "type": "string" + }, + "type": "array" + }, + "created_at": { + "description": "When it was imported or made", + "type": "string" + }, + "duration_seconds": { + "type": [ + "number", + "null" + ] + }, + "error": { + "properties": { + "code": { + "enum": [ + "bad_request", + "no_audio", + "not_found", + "expired", + "media_not_ready", + "media_offline", + "engine_error", + "render_failed", + "interrupted", + "stalled", + "cancelled", + "credential_required", + "credential_invalid", + "output_exists", + "path_not_allowed", + "limit_exceeded", + "quota_exceeded", + "subscription_required", + "email_unverified", + "service_unavailable", + "server_busy" + ], + "type": "string" + }, + "detail": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "hint": { + "type": "string" + }, + "message": { + "type": "string" + } + }, + "required": [ + "code", + "message" + ], + "type": "object" + }, + "expires_at": { + "description": "When url and page_url stop working", + "type": "string" + }, + "extension": { + "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", + "type": "string" + }, + "faces": { + "properties": { + "frames": { + "items": { + "properties": { + "faces": { + "items": { + "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", + "properties": { + "h": { + "type": "number" + }, + "score": { + "type": "number" + }, + "w": { + "type": "number" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "x", + "y", + "w", + "h", + "score" + ], + "type": "object" + }, + "type": "array" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "faces" + ], + "type": "object" + }, + "type": "array" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames" + ], + "type": "object" + }, + "fingerprint": { + "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", + "type": [ + "string", + "null" + ] + }, + "font_family": { + "description": "Fonts: the family; use font_media_id in a text style to set text in it", + "type": "string" + }, + "fps": { + "anyOf": [ + { + "properties": { + "den": { + "type": "number" + }, + "num": { + "type": "number" + } + }, + "required": [ + "num", + "den" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "has_audio": { + "type": "boolean" + }, + "has_video": { + "type": "boolean" + }, + "height": { + "type": [ + "number", + "null" + ] + }, + "job": { + "description": "media.get, while importing or being made: the job doing it", + "properties": { + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "progress": { + "type": "number" + }, + "seconds_left": { + "type": "number" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_message": { + "type": "string" + }, + "tool": { + "type": "string" + } + }, + "required": [ + "job_id", + "tool", + "status" + ], + "type": "object" + }, + "kind": { + "anyOf": [ + { + "enum": [ + "video", + "audio", + "image", + "font", + "lut" + ], + "type": "string" + }, + { + "type": "null" + } + ] + }, + "look": { + "properties": { + "runs": { + "items": { + "properties": { + "frames": { + "items": { + "properties": { + "answers": { + "additionalProperties": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "answers" + ], + "type": "object" + }, + "type": "array" + }, + "id": { + "type": "string" + }, + "model": { + "type": "string" + }, + "questions": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "state": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "with_previous": { + "type": "boolean" + } + }, + "required": [ + "id", + "model", + "questions", + "with_previous", + "frames" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "runs" + ], + "type": "object" + }, + "media_id": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "mime_type": { + "description": "Its MIME type, likewise", + "type": "string" + }, + "name": { + "type": "string" + }, + "origin": { + "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", + "properties": { + "attached": { + "description": "url: the file the user attached in the chat, passed as media.import file", + "type": "boolean" + }, + "file_name": { + "description": "upload: the name the file was sent with", + "type": "string" + }, + "frame": { + "description": "still: the frame", + "type": "number" + }, + "from": { + "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", + "enum": [ + "upload", + "url", + "path", + "render", + "created", + "still" + ], + "type": "string" + }, + "job_id": { + "description": "render: the export.video job; created: the create.* job", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "still: the video the frame is from", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "media_ids": { + "description": "created: the media it was made from (a cutout's source, a page's assets)", + "items": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "type": "array" + }, + "path": { + "description": "path: the file it was imported from", + "type": "string" + }, + "preset": { + "description": "render: the export preset", + "type": "string" + }, + "range": { + "description": "render: the frames exported, when not the whole timeline", + "properties": { + "end": { + "type": "number" + }, + "start": { + "type": "number" + } + }, + "required": [ + "start", + "end" + ], + "type": "object" + }, + "recipe": { + "description": "created: a hash of what it was made from; the same recipe makes the same media", + "type": "string" + }, + "sources": { + "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", + "items": { + "properties": { + "end": { + "type": "number" + }, + "kind": { + "enum": [ + "video", + "audio" + ], + "type": "string" + }, + "media_id": { + "description": "The media this stretch plays", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "source_in": { + "description": "Seconds into that media", + "type": "number" + }, + "source_out": { + "type": "number" + }, + "speed": { + "additionalProperties": {}, + "description": "The clip's speed change, as clip.set_speed set it", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "start": { + "description": "Seconds into this file", + "type": "number" + }, + "track_id": { + "description": "Track of the timeline; outer/inner inside a nested one", + "type": "string" + } + }, + "required": [ + "track_id", + "kind", + "start", + "end", + "media_id", + "source_in", + "source_out" + ], + "type": "object" + }, + "type": "array" + }, + "timeline_id": { + "description": "render: the timeline that was exported; still: the edit the frame is from", + "pattern": "^tl_[0-9a-f]{32}$", + "type": "string" + }, + "tool": { + "description": "created: the create.* tool that made it", + "type": "string" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "url: the address it was downloaded from, without its query string", + "type": "string" + }, + "via": { + "description": "upload: sent through a media.upload link, dropped on the cuttingfloor site, or a data: URI", + "enum": [ + "media.upload", + "account_page", + "data_uri" + ], + "type": "string" + } + }, + "required": [ + "from" + ], + "type": "object" + }, + "page_url": { + "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", + "type": "string" + }, + "pages": { + "additionalProperties": { + "properties": { + "first": { + "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", + "type": "integer" + }, + "list": { + "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", + "type": "string" + }, + "matching": { + "description": "Items in the time range (the whole list without one)", + "type": "integer" + }, + "next_offset": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "The offset that gives the next page of this range, or null at its end" + }, + "returned": { + "type": "integer" + }, + "total": { + "description": "Items in the whole list", + "type": "integer" + } + }, + "required": [ + "list", + "first", + "returned", + "matching", + "total", + "next_offset" + ], + "type": "object" + }, + "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", + "propertyNames": { + "enum": [ + "transcript", + "beats", + "faces", + "look", + "speech", + "sounds", + "speakers", + "voice", + "colour", + "picture" + ], + "type": "string" + }, + "type": "object" + }, + "probe": { + "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", + "properties": { + "format": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "streams": { + "items": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "format", + "streams" + ], + "type": "object" + }, + "sound_seconds": { + "description": "How long the sound runs; under duration_seconds, the end is soundless", + "type": "number" + }, + "sounds": { + "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", + "properties": { + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + }, + "peaks": { + "additionalProperties": { + "type": "number" + }, + "description": "Per sound reported: the highest probability it reaches anywhere", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "scores": { + "description": "Every probability, for your own cut-off or arithmetic", + "properties": { + "start_seconds": { + "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", + "type": "number" + }, + "step_seconds": { + "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", + "type": "number" + }, + "values": { + "additionalProperties": { + "items": { + "type": "number" + }, + "type": "array" + }, + "description": "Per sound, the highest probability in each step, 0 to 1", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "start_seconds", + "step_seconds", + "values" + ], + "type": "object" + }, + "spans": { + "additionalProperties": { + "items": { + "properties": { + "end": { + "description": "Where it ends, on the same clock", + "type": "number" + }, + "peak_probability": { + "type": "number" + }, + "peak_seconds": { + "description": "The middle of the most likely frame", + "type": "number" + }, + "start": { + "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", + "type": "number" + } + }, + "required": [ + "start", + "end", + "peak_seconds", + "peak_probability" + ], + "type": "object" + }, + "type": "array" + }, + "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "model", + "min_probability", + "spans", + "peaks" + ], + "type": "object" + }, + "speech": { + "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", + "properties": { + "frame_seconds": { + "description": "Each probability covers this long", + "type": "number" + }, + "from": { + "description": "Seconds where the first probability starts", + "type": "number" + }, + "model": { + "type": "string" + }, + "probabilities": { + "description": "Speech probability 0–1, one per frame, in order", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "model", + "frame_seconds", + "from", + "probabilities" + ], + "type": "object" + }, + "status": { + "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", + "enum": [ + "importing", + "processing", + "ready", + "failed" + ], + "type": "string" + }, + "transcript": { + "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", + "properties": { + "aligned_by": { + "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", + "type": [ + "string", + "null" + ] + }, + "from": { + "description": "The stretch transcribed, if not all", + "type": "number" + }, + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "text": { + "type": "string" + }, + "until": { + "type": [ + "number", + "null" + ] + }, + "words": { + "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", + "items": { + "properties": { + "confidence": { + "description": "0–1: how sure whisper is of the word itself", + "type": "number" + }, + "end": { + "type": "number" + }, + "speaker": { + "description": "Who says it, from the latest speakers run", + "type": [ + "number", + "null" + ] + }, + "start": { + "type": "number" + }, + "text": { + "type": "string" + }, + "timing_confidence": { + "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "text", + "start", + "end", + "confidence" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "model", + "language", + "text", + "words" + ], + "type": "object" + }, + "upload": { + "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", + "properties": { + "expires_at": { + "type": "string" + }, + "max_bytes": { + "type": "number" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "Signed: PUT the file here; no other credentials needed", + "type": "string" + } + }, + "required": [ + "upload_id", + "url", + "expires_at", + "max_bytes" + ], + "type": "object" + }, + "url": { + "description": "media.get: a signed download of the file itself, once it has one", + "type": "string" + }, + "warnings": { + "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", + "items": { + "type": "string" + }, + "type": "array" + }, + "width": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "media_id", + "name", + "status", + "kind", + "duration_seconds", + "fps", + "width", + "height", + "has_video", + "has_audio", + "codecs", + "conform", + "analysis" + ], + "type": "object" + }, + { + "properties": { + "hint": { + "type": "string" + }, + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "The media it is making; usable in edits once processing", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_uri": { + "type": "string" + } + }, + "required": [ + "job_id", + "status", + "status_uri", + "hint" + ], + "type": "object" + } +]
- Changed
create.matte1 field changed- changed
Output schema / anyOfPrevious value: -[ - { - "properties": { - "analyses": { - "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", - "properties": { - "beats": { - "properties": { - "beats": { - "type": "number" - }, - "bpm": { - "type": "number" - } - }, - "required": [ - "bpm", - "beats" - ], - "type": "object" - }, - "colour": { - "properties": { - "frames": { - "type": "number" - }, - "regions": { - "items": { - "type": "string" - }, - "type": "array" - } - }, - "required": [ - "frames", - "regions" - ], - "type": "object" - }, - "faces": { - "properties": { - "frames_with_faces": { - "type": "number" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames_with_faces" - ], - "type": "object" - }, - "look": { - "description": "frames and questions: the latest run", - "properties": { - "frames": { - "type": "number" - }, - "questions": { - "items": { - "type": "string" - }, - "type": "array" - }, - "runs": { - "type": "number" - } - }, - "required": [ - "runs", - "frames", - "questions" - ], - "type": "object" - }, - "picture": { - "properties": { - "frames": { - "type": "number" - }, - "from": { - "type": "number" - } - }, - "required": [ - "frames", - "from" - ], - "type": "object" - }, - "sounds": { - "description": "heard: the sounds the latest run found at or above its min_probability", - "properties": { - "heard": { - "items": { - "type": "string" - }, - "type": "array" - }, - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "min_probability", - "heard" - ], - "type": "object" - }, - "speakers": { - "properties": { - "model": { - "type": "string" - }, - "speakers": { - "type": "number" - }, - "turns": { - "type": "number" - } - }, - "required": [ - "model", - "speakers", - "turns" - ], - "type": "object" - }, - "speech": { - "properties": { - "frame_seconds": { - "type": "number" - }, - "frames": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "frames", - "frame_seconds" - ], - "type": "object" - }, - "transcript": { - "properties": { - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "words": { - "description": "How many words; the words themselves are in transcript.words", - "type": "number" - } - }, - "required": [ - "model", - "language", - "words" - ], - "type": "object" - }, - "voice": { - "additionalProperties": {}, - "description": "frames, and the summary media.analyze voice gives", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "type": "object" - }, - "analysis": { - "anyOf": [ - { - "properties": { - "channels": { - "description": "Each channel at import", - "items": { - "properties": { - "channel": { - "exclusiveMinimum": 0, - "type": "integer" - }, - "noise_floor_dbfs": { - "type": [ - "number", - "null" - ] - }, - "peak_dbfs": { - "type": [ - "number", - "null" - ] - }, - "rms_dbfs": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "channel", - "rms_dbfs", - "peak_dbfs", - "noise_floor_dbfs" - ], - "type": "object" - }, - "type": "array" - }, - "loudness_lufs": { - "description": "Integrated loudness at import", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "loudness_lufs" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "beats": { - "properties": { - "beats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "bpm": { - "type": "number" - }, - "downbeats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "strengths": { - "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "bpm", - "beats", - "downbeats" - ], - "type": "object" - }, - "codecs": { - "properties": { - "audio": { - "type": [ - "string", - "null" - ] - }, - "video": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "video", - "audio" - ], - "type": "object" - }, - "conform": { - "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", - "items": { - "type": "string" - }, - "type": "array" - }, - "created_at": { - "description": "When it was imported or made", - "type": "string" - }, - "duration_seconds": { - "type": [ - "number", - "null" - ] - }, - "error": { - "properties": { - "code": { - "enum": [ - "bad_request", - "no_audio", - "not_found", - "expired", - "media_not_ready", - "media_offline", - "engine_error", - "render_failed", - "interrupted", - "stalled", - "cancelled", - "credential_required", - "credential_invalid", - "output_exists", - "path_not_allowed", - "limit_exceeded", - "quota_exceeded", - "subscription_required", - "email_unverified", - "service_unavailable", - "server_busy" - ], - "type": "string" - }, - "detail": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "hint": { - "type": "string" - }, - "message": { - "type": "string" - } - }, - "required": [ - "code", - "message" - ], - "type": "object" - }, - "expires_at": { - "description": "When url and page_url stop working", - "type": "string" - }, - "extension": { - "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", - "type": "string" - }, - "faces": { - "properties": { - "frames": { - "items": { - "properties": { - "faces": { - "items": { - "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", - "properties": { - "h": { - "type": "number" - }, - "score": { - "type": "number" - }, - "w": { - "type": "number" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "x", - "y", - "w", - "h", - "score" - ], - "type": "object" - }, - "type": "array" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "faces" - ], - "type": "object" - }, - "type": "array" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames" - ], - "type": "object" - }, - "fingerprint": { - "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", - "type": [ - "string", - "null" - ] - }, - "font_family": { - "description": "Fonts: the family; use font_media_id in a text style to set text in it", - "type": "string" - }, - "fps": { - "anyOf": [ - { - "properties": { - "den": { - "type": "number" - }, - "num": { - "type": "number" - } - }, - "required": [ - "num", - "den" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "has_audio": { - "type": "boolean" - }, - "has_video": { - "type": "boolean" - }, - "height": { - "type": [ - "number", - "null" - ] - }, - "job": { - "description": "media.get, while importing or being made: the job doing it", - "properties": { - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "progress": { - "type": "number" - }, - "seconds_left": { - "type": "number" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_message": { - "type": "string" - }, - "tool": { - "type": "string" - } - }, - "required": [ - "job_id", - "tool", - "status" - ], - "type": "object" - }, - "kind": { - "anyOf": [ - { - "enum": [ - "video", - "audio", - "image", - "font", - "lut" - ], - "type": "string" - }, - { - "type": "null" - } - ] - }, - "look": { - "properties": { - "runs": { - "items": { - "properties": { - "frames": { - "items": { - "properties": { - "answers": { - "additionalProperties": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "answers" - ], - "type": "object" - }, - "type": "array" - }, - "id": { - "type": "string" - }, - "model": { - "type": "string" - }, - "questions": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "state": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "with_previous": { - "type": "boolean" - } - }, - "required": [ - "id", - "model", - "questions", - "with_previous", - "frames" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "runs" - ], - "type": "object" - }, - "media_id": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "mime_type": { - "description": "Its MIME type, likewise", - "type": "string" - }, - "name": { - "type": "string" - }, - "origin": { - "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", - "properties": { - "attached": { - "description": "url: the file the user attached in the chat, passed as media.import file", - "type": "boolean" - }, - "file_name": { - "description": "upload: the name the file was sent with", - "type": "string" - }, - "frame": { - "description": "still: the frame", - "type": "number" - }, - "from": { - "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", - "enum": [ - "upload", - "url", - "path", - "render", - "created", - "still" - ], - "type": "string" - }, - "job_id": { - "description": "render: the export.video job; created: the create.* job", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "still: the video the frame is from", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "media_ids": { - "description": "created: the media it was made from (a cutout's source, a page's assets)", - "items": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "type": "array" - }, - "path": { - "description": "path: the file it was imported from", - "type": "string" - }, - "preset": { - "description": "render: the export preset", - "type": "string" - }, - "range": { - "description": "render: the frames exported, when not the whole timeline", - "properties": { - "end": { - "type": "number" - }, - "start": { - "type": "number" - } - }, - "required": [ - "start", - "end" - ], - "type": "object" - }, - "recipe": { - "description": "created: a hash of what it was made from; the same recipe makes the same media", - "type": "string" - }, - "sources": { - "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", - "items": { - "properties": { - "end": { - "type": "number" - }, - "kind": { - "enum": [ - "video", - "audio" - ], - "type": "string" - }, - "media_id": { - "description": "The media this stretch plays", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "source_in": { - "description": "Seconds into that media", - "type": "number" - }, - "source_out": { - "type": "number" - }, - "speed": { - "additionalProperties": {}, - "description": "The clip's speed change, as clip.set_speed set it", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "start": { - "description": "Seconds into this file", - "type": "number" - }, - "track_id": { - "description": "Track of the timeline; outer/inner inside a nested one", - "type": "string" - } - }, - "required": [ - "track_id", - "kind", - "start", - "end", - "media_id", - "source_in", - "source_out" - ], - "type": "object" - }, - "type": "array" - }, - "timeline_id": { - "description": "render: the timeline that was exported; still: the edit the frame is from", - "pattern": "^tl_[0-9a-f]{32}$", - "type": "string" - }, - "tool": { - "description": "created: the create.* tool that made it", - "type": "string" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "url: the address it was downloaded from, without its query string", - "type": "string" - }, - "via": { - "description": "upload: sent through a media.upload link, dropped on the account page, or a data: URI", - "enum": [ - "media.upload", - "account_page", - "data_uri" - ], - "type": "string" - } - }, - "required": [ - "from" - ], - "type": "object" - }, - "page_url": { - "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", - "type": "string" - }, - "pages": { - "additionalProperties": { - "properties": { - "first": { - "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", - "type": "integer" - }, - "list": { - "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", - "type": "string" - }, - "matching": { - "description": "Items in the time range (the whole list without one)", - "type": "integer" - }, - "next_offset": { - "anyOf": [ - { - "type": "integer" - }, - { - "type": "null" - } - ], - "description": "The offset that gives the next page of this range, or null at its end" - }, - "returned": { - "type": "integer" - }, - "total": { - "description": "Items in the whole list", - "type": "integer" - } - }, - "required": [ - "list", - "first", - "returned", - "matching", - "total", - "next_offset" - ], - "type": "object" - }, - "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", - "propertyNames": { - "enum": [ - "transcript", - "beats", - "faces", - "look", - "speech", - "sounds", - "speakers", - "voice", - "colour", - "picture" - ], - "type": "string" - }, - "type": "object" - }, - "probe": { - "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", - "properties": { - "format": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "streams": { - "items": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "format", - "streams" - ], - "type": "object" - }, - "sound_seconds": { - "description": "How long the sound runs; under duration_seconds, the end is soundless", - "type": "number" - }, - "sounds": { - "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", - "properties": { - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - }, - "peaks": { - "additionalProperties": { - "type": "number" - }, - "description": "Per sound reported: the highest probability it reaches anywhere", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "scores": { - "description": "Every probability, for your own cut-off or arithmetic", - "properties": { - "start_seconds": { - "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", - "type": "number" - }, - "step_seconds": { - "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", - "type": "number" - }, - "values": { - "additionalProperties": { - "items": { - "type": "number" - }, - "type": "array" - }, - "description": "Per sound, the highest probability in each step, 0 to 1", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "start_seconds", - "step_seconds", - "values" - ], - "type": "object" - }, - "spans": { - "additionalProperties": { - "items": { - "properties": { - "end": { - "description": "Where it ends, on the same clock", - "type": "number" - }, - "peak_probability": { - "type": "number" - }, - "peak_seconds": { - "description": "The middle of the most likely frame", - "type": "number" - }, - "start": { - "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", - "type": "number" - } - }, - "required": [ - "start", - "end", - "peak_seconds", - "peak_probability" - ], - "type": "object" - }, - "type": "array" - }, - "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "model", - "min_probability", - "spans", - "peaks" - ], - "type": "object" - }, - "speech": { - "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", - "properties": { - "frame_seconds": { - "description": "Each probability covers this long", - "type": "number" - }, - "from": { - "description": "Seconds where the first probability starts", - "type": "number" - }, - "model": { - "type": "string" - }, - "probabilities": { - "description": "Speech probability 0–1, one per frame, in order", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "model", - "frame_seconds", - "from", - "probabilities" - ], - "type": "object" - }, - "status": { - "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", - "enum": [ - "importing", - "processing", - "ready", - "failed" - ], - "type": "string" - }, - "transcript": { - "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", - "properties": { - "aligned_by": { - "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", - "type": [ - "string", - "null" - ] - }, - "from": { - "description": "The stretch transcribed, if not all", - "type": "number" - }, - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "text": { - "type": "string" - }, - "until": { - "type": [ - "number", - "null" - ] - }, - "words": { - "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", - "items": { - "properties": { - "confidence": { - "description": "0–1: how sure whisper is of the word itself", - "type": "number" - }, - "end": { - "type": "number" - }, - "speaker": { - "description": "Who says it, from the latest speakers run", - "type": [ - "number", - "null" - ] - }, - "start": { - "type": "number" - }, - "text": { - "type": "string" - }, - "timing_confidence": { - "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "text", - "start", - "end", - "confidence" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "model", - "language", - "text", - "words" - ], - "type": "object" - }, - "upload": { - "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", - "properties": { - "expires_at": { - "type": "string" - }, - "max_bytes": { - "type": "number" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "Signed: PUT the file here; no other credentials needed", - "type": "string" - } - }, - "required": [ - "upload_id", - "url", - "expires_at", - "max_bytes" - ], - "type": "object" - }, - "url": { - "description": "media.get: a signed download of the file itself, once it has one", - "type": "string" - }, - "warnings": { - "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", - "items": { - "type": "string" - }, - "type": "array" - }, - "width": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "media_id", - "name", - "status", - "kind", - "duration_seconds", - "fps", - "width", - "height", - "has_video", - "has_audio", - "codecs", - "conform", - "analysis" - ], - "type": "object" - }, - { - "properties": { - "hint": { - "type": "string" - }, - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "The media it is making; usable in edits once processing", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_uri": { - "type": "string" - } - }, - "required": [ - "job_id", - "status", - "status_uri", - "hint" - ], - "type": "object" - } -]New value: +[ + { + "properties": { + "analyses": { + "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", + "properties": { + "beats": { + "properties": { + "beats": { + "type": "number" + }, + "bpm": { + "type": "number" + } + }, + "required": [ + "bpm", + "beats" + ], + "type": "object" + }, + "colour": { + "properties": { + "frames": { + "type": "number" + }, + "regions": { + "items": { + "type": "string" + }, + "type": "array" + } + }, + "required": [ + "frames", + "regions" + ], + "type": "object" + }, + "faces": { + "properties": { + "frames_with_faces": { + "type": "number" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames_with_faces" + ], + "type": "object" + }, + "look": { + "description": "frames and questions: the latest run", + "properties": { + "frames": { + "type": "number" + }, + "questions": { + "items": { + "type": "string" + }, + "type": "array" + }, + "runs": { + "type": "number" + } + }, + "required": [ + "runs", + "frames", + "questions" + ], + "type": "object" + }, + "picture": { + "properties": { + "frames": { + "type": "number" + }, + "from": { + "type": "number" + } + }, + "required": [ + "frames", + "from" + ], + "type": "object" + }, + "sounds": { + "description": "heard: the sounds the latest run found at or above its min_probability", + "properties": { + "heard": { + "items": { + "type": "string" + }, + "type": "array" + }, + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "min_probability", + "heard" + ], + "type": "object" + }, + "speakers": { + "properties": { + "model": { + "type": "string" + }, + "speakers": { + "type": "number" + }, + "turns": { + "type": "number" + } + }, + "required": [ + "model", + "speakers", + "turns" + ], + "type": "object" + }, + "speech": { + "properties": { + "frame_seconds": { + "type": "number" + }, + "frames": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "frames", + "frame_seconds" + ], + "type": "object" + }, + "transcript": { + "properties": { + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "words": { + "description": "How many words; the words themselves are in transcript.words", + "type": "number" + } + }, + "required": [ + "model", + "language", + "words" + ], + "type": "object" + }, + "voice": { + "additionalProperties": {}, + "description": "frames, and the summary media.analyze voice gives", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "type": "object" + }, + "analysis": { + "anyOf": [ + { + "properties": { + "channels": { + "description": "Each channel at import", + "items": { + "properties": { + "channel": { + "exclusiveMinimum": 0, + "type": "integer" + }, + "noise_floor_dbfs": { + "type": [ + "number", + "null" + ] + }, + "peak_dbfs": { + "type": [ + "number", + "null" + ] + }, + "rms_dbfs": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "channel", + "rms_dbfs", + "peak_dbfs", + "noise_floor_dbfs" + ], + "type": "object" + }, + "type": "array" + }, + "loudness_lufs": { + "description": "Integrated loudness at import", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "loudness_lufs" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "beats": { + "properties": { + "beats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "bpm": { + "type": "number" + }, + "downbeats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "strengths": { + "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "bpm", + "beats", + "downbeats" + ], + "type": "object" + }, + "codecs": { + "properties": { + "audio": { + "type": [ + "string", + "null" + ] + }, + "video": { + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "video", + "audio" + ], + "type": "object" + }, + "conform": { + "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", + "items": { + "type": "string" + }, + "type": "array" + }, + "created_at": { + "description": "When it was imported or made", + "type": "string" + }, + "duration_seconds": { + "type": [ + "number", + "null" + ] + }, + "error": { + "properties": { + "code": { + "enum": [ + "bad_request", + "no_audio", + "not_found", + "expired", + "media_not_ready", + "media_offline", + "engine_error", + "render_failed", + "interrupted", + "stalled", + "cancelled", + "credential_required", + "credential_invalid", + "output_exists", + "path_not_allowed", + "limit_exceeded", + "quota_exceeded", + "subscription_required", + "email_unverified", + "service_unavailable", + "server_busy" + ], + "type": "string" + }, + "detail": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "hint": { + "type": "string" + }, + "message": { + "type": "string" + } + }, + "required": [ + "code", + "message" + ], + "type": "object" + }, + "expires_at": { + "description": "When url and page_url stop working", + "type": "string" + }, + "extension": { + "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", + "type": "string" + }, + "faces": { + "properties": { + "frames": { + "items": { + "properties": { + "faces": { + "items": { + "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", + "properties": { + "h": { + "type": "number" + }, + "score": { + "type": "number" + }, + "w": { + "type": "number" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "x", + "y", + "w", + "h", + "score" + ], + "type": "object" + }, + "type": "array" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "faces" + ], + "type": "object" + }, + "type": "array" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames" + ], + "type": "object" + }, + "fingerprint": { + "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", + "type": [ + "string", + "null" + ] + }, + "font_family": { + "description": "Fonts: the family; use font_media_id in a text style to set text in it", + "type": "string" + }, + "fps": { + "anyOf": [ + { + "properties": { + "den": { + "type": "number" + }, + "num": { + "type": "number" + } + }, + "required": [ + "num", + "den" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "has_audio": { + "type": "boolean" + }, + "has_video": { + "type": "boolean" + }, + "height": { + "type": [ + "number", + "null" + ] + }, + "job": { + "description": "media.get, while importing or being made: the job doing it", + "properties": { + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "progress": { + "type": "number" + }, + "seconds_left": { + "type": "number" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_message": { + "type": "string" + }, + "tool": { + "type": "string" + } + }, + "required": [ + "job_id", + "tool", + "status" + ], + "type": "object" + }, + "kind": { + "anyOf": [ + { + "enum": [ + "video", + "audio", + "image", + "font", + "lut" + ], + "type": "string" + }, + { + "type": "null" + } + ] + }, + "look": { + "properties": { + "runs": { + "items": { + "properties": { + "frames": { + "items": { + "properties": { + "answers": { + "additionalProperties": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "answers" + ], + "type": "object" + }, + "type": "array" + }, + "id": { + "type": "string" + }, + "model": { + "type": "string" + }, + "questions": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "state": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "with_previous": { + "type": "boolean" + } + }, + "required": [ + "id", + "model", + "questions", + "with_previous", + "frames" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "runs" + ], + "type": "object" + }, + "media_id": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "mime_type": { + "description": "Its MIME type, likewise", + "type": "string" + }, + "name": { + "type": "string" + }, + "origin": { + "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", + "properties": { + "attached": { + "description": "url: the file the user attached in the chat, passed as media.import file", + "type": "boolean" + }, + "file_name": { + "description": "upload: the name the file was sent with", + "type": "string" + }, + "frame": { + "description": "still: the frame", + "type": "number" + }, + "from": { + "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", + "enum": [ + "upload", + "url", + "path", + "render", + "created", + "still" + ], + "type": "string" + }, + "job_id": { + "description": "render: the export.video job; created: the create.* job", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "still: the video the frame is from", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "media_ids": { + "description": "created: the media it was made from (a cutout's source, a page's assets)", + "items": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "type": "array" + }, + "path": { + "description": "path: the file it was imported from", + "type": "string" + }, + "preset": { + "description": "render: the export preset", + "type": "string" + }, + "range": { + "description": "render: the frames exported, when not the whole timeline", + "properties": { + "end": { + "type": "number" + }, + "start": { + "type": "number" + } + }, + "required": [ + "start", + "end" + ], + "type": "object" + }, + "recipe": { + "description": "created: a hash of what it was made from; the same recipe makes the same media", + "type": "string" + }, + "sources": { + "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", + "items": { + "properties": { + "end": { + "type": "number" + }, + "kind": { + "enum": [ + "video", + "audio" + ], + "type": "string" + }, + "media_id": { + "description": "The media this stretch plays", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "source_in": { + "description": "Seconds into that media", + "type": "number" + }, + "source_out": { + "type": "number" + }, + "speed": { + "additionalProperties": {}, + "description": "The clip's speed change, as clip.set_speed set it", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "start": { + "description": "Seconds into this file", + "type": "number" + }, + "track_id": { + "description": "Track of the timeline; outer/inner inside a nested one", + "type": "string" + } + }, + "required": [ + "track_id", + "kind", + "start", + "end", + "media_id", + "source_in", + "source_out" + ], + "type": "object" + }, + "type": "array" + }, + "timeline_id": { + "description": "render: the timeline that was exported; still: the edit the frame is from", + "pattern": "^tl_[0-9a-f]{32}$", + "type": "string" + }, + "tool": { + "description": "created: the create.* tool that made it", + "type": "string" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "url: the address it was downloaded from, without its query string", + "type": "string" + }, + "via": { + "description": "upload: sent through a media.upload link, dropped on the cuttingfloor site, or a data: URI", + "enum": [ + "media.upload", + "account_page", + "data_uri" + ], + "type": "string" + } + }, + "required": [ + "from" + ], + "type": "object" + }, + "page_url": { + "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", + "type": "string" + }, + "pages": { + "additionalProperties": { + "properties": { + "first": { + "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", + "type": "integer" + }, + "list": { + "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", + "type": "string" + }, + "matching": { + "description": "Items in the time range (the whole list without one)", + "type": "integer" + }, + "next_offset": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "The offset that gives the next page of this range, or null at its end" + }, + "returned": { + "type": "integer" + }, + "total": { + "description": "Items in the whole list", + "type": "integer" + } + }, + "required": [ + "list", + "first", + "returned", + "matching", + "total", + "next_offset" + ], + "type": "object" + }, + "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", + "propertyNames": { + "enum": [ + "transcript", + "beats", + "faces", + "look", + "speech", + "sounds", + "speakers", + "voice", + "colour", + "picture" + ], + "type": "string" + }, + "type": "object" + }, + "probe": { + "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", + "properties": { + "format": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "streams": { + "items": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "format", + "streams" + ], + "type": "object" + }, + "sound_seconds": { + "description": "How long the sound runs; under duration_seconds, the end is soundless", + "type": "number" + }, + "sounds": { + "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", + "properties": { + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + }, + "peaks": { + "additionalProperties": { + "type": "number" + }, + "description": "Per sound reported: the highest probability it reaches anywhere", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "scores": { + "description": "Every probability, for your own cut-off or arithmetic", + "properties": { + "start_seconds": { + "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", + "type": "number" + }, + "step_seconds": { + "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", + "type": "number" + }, + "values": { + "additionalProperties": { + "items": { + "type": "number" + }, + "type": "array" + }, + "description": "Per sound, the highest probability in each step, 0 to 1", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "start_seconds", + "step_seconds", + "values" + ], + "type": "object" + }, + "spans": { + "additionalProperties": { + "items": { + "properties": { + "end": { + "description": "Where it ends, on the same clock", + "type": "number" + }, + "peak_probability": { + "type": "number" + }, + "peak_seconds": { + "description": "The middle of the most likely frame", + "type": "number" + }, + "start": { + "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", + "type": "number" + } + }, + "required": [ + "start", + "end", + "peak_seconds", + "peak_probability" + ], + "type": "object" + }, + "type": "array" + }, + "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "model", + "min_probability", + "spans", + "peaks" + ], + "type": "object" + }, + "speech": { + "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", + "properties": { + "frame_seconds": { + "description": "Each probability covers this long", + "type": "number" + }, + "from": { + "description": "Seconds where the first probability starts", + "type": "number" + }, + "model": { + "type": "string" + }, + "probabilities": { + "description": "Speech probability 0–1, one per frame, in order", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "model", + "frame_seconds", + "from", + "probabilities" + ], + "type": "object" + }, + "status": { + "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", + "enum": [ + "importing", + "processing", + "ready", + "failed" + ], + "type": "string" + }, + "transcript": { + "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", + "properties": { + "aligned_by": { + "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", + "type": [ + "string", + "null" + ] + }, + "from": { + "description": "The stretch transcribed, if not all", + "type": "number" + }, + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "text": { + "type": "string" + }, + "until": { + "type": [ + "number", + "null" + ] + }, + "words": { + "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", + "items": { + "properties": { + "confidence": { + "description": "0–1: how sure whisper is of the word itself", + "type": "number" + }, + "end": { + "type": "number" + }, + "speaker": { + "description": "Who says it, from the latest speakers run", + "type": [ + "number", + "null" + ] + }, + "start": { + "type": "number" + }, + "text": { + "type": "string" + }, + "timing_confidence": { + "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "text", + "start", + "end", + "confidence" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "model", + "language", + "text", + "words" + ], + "type": "object" + }, + "upload": { + "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", + "properties": { + "expires_at": { + "type": "string" + }, + "max_bytes": { + "type": "number" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "Signed: PUT the file here; no other credentials needed", + "type": "string" + } + }, + "required": [ + "upload_id", + "url", + "expires_at", + "max_bytes" + ], + "type": "object" + }, + "url": { + "description": "media.get: a signed download of the file itself, once it has one", + "type": "string" + }, + "warnings": { + "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", + "items": { + "type": "string" + }, + "type": "array" + }, + "width": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "media_id", + "name", + "status", + "kind", + "duration_seconds", + "fps", + "width", + "height", + "has_video", + "has_audio", + "codecs", + "conform", + "analysis" + ], + "type": "object" + }, + { + "properties": { + "hint": { + "type": "string" + }, + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "The media it is making; usable in edits once processing", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_uri": { + "type": "string" + } + }, + "required": [ + "job_id", + "status", + "status_uri", + "hint" + ], + "type": "object" + } +]
- Changed
create.motion1 field changed- changed
Output schema / anyOfPrevious value: -[ - { - "properties": { - "analyses": { - "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", - "properties": { - "beats": { - "properties": { - "beats": { - "type": "number" - }, - "bpm": { - "type": "number" - } - }, - "required": [ - "bpm", - "beats" - ], - "type": "object" - }, - "colour": { - "properties": { - "frames": { - "type": "number" - }, - "regions": { - "items": { - "type": "string" - }, - "type": "array" - } - }, - "required": [ - "frames", - "regions" - ], - "type": "object" - }, - "faces": { - "properties": { - "frames_with_faces": { - "type": "number" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames_with_faces" - ], - "type": "object" - }, - "look": { - "description": "frames and questions: the latest run", - "properties": { - "frames": { - "type": "number" - }, - "questions": { - "items": { - "type": "string" - }, - "type": "array" - }, - "runs": { - "type": "number" - } - }, - "required": [ - "runs", - "frames", - "questions" - ], - "type": "object" - }, - "picture": { - "properties": { - "frames": { - "type": "number" - }, - "from": { - "type": "number" - } - }, - "required": [ - "frames", - "from" - ], - "type": "object" - }, - "sounds": { - "description": "heard: the sounds the latest run found at or above its min_probability", - "properties": { - "heard": { - "items": { - "type": "string" - }, - "type": "array" - }, - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "min_probability", - "heard" - ], - "type": "object" - }, - "speakers": { - "properties": { - "model": { - "type": "string" - }, - "speakers": { - "type": "number" - }, - "turns": { - "type": "number" - } - }, - "required": [ - "model", - "speakers", - "turns" - ], - "type": "object" - }, - "speech": { - "properties": { - "frame_seconds": { - "type": "number" - }, - "frames": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "frames", - "frame_seconds" - ], - "type": "object" - }, - "transcript": { - "properties": { - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "words": { - "description": "How many words; the words themselves are in transcript.words", - "type": "number" - } - }, - "required": [ - "model", - "language", - "words" - ], - "type": "object" - }, - "voice": { - "additionalProperties": {}, - "description": "frames, and the summary media.analyze voice gives", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "type": "object" - }, - "analysis": { - "anyOf": [ - { - "properties": { - "channels": { - "description": "Each channel at import", - "items": { - "properties": { - "channel": { - "exclusiveMinimum": 0, - "type": "integer" - }, - "noise_floor_dbfs": { - "type": [ - "number", - "null" - ] - }, - "peak_dbfs": { - "type": [ - "number", - "null" - ] - }, - "rms_dbfs": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "channel", - "rms_dbfs", - "peak_dbfs", - "noise_floor_dbfs" - ], - "type": "object" - }, - "type": "array" - }, - "loudness_lufs": { - "description": "Integrated loudness at import", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "loudness_lufs" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "beats": { - "properties": { - "beats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "bpm": { - "type": "number" - }, - "downbeats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "strengths": { - "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "bpm", - "beats", - "downbeats" - ], - "type": "object" - }, - "codecs": { - "properties": { - "audio": { - "type": [ - "string", - "null" - ] - }, - "video": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "video", - "audio" - ], - "type": "object" - }, - "conform": { - "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", - "items": { - "type": "string" - }, - "type": "array" - }, - "created_at": { - "description": "When it was imported or made", - "type": "string" - }, - "duration_seconds": { - "type": [ - "number", - "null" - ] - }, - "error": { - "properties": { - "code": { - "enum": [ - "bad_request", - "no_audio", - "not_found", - "expired", - "media_not_ready", - "media_offline", - "engine_error", - "render_failed", - "interrupted", - "stalled", - "cancelled", - "credential_required", - "credential_invalid", - "output_exists", - "path_not_allowed", - "limit_exceeded", - "quota_exceeded", - "subscription_required", - "email_unverified", - "service_unavailable", - "server_busy" - ], - "type": "string" - }, - "detail": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "hint": { - "type": "string" - }, - "message": { - "type": "string" - } - }, - "required": [ - "code", - "message" - ], - "type": "object" - }, - "expires_at": { - "description": "When url and page_url stop working", - "type": "string" - }, - "extension": { - "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", - "type": "string" - }, - "faces": { - "properties": { - "frames": { - "items": { - "properties": { - "faces": { - "items": { - "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", - "properties": { - "h": { - "type": "number" - }, - "score": { - "type": "number" - }, - "w": { - "type": "number" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "x", - "y", - "w", - "h", - "score" - ], - "type": "object" - }, - "type": "array" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "faces" - ], - "type": "object" - }, - "type": "array" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames" - ], - "type": "object" - }, - "fingerprint": { - "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", - "type": [ - "string", - "null" - ] - }, - "font_family": { - "description": "Fonts: the family; use font_media_id in a text style to set text in it", - "type": "string" - }, - "fps": { - "anyOf": [ - { - "properties": { - "den": { - "type": "number" - }, - "num": { - "type": "number" - } - }, - "required": [ - "num", - "den" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "has_audio": { - "type": "boolean" - }, - "has_video": { - "type": "boolean" - }, - "height": { - "type": [ - "number", - "null" - ] - }, - "job": { - "description": "media.get, while importing or being made: the job doing it", - "properties": { - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "progress": { - "type": "number" - }, - "seconds_left": { - "type": "number" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_message": { - "type": "string" - }, - "tool": { - "type": "string" - } - }, - "required": [ - "job_id", - "tool", - "status" - ], - "type": "object" - }, - "kind": { - "anyOf": [ - { - "enum": [ - "video", - "audio", - "image", - "font", - "lut" - ], - "type": "string" - }, - { - "type": "null" - } - ] - }, - "look": { - "properties": { - "runs": { - "items": { - "properties": { - "frames": { - "items": { - "properties": { - "answers": { - "additionalProperties": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "answers" - ], - "type": "object" - }, - "type": "array" - }, - "id": { - "type": "string" - }, - "model": { - "type": "string" - }, - "questions": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "state": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "with_previous": { - "type": "boolean" - } - }, - "required": [ - "id", - "model", - "questions", - "with_previous", - "frames" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "runs" - ], - "type": "object" - }, - "media_id": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "mime_type": { - "description": "Its MIME type, likewise", - "type": "string" - }, - "name": { - "type": "string" - }, - "origin": { - "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", - "properties": { - "attached": { - "description": "url: the file the user attached in the chat, passed as media.import file", - "type": "boolean" - }, - "file_name": { - "description": "upload: the name the file was sent with", - "type": "string" - }, - "frame": { - "description": "still: the frame", - "type": "number" - }, - "from": { - "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", - "enum": [ - "upload", - "url", - "path", - "render", - "created", - "still" - ], - "type": "string" - }, - "job_id": { - "description": "render: the export.video job; created: the create.* job", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "still: the video the frame is from", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "media_ids": { - "description": "created: the media it was made from (a cutout's source, a page's assets)", - "items": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "type": "array" - }, - "path": { - "description": "path: the file it was imported from", - "type": "string" - }, - "preset": { - "description": "render: the export preset", - "type": "string" - }, - "range": { - "description": "render: the frames exported, when not the whole timeline", - "properties": { - "end": { - "type": "number" - }, - "start": { - "type": "number" - } - }, - "required": [ - "start", - "end" - ], - "type": "object" - }, - "recipe": { - "description": "created: a hash of what it was made from; the same recipe makes the same media", - "type": "string" - }, - "sources": { - "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", - "items": { - "properties": { - "end": { - "type": "number" - }, - "kind": { - "enum": [ - "video", - "audio" - ], - "type": "string" - }, - "media_id": { - "description": "The media this stretch plays", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "source_in": { - "description": "Seconds into that media", - "type": "number" - }, - "source_out": { - "type": "number" - }, - "speed": { - "additionalProperties": {}, - "description": "The clip's speed change, as clip.set_speed set it", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "start": { - "description": "Seconds into this file", - "type": "number" - }, - "track_id": { - "description": "Track of the timeline; outer/inner inside a nested one", - "type": "string" - } - }, - "required": [ - "track_id", - "kind", - "start", - "end", - "media_id", - "source_in", - "source_out" - ], - "type": "object" - }, - "type": "array" - }, - "timeline_id": { - "description": "render: the timeline that was exported; still: the edit the frame is from", - "pattern": "^tl_[0-9a-f]{32}$", - "type": "string" - }, - "tool": { - "description": "created: the create.* tool that made it", - "type": "string" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "url: the address it was downloaded from, without its query string", - "type": "string" - }, - "via": { - "description": "upload: sent through a media.upload link, dropped on the account page, or a data: URI", - "enum": [ - "media.upload", - "account_page", - "data_uri" - ], - "type": "string" - } - }, - "required": [ - "from" - ], - "type": "object" - }, - "page_url": { - "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", - "type": "string" - }, - "pages": { - "additionalProperties": { - "properties": { - "first": { - "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", - "type": "integer" - }, - "list": { - "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", - "type": "string" - }, - "matching": { - "description": "Items in the time range (the whole list without one)", - "type": "integer" - }, - "next_offset": { - "anyOf": [ - { - "type": "integer" - }, - { - "type": "null" - } - ], - "description": "The offset that gives the next page of this range, or null at its end" - }, - "returned": { - "type": "integer" - }, - "total": { - "description": "Items in the whole list", - "type": "integer" - } - }, - "required": [ - "list", - "first", - "returned", - "matching", - "total", - "next_offset" - ], - "type": "object" - }, - "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", - "propertyNames": { - "enum": [ - "transcript", - "beats", - "faces", - "look", - "speech", - "sounds", - "speakers", - "voice", - "colour", - "picture" - ], - "type": "string" - }, - "type": "object" - }, - "probe": { - "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", - "properties": { - "format": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "streams": { - "items": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "format", - "streams" - ], - "type": "object" - }, - "sound_seconds": { - "description": "How long the sound runs; under duration_seconds, the end is soundless", - "type": "number" - }, - "sounds": { - "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", - "properties": { - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - }, - "peaks": { - "additionalProperties": { - "type": "number" - }, - "description": "Per sound reported: the highest probability it reaches anywhere", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "scores": { - "description": "Every probability, for your own cut-off or arithmetic", - "properties": { - "start_seconds": { - "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", - "type": "number" - }, - "step_seconds": { - "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", - "type": "number" - }, - "values": { - "additionalProperties": { - "items": { - "type": "number" - }, - "type": "array" - }, - "description": "Per sound, the highest probability in each step, 0 to 1", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "start_seconds", - "step_seconds", - "values" - ], - "type": "object" - }, - "spans": { - "additionalProperties": { - "items": { - "properties": { - "end": { - "description": "Where it ends, on the same clock", - "type": "number" - }, - "peak_probability": { - "type": "number" - }, - "peak_seconds": { - "description": "The middle of the most likely frame", - "type": "number" - }, - "start": { - "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", - "type": "number" - } - }, - "required": [ - "start", - "end", - "peak_seconds", - "peak_probability" - ], - "type": "object" - }, - "type": "array" - }, - "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "model", - "min_probability", - "spans", - "peaks" - ], - "type": "object" - }, - "speech": { - "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", - "properties": { - "frame_seconds": { - "description": "Each probability covers this long", - "type": "number" - }, - "from": { - "description": "Seconds where the first probability starts", - "type": "number" - }, - "model": { - "type": "string" - }, - "probabilities": { - "description": "Speech probability 0–1, one per frame, in order", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "model", - "frame_seconds", - "from", - "probabilities" - ], - "type": "object" - }, - "status": { - "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", - "enum": [ - "importing", - "processing", - "ready", - "failed" - ], - "type": "string" - }, - "transcript": { - "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", - "properties": { - "aligned_by": { - "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", - "type": [ - "string", - "null" - ] - }, - "from": { - "description": "The stretch transcribed, if not all", - "type": "number" - }, - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "text": { - "type": "string" - }, - "until": { - "type": [ - "number", - "null" - ] - }, - "words": { - "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", - "items": { - "properties": { - "confidence": { - "description": "0–1: how sure whisper is of the word itself", - "type": "number" - }, - "end": { - "type": "number" - }, - "speaker": { - "description": "Who says it, from the latest speakers run", - "type": [ - "number", - "null" - ] - }, - "start": { - "type": "number" - }, - "text": { - "type": "string" - }, - "timing_confidence": { - "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "text", - "start", - "end", - "confidence" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "model", - "language", - "text", - "words" - ], - "type": "object" - }, - "upload": { - "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", - "properties": { - "expires_at": { - "type": "string" - }, - "max_bytes": { - "type": "number" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "Signed: PUT the file here; no other credentials needed", - "type": "string" - } - }, - "required": [ - "upload_id", - "url", - "expires_at", - "max_bytes" - ], - "type": "object" - }, - "url": { - "description": "media.get: a signed download of the file itself, once it has one", - "type": "string" - }, - "warnings": { - "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", - "items": { - "type": "string" - }, - "type": "array" - }, - "width": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "media_id", - "name", - "status", - "kind", - "duration_seconds", - "fps", - "width", - "height", - "has_video", - "has_audio", - "codecs", - "conform", - "analysis" - ], - "type": "object" - }, - { - "properties": { - "hint": { - "type": "string" - }, - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "The media it is making; usable in edits once processing", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_uri": { - "type": "string" - } - }, - "required": [ - "job_id", - "status", - "status_uri", - "hint" - ], - "type": "object" - } -]New value: +[ + { + "properties": { + "analyses": { + "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", + "properties": { + "beats": { + "properties": { + "beats": { + "type": "number" + }, + "bpm": { + "type": "number" + } + }, + "required": [ + "bpm", + "beats" + ], + "type": "object" + }, + "colour": { + "properties": { + "frames": { + "type": "number" + }, + "regions": { + "items": { + "type": "string" + }, + "type": "array" + } + }, + "required": [ + "frames", + "regions" + ], + "type": "object" + }, + "faces": { + "properties": { + "frames_with_faces": { + "type": "number" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames_with_faces" + ], + "type": "object" + }, + "look": { + "description": "frames and questions: the latest run", + "properties": { + "frames": { + "type": "number" + }, + "questions": { + "items": { + "type": "string" + }, + "type": "array" + }, + "runs": { + "type": "number" + } + }, + "required": [ + "runs", + "frames", + "questions" + ], + "type": "object" + }, + "picture": { + "properties": { + "frames": { + "type": "number" + }, + "from": { + "type": "number" + } + }, + "required": [ + "frames", + "from" + ], + "type": "object" + }, + "sounds": { + "description": "heard: the sounds the latest run found at or above its min_probability", + "properties": { + "heard": { + "items": { + "type": "string" + }, + "type": "array" + }, + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "min_probability", + "heard" + ], + "type": "object" + }, + "speakers": { + "properties": { + "model": { + "type": "string" + }, + "speakers": { + "type": "number" + }, + "turns": { + "type": "number" + } + }, + "required": [ + "model", + "speakers", + "turns" + ], + "type": "object" + }, + "speech": { + "properties": { + "frame_seconds": { + "type": "number" + }, + "frames": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "frames", + "frame_seconds" + ], + "type": "object" + }, + "transcript": { + "properties": { + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "words": { + "description": "How many words; the words themselves are in transcript.words", + "type": "number" + } + }, + "required": [ + "model", + "language", + "words" + ], + "type": "object" + }, + "voice": { + "additionalProperties": {}, + "description": "frames, and the summary media.analyze voice gives", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "type": "object" + }, + "analysis": { + "anyOf": [ + { + "properties": { + "channels": { + "description": "Each channel at import", + "items": { + "properties": { + "channel": { + "exclusiveMinimum": 0, + "type": "integer" + }, + "noise_floor_dbfs": { + "type": [ + "number", + "null" + ] + }, + "peak_dbfs": { + "type": [ + "number", + "null" + ] + }, + "rms_dbfs": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "channel", + "rms_dbfs", + "peak_dbfs", + "noise_floor_dbfs" + ], + "type": "object" + }, + "type": "array" + }, + "loudness_lufs": { + "description": "Integrated loudness at import", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "loudness_lufs" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "beats": { + "properties": { + "beats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "bpm": { + "type": "number" + }, + "downbeats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "strengths": { + "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "bpm", + "beats", + "downbeats" + ], + "type": "object" + }, + "codecs": { + "properties": { + "audio": { + "type": [ + "string", + "null" + ] + }, + "video": { + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "video", + "audio" + ], + "type": "object" + }, + "conform": { + "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", + "items": { + "type": "string" + }, + "type": "array" + }, + "created_at": { + "description": "When it was imported or made", + "type": "string" + }, + "duration_seconds": { + "type": [ + "number", + "null" + ] + }, + "error": { + "properties": { + "code": { + "enum": [ + "bad_request", + "no_audio", + "not_found", + "expired", + "media_not_ready", + "media_offline", + "engine_error", + "render_failed", + "interrupted", + "stalled", + "cancelled", + "credential_required", + "credential_invalid", + "output_exists", + "path_not_allowed", + "limit_exceeded", + "quota_exceeded", + "subscription_required", + "email_unverified", + "service_unavailable", + "server_busy" + ], + "type": "string" + }, + "detail": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "hint": { + "type": "string" + }, + "message": { + "type": "string" + } + }, + "required": [ + "code", + "message" + ], + "type": "object" + }, + "expires_at": { + "description": "When url and page_url stop working", + "type": "string" + }, + "extension": { + "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", + "type": "string" + }, + "faces": { + "properties": { + "frames": { + "items": { + "properties": { + "faces": { + "items": { + "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", + "properties": { + "h": { + "type": "number" + }, + "score": { + "type": "number" + }, + "w": { + "type": "number" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "x", + "y", + "w", + "h", + "score" + ], + "type": "object" + }, + "type": "array" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "faces" + ], + "type": "object" + }, + "type": "array" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames" + ], + "type": "object" + }, + "fingerprint": { + "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", + "type": [ + "string", + "null" + ] + }, + "font_family": { + "description": "Fonts: the family; use font_media_id in a text style to set text in it", + "type": "string" + }, + "fps": { + "anyOf": [ + { + "properties": { + "den": { + "type": "number" + }, + "num": { + "type": "number" + } + }, + "required": [ + "num", + "den" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "has_audio": { + "type": "boolean" + }, + "has_video": { + "type": "boolean" + }, + "height": { + "type": [ + "number", + "null" + ] + }, + "job": { + "description": "media.get, while importing or being made: the job doing it", + "properties": { + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "progress": { + "type": "number" + }, + "seconds_left": { + "type": "number" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_message": { + "type": "string" + }, + "tool": { + "type": "string" + } + }, + "required": [ + "job_id", + "tool", + "status" + ], + "type": "object" + }, + "kind": { + "anyOf": [ + { + "enum": [ + "video", + "audio", + "image", + "font", + "lut" + ], + "type": "string" + }, + { + "type": "null" + } + ] + }, + "look": { + "properties": { + "runs": { + "items": { + "properties": { + "frames": { + "items": { + "properties": { + "answers": { + "additionalProperties": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "answers" + ], + "type": "object" + }, + "type": "array" + }, + "id": { + "type": "string" + }, + "model": { + "type": "string" + }, + "questions": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "state": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "with_previous": { + "type": "boolean" + } + }, + "required": [ + "id", + "model", + "questions", + "with_previous", + "frames" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "runs" + ], + "type": "object" + }, + "media_id": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "mime_type": { + "description": "Its MIME type, likewise", + "type": "string" + }, + "name": { + "type": "string" + }, + "origin": { + "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", + "properties": { + "attached": { + "description": "url: the file the user attached in the chat, passed as media.import file", + "type": "boolean" + }, + "file_name": { + "description": "upload: the name the file was sent with", + "type": "string" + }, + "frame": { + "description": "still: the frame", + "type": "number" + }, + "from": { + "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", + "enum": [ + "upload", + "url", + "path", + "render", + "created", + "still" + ], + "type": "string" + }, + "job_id": { + "description": "render: the export.video job; created: the create.* job", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "still: the video the frame is from", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "media_ids": { + "description": "created: the media it was made from (a cutout's source, a page's assets)", + "items": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "type": "array" + }, + "path": { + "description": "path: the file it was imported from", + "type": "string" + }, + "preset": { + "description": "render: the export preset", + "type": "string" + }, + "range": { + "description": "render: the frames exported, when not the whole timeline", + "properties": { + "end": { + "type": "number" + }, + "start": { + "type": "number" + } + }, + "required": [ + "start", + "end" + ], + "type": "object" + }, + "recipe": { + "description": "created: a hash of what it was made from; the same recipe makes the same media", + "type": "string" + }, + "sources": { + "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", + "items": { + "properties": { + "end": { + "type": "number" + }, + "kind": { + "enum": [ + "video", + "audio" + ], + "type": "string" + }, + "media_id": { + "description": "The media this stretch plays", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "source_in": { + "description": "Seconds into that media", + "type": "number" + }, + "source_out": { + "type": "number" + }, + "speed": { + "additionalProperties": {}, + "description": "The clip's speed change, as clip.set_speed set it", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "start": { + "description": "Seconds into this file", + "type": "number" + }, + "track_id": { + "description": "Track of the timeline; outer/inner inside a nested one", + "type": "string" + } + }, + "required": [ + "track_id", + "kind", + "start", + "end", + "media_id", + "source_in", + "source_out" + ], + "type": "object" + }, + "type": "array" + }, + "timeline_id": { + "description": "render: the timeline that was exported; still: the edit the frame is from", + "pattern": "^tl_[0-9a-f]{32}$", + "type": "string" + }, + "tool": { + "description": "created: the create.* tool that made it", + "type": "string" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "url: the address it was downloaded from, without its query string", + "type": "string" + }, + "via": { + "description": "upload: sent through a media.upload link, dropped on the cuttingfloor site, or a data: URI", + "enum": [ + "media.upload", + "account_page", + "data_uri" + ], + "type": "string" + } + }, + "required": [ + "from" + ], + "type": "object" + }, + "page_url": { + "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", + "type": "string" + }, + "pages": { + "additionalProperties": { + "properties": { + "first": { + "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", + "type": "integer" + }, + "list": { + "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", + "type": "string" + }, + "matching": { + "description": "Items in the time range (the whole list without one)", + "type": "integer" + }, + "next_offset": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "The offset that gives the next page of this range, or null at its end" + }, + "returned": { + "type": "integer" + }, + "total": { + "description": "Items in the whole list", + "type": "integer" + } + }, + "required": [ + "list", + "first", + "returned", + "matching", + "total", + "next_offset" + ], + "type": "object" + }, + "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", + "propertyNames": { + "enum": [ + "transcript", + "beats", + "faces", + "look", + "speech", + "sounds", + "speakers", + "voice", + "colour", + "picture" + ], + "type": "string" + }, + "type": "object" + }, + "probe": { + "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", + "properties": { + "format": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "streams": { + "items": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "format", + "streams" + ], + "type": "object" + }, + "sound_seconds": { + "description": "How long the sound runs; under duration_seconds, the end is soundless", + "type": "number" + }, + "sounds": { + "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", + "properties": { + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + }, + "peaks": { + "additionalProperties": { + "type": "number" + }, + "description": "Per sound reported: the highest probability it reaches anywhere", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "scores": { + "description": "Every probability, for your own cut-off or arithmetic", + "properties": { + "start_seconds": { + "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", + "type": "number" + }, + "step_seconds": { + "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", + "type": "number" + }, + "values": { + "additionalProperties": { + "items": { + "type": "number" + }, + "type": "array" + }, + "description": "Per sound, the highest probability in each step, 0 to 1", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "start_seconds", + "step_seconds", + "values" + ], + "type": "object" + }, + "spans": { + "additionalProperties": { + "items": { + "properties": { + "end": { + "description": "Where it ends, on the same clock", + "type": "number" + }, + "peak_probability": { + "type": "number" + }, + "peak_seconds": { + "description": "The middle of the most likely frame", + "type": "number" + }, + "start": { + "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", + "type": "number" + } + }, + "required": [ + "start", + "end", + "peak_seconds", + "peak_probability" + ], + "type": "object" + }, + "type": "array" + }, + "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "model", + "min_probability", + "spans", + "peaks" + ], + "type": "object" + }, + "speech": { + "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", + "properties": { + "frame_seconds": { + "description": "Each probability covers this long", + "type": "number" + }, + "from": { + "description": "Seconds where the first probability starts", + "type": "number" + }, + "model": { + "type": "string" + }, + "probabilities": { + "description": "Speech probability 0–1, one per frame, in order", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "model", + "frame_seconds", + "from", + "probabilities" + ], + "type": "object" + }, + "status": { + "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", + "enum": [ + "importing", + "processing", + "ready", + "failed" + ], + "type": "string" + }, + "transcript": { + "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", + "properties": { + "aligned_by": { + "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", + "type": [ + "string", + "null" + ] + }, + "from": { + "description": "The stretch transcribed, if not all", + "type": "number" + }, + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "text": { + "type": "string" + }, + "until": { + "type": [ + "number", + "null" + ] + }, + "words": { + "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", + "items": { + "properties": { + "confidence": { + "description": "0–1: how sure whisper is of the word itself", + "type": "number" + }, + "end": { + "type": "number" + }, + "speaker": { + "description": "Who says it, from the latest speakers run", + "type": [ + "number", + "null" + ] + }, + "start": { + "type": "number" + }, + "text": { + "type": "string" + }, + "timing_confidence": { + "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "text", + "start", + "end", + "confidence" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "model", + "language", + "text", + "words" + ], + "type": "object" + }, + "upload": { + "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", + "properties": { + "expires_at": { + "type": "string" + }, + "max_bytes": { + "type": "number" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "Signed: PUT the file here; no other credentials needed", + "type": "string" + } + }, + "required": [ + "upload_id", + "url", + "expires_at", + "max_bytes" + ], + "type": "object" + }, + "url": { + "description": "media.get: a signed download of the file itself, once it has one", + "type": "string" + }, + "warnings": { + "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", + "items": { + "type": "string" + }, + "type": "array" + }, + "width": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "media_id", + "name", + "status", + "kind", + "duration_seconds", + "fps", + "width", + "height", + "has_video", + "has_audio", + "codecs", + "conform", + "analysis" + ], + "type": "object" + }, + { + "properties": { + "hint": { + "type": "string" + }, + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "The media it is making; usable in edits once processing", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_uri": { + "type": "string" + } + }, + "required": [ + "job_id", + "status", + "status_uri", + "hint" + ], + "type": "object" + } +]
- Changed
create.stabilized1 field changed- changed
Output schema / anyOfPrevious value: -[ - { - "properties": { - "analyses": { - "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", - "properties": { - "beats": { - "properties": { - "beats": { - "type": "number" - }, - "bpm": { - "type": "number" - } - }, - "required": [ - "bpm", - "beats" - ], - "type": "object" - }, - "colour": { - "properties": { - "frames": { - "type": "number" - }, - "regions": { - "items": { - "type": "string" - }, - "type": "array" - } - }, - "required": [ - "frames", - "regions" - ], - "type": "object" - }, - "faces": { - "properties": { - "frames_with_faces": { - "type": "number" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames_with_faces" - ], - "type": "object" - }, - "look": { - "description": "frames and questions: the latest run", - "properties": { - "frames": { - "type": "number" - }, - "questions": { - "items": { - "type": "string" - }, - "type": "array" - }, - "runs": { - "type": "number" - } - }, - "required": [ - "runs", - "frames", - "questions" - ], - "type": "object" - }, - "picture": { - "properties": { - "frames": { - "type": "number" - }, - "from": { - "type": "number" - } - }, - "required": [ - "frames", - "from" - ], - "type": "object" - }, - "sounds": { - "description": "heard: the sounds the latest run found at or above its min_probability", - "properties": { - "heard": { - "items": { - "type": "string" - }, - "type": "array" - }, - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "min_probability", - "heard" - ], - "type": "object" - }, - "speakers": { - "properties": { - "model": { - "type": "string" - }, - "speakers": { - "type": "number" - }, - "turns": { - "type": "number" - } - }, - "required": [ - "model", - "speakers", - "turns" - ], - "type": "object" - }, - "speech": { - "properties": { - "frame_seconds": { - "type": "number" - }, - "frames": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "frames", - "frame_seconds" - ], - "type": "object" - }, - "transcript": { - "properties": { - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "words": { - "description": "How many words; the words themselves are in transcript.words", - "type": "number" - } - }, - "required": [ - "model", - "language", - "words" - ], - "type": "object" - }, - "voice": { - "additionalProperties": {}, - "description": "frames, and the summary media.analyze voice gives", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "type": "object" - }, - "analysis": { - "anyOf": [ - { - "properties": { - "channels": { - "description": "Each channel at import", - "items": { - "properties": { - "channel": { - "exclusiveMinimum": 0, - "type": "integer" - }, - "noise_floor_dbfs": { - "type": [ - "number", - "null" - ] - }, - "peak_dbfs": { - "type": [ - "number", - "null" - ] - }, - "rms_dbfs": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "channel", - "rms_dbfs", - "peak_dbfs", - "noise_floor_dbfs" - ], - "type": "object" - }, - "type": "array" - }, - "loudness_lufs": { - "description": "Integrated loudness at import", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "loudness_lufs" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "beats": { - "properties": { - "beats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "bpm": { - "type": "number" - }, - "downbeats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "strengths": { - "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "bpm", - "beats", - "downbeats" - ], - "type": "object" - }, - "codecs": { - "properties": { - "audio": { - "type": [ - "string", - "null" - ] - }, - "video": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "video", - "audio" - ], - "type": "object" - }, - "conform": { - "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", - "items": { - "type": "string" - }, - "type": "array" - }, - "created_at": { - "description": "When it was imported or made", - "type": "string" - }, - "duration_seconds": { - "type": [ - "number", - "null" - ] - }, - "error": { - "properties": { - "code": { - "enum": [ - "bad_request", - "no_audio", - "not_found", - "expired", - "media_not_ready", - "media_offline", - "engine_error", - "render_failed", - "interrupted", - "stalled", - "cancelled", - "credential_required", - "credential_invalid", - "output_exists", - "path_not_allowed", - "limit_exceeded", - "quota_exceeded", - "subscription_required", - "email_unverified", - "service_unavailable", - "server_busy" - ], - "type": "string" - }, - "detail": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "hint": { - "type": "string" - }, - "message": { - "type": "string" - } - }, - "required": [ - "code", - "message" - ], - "type": "object" - }, - "expires_at": { - "description": "When url and page_url stop working", - "type": "string" - }, - "extension": { - "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", - "type": "string" - }, - "faces": { - "properties": { - "frames": { - "items": { - "properties": { - "faces": { - "items": { - "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", - "properties": { - "h": { - "type": "number" - }, - "score": { - "type": "number" - }, - "w": { - "type": "number" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "x", - "y", - "w", - "h", - "score" - ], - "type": "object" - }, - "type": "array" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "faces" - ], - "type": "object" - }, - "type": "array" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames" - ], - "type": "object" - }, - "fingerprint": { - "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", - "type": [ - "string", - "null" - ] - }, - "font_family": { - "description": "Fonts: the family; use font_media_id in a text style to set text in it", - "type": "string" - }, - "fps": { - "anyOf": [ - { - "properties": { - "den": { - "type": "number" - }, - "num": { - "type": "number" - } - }, - "required": [ - "num", - "den" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "has_audio": { - "type": "boolean" - }, - "has_video": { - "type": "boolean" - }, - "height": { - "type": [ - "number", - "null" - ] - }, - "job": { - "description": "media.get, while importing or being made: the job doing it", - "properties": { - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "progress": { - "type": "number" - }, - "seconds_left": { - "type": "number" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_message": { - "type": "string" - }, - "tool": { - "type": "string" - } - }, - "required": [ - "job_id", - "tool", - "status" - ], - "type": "object" - }, - "kind": { - "anyOf": [ - { - "enum": [ - "video", - "audio", - "image", - "font", - "lut" - ], - "type": "string" - }, - { - "type": "null" - } - ] - }, - "look": { - "properties": { - "runs": { - "items": { - "properties": { - "frames": { - "items": { - "properties": { - "answers": { - "additionalProperties": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "answers" - ], - "type": "object" - }, - "type": "array" - }, - "id": { - "type": "string" - }, - "model": { - "type": "string" - }, - "questions": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "state": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "with_previous": { - "type": "boolean" - } - }, - "required": [ - "id", - "model", - "questions", - "with_previous", - "frames" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "runs" - ], - "type": "object" - }, - "media_id": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "mime_type": { - "description": "Its MIME type, likewise", - "type": "string" - }, - "name": { - "type": "string" - }, - "origin": { - "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", - "properties": { - "attached": { - "description": "url: the file the user attached in the chat, passed as media.import file", - "type": "boolean" - }, - "file_name": { - "description": "upload: the name the file was sent with", - "type": "string" - }, - "frame": { - "description": "still: the frame", - "type": "number" - }, - "from": { - "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", - "enum": [ - "upload", - "url", - "path", - "render", - "created", - "still" - ], - "type": "string" - }, - "job_id": { - "description": "render: the export.video job; created: the create.* job", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "still: the video the frame is from", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "media_ids": { - "description": "created: the media it was made from (a cutout's source, a page's assets)", - "items": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "type": "array" - }, - "path": { - "description": "path: the file it was imported from", - "type": "string" - }, - "preset": { - "description": "render: the export preset", - "type": "string" - }, - "range": { - "description": "render: the frames exported, when not the whole timeline", - "properties": { - "end": { - "type": "number" - }, - "start": { - "type": "number" - } - }, - "required": [ - "start", - "end" - ], - "type": "object" - }, - "recipe": { - "description": "created: a hash of what it was made from; the same recipe makes the same media", - "type": "string" - }, - "sources": { - "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", - "items": { - "properties": { - "end": { - "type": "number" - }, - "kind": { - "enum": [ - "video", - "audio" - ], - "type": "string" - }, - "media_id": { - "description": "The media this stretch plays", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "source_in": { - "description": "Seconds into that media", - "type": "number" - }, - "source_out": { - "type": "number" - }, - "speed": { - "additionalProperties": {}, - "description": "The clip's speed change, as clip.set_speed set it", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "start": { - "description": "Seconds into this file", - "type": "number" - }, - "track_id": { - "description": "Track of the timeline; outer/inner inside a nested one", - "type": "string" - } - }, - "required": [ - "track_id", - "kind", - "start", - "end", - "media_id", - "source_in", - "source_out" - ], - "type": "object" - }, - "type": "array" - }, - "timeline_id": { - "description": "render: the timeline that was exported; still: the edit the frame is from", - "pattern": "^tl_[0-9a-f]{32}$", - "type": "string" - }, - "tool": { - "description": "created: the create.* tool that made it", - "type": "string" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "url: the address it was downloaded from, without its query string", - "type": "string" - }, - "via": { - "description": "upload: sent through a media.upload link, dropped on the account page, or a data: URI", - "enum": [ - "media.upload", - "account_page", - "data_uri" - ], - "type": "string" - } - }, - "required": [ - "from" - ], - "type": "object" - }, - "page_url": { - "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", - "type": "string" - }, - "pages": { - "additionalProperties": { - "properties": { - "first": { - "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", - "type": "integer" - }, - "list": { - "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", - "type": "string" - }, - "matching": { - "description": "Items in the time range (the whole list without one)", - "type": "integer" - }, - "next_offset": { - "anyOf": [ - { - "type": "integer" - }, - { - "type": "null" - } - ], - "description": "The offset that gives the next page of this range, or null at its end" - }, - "returned": { - "type": "integer" - }, - "total": { - "description": "Items in the whole list", - "type": "integer" - } - }, - "required": [ - "list", - "first", - "returned", - "matching", - "total", - "next_offset" - ], - "type": "object" - }, - "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", - "propertyNames": { - "enum": [ - "transcript", - "beats", - "faces", - "look", - "speech", - "sounds", - "speakers", - "voice", - "colour", - "picture" - ], - "type": "string" - }, - "type": "object" - }, - "probe": { - "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", - "properties": { - "format": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "streams": { - "items": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "format", - "streams" - ], - "type": "object" - }, - "sound_seconds": { - "description": "How long the sound runs; under duration_seconds, the end is soundless", - "type": "number" - }, - "sounds": { - "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", - "properties": { - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - }, - "peaks": { - "additionalProperties": { - "type": "number" - }, - "description": "Per sound reported: the highest probability it reaches anywhere", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "scores": { - "description": "Every probability, for your own cut-off or arithmetic", - "properties": { - "start_seconds": { - "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", - "type": "number" - }, - "step_seconds": { - "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", - "type": "number" - }, - "values": { - "additionalProperties": { - "items": { - "type": "number" - }, - "type": "array" - }, - "description": "Per sound, the highest probability in each step, 0 to 1", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "start_seconds", - "step_seconds", - "values" - ], - "type": "object" - }, - "spans": { - "additionalProperties": { - "items": { - "properties": { - "end": { - "description": "Where it ends, on the same clock", - "type": "number" - }, - "peak_probability": { - "type": "number" - }, - "peak_seconds": { - "description": "The middle of the most likely frame", - "type": "number" - }, - "start": { - "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", - "type": "number" - } - }, - "required": [ - "start", - "end", - "peak_seconds", - "peak_probability" - ], - "type": "object" - }, - "type": "array" - }, - "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "model", - "min_probability", - "spans", - "peaks" - ], - "type": "object" - }, - "speech": { - "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", - "properties": { - "frame_seconds": { - "description": "Each probability covers this long", - "type": "number" - }, - "from": { - "description": "Seconds where the first probability starts", - "type": "number" - }, - "model": { - "type": "string" - }, - "probabilities": { - "description": "Speech probability 0–1, one per frame, in order", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "model", - "frame_seconds", - "from", - "probabilities" - ], - "type": "object" - }, - "status": { - "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", - "enum": [ - "importing", - "processing", - "ready", - "failed" - ], - "type": "string" - }, - "transcript": { - "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", - "properties": { - "aligned_by": { - "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", - "type": [ - "string", - "null" - ] - }, - "from": { - "description": "The stretch transcribed, if not all", - "type": "number" - }, - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "text": { - "type": "string" - }, - "until": { - "type": [ - "number", - "null" - ] - }, - "words": { - "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", - "items": { - "properties": { - "confidence": { - "description": "0–1: how sure whisper is of the word itself", - "type": "number" - }, - "end": { - "type": "number" - }, - "speaker": { - "description": "Who says it, from the latest speakers run", - "type": [ - "number", - "null" - ] - }, - "start": { - "type": "number" - }, - "text": { - "type": "string" - }, - "timing_confidence": { - "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "text", - "start", - "end", - "confidence" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "model", - "language", - "text", - "words" - ], - "type": "object" - }, - "upload": { - "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", - "properties": { - "expires_at": { - "type": "string" - }, - "max_bytes": { - "type": "number" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "Signed: PUT the file here; no other credentials needed", - "type": "string" - } - }, - "required": [ - "upload_id", - "url", - "expires_at", - "max_bytes" - ], - "type": "object" - }, - "url": { - "description": "media.get: a signed download of the file itself, once it has one", - "type": "string" - }, - "warnings": { - "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", - "items": { - "type": "string" - }, - "type": "array" - }, - "width": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "media_id", - "name", - "status", - "kind", - "duration_seconds", - "fps", - "width", - "height", - "has_video", - "has_audio", - "codecs", - "conform", - "analysis" - ], - "type": "object" - }, - { - "properties": { - "hint": { - "type": "string" - }, - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "The media it is making; usable in edits once processing", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_uri": { - "type": "string" - } - }, - "required": [ - "job_id", - "status", - "status_uri", - "hint" - ], - "type": "object" - } -]New value: +[ + { + "properties": { + "analyses": { + "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", + "properties": { + "beats": { + "properties": { + "beats": { + "type": "number" + }, + "bpm": { + "type": "number" + } + }, + "required": [ + "bpm", + "beats" + ], + "type": "object" + }, + "colour": { + "properties": { + "frames": { + "type": "number" + }, + "regions": { + "items": { + "type": "string" + }, + "type": "array" + } + }, + "required": [ + "frames", + "regions" + ], + "type": "object" + }, + "faces": { + "properties": { + "frames_with_faces": { + "type": "number" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames_with_faces" + ], + "type": "object" + }, + "look": { + "description": "frames and questions: the latest run", + "properties": { + "frames": { + "type": "number" + }, + "questions": { + "items": { + "type": "string" + }, + "type": "array" + }, + "runs": { + "type": "number" + } + }, + "required": [ + "runs", + "frames", + "questions" + ], + "type": "object" + }, + "picture": { + "properties": { + "frames": { + "type": "number" + }, + "from": { + "type": "number" + } + }, + "required": [ + "frames", + "from" + ], + "type": "object" + }, + "sounds": { + "description": "heard: the sounds the latest run found at or above its min_probability", + "properties": { + "heard": { + "items": { + "type": "string" + }, + "type": "array" + }, + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "min_probability", + "heard" + ], + "type": "object" + }, + "speakers": { + "properties": { + "model": { + "type": "string" + }, + "speakers": { + "type": "number" + }, + "turns": { + "type": "number" + } + }, + "required": [ + "model", + "speakers", + "turns" + ], + "type": "object" + }, + "speech": { + "properties": { + "frame_seconds": { + "type": "number" + }, + "frames": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "frames", + "frame_seconds" + ], + "type": "object" + }, + "transcript": { + "properties": { + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "words": { + "description": "How many words; the words themselves are in transcript.words", + "type": "number" + } + }, + "required": [ + "model", + "language", + "words" + ], + "type": "object" + }, + "voice": { + "additionalProperties": {}, + "description": "frames, and the summary media.analyze voice gives", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "type": "object" + }, + "analysis": { + "anyOf": [ + { + "properties": { + "channels": { + "description": "Each channel at import", + "items": { + "properties": { + "channel": { + "exclusiveMinimum": 0, + "type": "integer" + }, + "noise_floor_dbfs": { + "type": [ + "number", + "null" + ] + }, + "peak_dbfs": { + "type": [ + "number", + "null" + ] + }, + "rms_dbfs": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "channel", + "rms_dbfs", + "peak_dbfs", + "noise_floor_dbfs" + ], + "type": "object" + }, + "type": "array" + }, + "loudness_lufs": { + "description": "Integrated loudness at import", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "loudness_lufs" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "beats": { + "properties": { + "beats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "bpm": { + "type": "number" + }, + "downbeats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "strengths": { + "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "bpm", + "beats", + "downbeats" + ], + "type": "object" + }, + "codecs": { + "properties": { + "audio": { + "type": [ + "string", + "null" + ] + }, + "video": { + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "video", + "audio" + ], + "type": "object" + }, + "conform": { + "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", + "items": { + "type": "string" + }, + "type": "array" + }, + "created_at": { + "description": "When it was imported or made", + "type": "string" + }, + "duration_seconds": { + "type": [ + "number", + "null" + ] + }, + "error": { + "properties": { + "code": { + "enum": [ + "bad_request", + "no_audio", + "not_found", + "expired", + "media_not_ready", + "media_offline", + "engine_error", + "render_failed", + "interrupted", + "stalled", + "cancelled", + "credential_required", + "credential_invalid", + "output_exists", + "path_not_allowed", + "limit_exceeded", + "quota_exceeded", + "subscription_required", + "email_unverified", + "service_unavailable", + "server_busy" + ], + "type": "string" + }, + "detail": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "hint": { + "type": "string" + }, + "message": { + "type": "string" + } + }, + "required": [ + "code", + "message" + ], + "type": "object" + }, + "expires_at": { + "description": "When url and page_url stop working", + "type": "string" + }, + "extension": { + "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", + "type": "string" + }, + "faces": { + "properties": { + "frames": { + "items": { + "properties": { + "faces": { + "items": { + "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", + "properties": { + "h": { + "type": "number" + }, + "score": { + "type": "number" + }, + "w": { + "type": "number" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "x", + "y", + "w", + "h", + "score" + ], + "type": "object" + }, + "type": "array" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "faces" + ], + "type": "object" + }, + "type": "array" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames" + ], + "type": "object" + }, + "fingerprint": { + "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", + "type": [ + "string", + "null" + ] + }, + "font_family": { + "description": "Fonts: the family; use font_media_id in a text style to set text in it", + "type": "string" + }, + "fps": { + "anyOf": [ + { + "properties": { + "den": { + "type": "number" + }, + "num": { + "type": "number" + } + }, + "required": [ + "num", + "den" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "has_audio": { + "type": "boolean" + }, + "has_video": { + "type": "boolean" + }, + "height": { + "type": [ + "number", + "null" + ] + }, + "job": { + "description": "media.get, while importing or being made: the job doing it", + "properties": { + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "progress": { + "type": "number" + }, + "seconds_left": { + "type": "number" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_message": { + "type": "string" + }, + "tool": { + "type": "string" + } + }, + "required": [ + "job_id", + "tool", + "status" + ], + "type": "object" + }, + "kind": { + "anyOf": [ + { + "enum": [ + "video", + "audio", + "image", + "font", + "lut" + ], + "type": "string" + }, + { + "type": "null" + } + ] + }, + "look": { + "properties": { + "runs": { + "items": { + "properties": { + "frames": { + "items": { + "properties": { + "answers": { + "additionalProperties": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "answers" + ], + "type": "object" + }, + "type": "array" + }, + "id": { + "type": "string" + }, + "model": { + "type": "string" + }, + "questions": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "state": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "with_previous": { + "type": "boolean" + } + }, + "required": [ + "id", + "model", + "questions", + "with_previous", + "frames" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "runs" + ], + "type": "object" + }, + "media_id": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "mime_type": { + "description": "Its MIME type, likewise", + "type": "string" + }, + "name": { + "type": "string" + }, + "origin": { + "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", + "properties": { + "attached": { + "description": "url: the file the user attached in the chat, passed as media.import file", + "type": "boolean" + }, + "file_name": { + "description": "upload: the name the file was sent with", + "type": "string" + }, + "frame": { + "description": "still: the frame", + "type": "number" + }, + "from": { + "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", + "enum": [ + "upload", + "url", + "path", + "render", + "created", + "still" + ], + "type": "string" + }, + "job_id": { + "description": "render: the export.video job; created: the create.* job", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "still: the video the frame is from", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "media_ids": { + "description": "created: the media it was made from (a cutout's source, a page's assets)", + "items": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "type": "array" + }, + "path": { + "description": "path: the file it was imported from", + "type": "string" + }, + "preset": { + "description": "render: the export preset", + "type": "string" + }, + "range": { + "description": "render: the frames exported, when not the whole timeline", + "properties": { + "end": { + "type": "number" + }, + "start": { + "type": "number" + } + }, + "required": [ + "start", + "end" + ], + "type": "object" + }, + "recipe": { + "description": "created: a hash of what it was made from; the same recipe makes the same media", + "type": "string" + }, + "sources": { + "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", + "items": { + "properties": { + "end": { + "type": "number" + }, + "kind": { + "enum": [ + "video", + "audio" + ], + "type": "string" + }, + "media_id": { + "description": "The media this stretch plays", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "source_in": { + "description": "Seconds into that media", + "type": "number" + }, + "source_out": { + "type": "number" + }, + "speed": { + "additionalProperties": {}, + "description": "The clip's speed change, as clip.set_speed set it", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "start": { + "description": "Seconds into this file", + "type": "number" + }, + "track_id": { + "description": "Track of the timeline; outer/inner inside a nested one", + "type": "string" + } + }, + "required": [ + "track_id", + "kind", + "start", + "end", + "media_id", + "source_in", + "source_out" + ], + "type": "object" + }, + "type": "array" + }, + "timeline_id": { + "description": "render: the timeline that was exported; still: the edit the frame is from", + "pattern": "^tl_[0-9a-f]{32}$", + "type": "string" + }, + "tool": { + "description": "created: the create.* tool that made it", + "type": "string" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "url: the address it was downloaded from, without its query string", + "type": "string" + }, + "via": { + "description": "upload: sent through a media.upload link, dropped on the cuttingfloor site, or a data: URI", + "enum": [ + "media.upload", + "account_page", + "data_uri" + ], + "type": "string" + } + }, + "required": [ + "from" + ], + "type": "object" + }, + "page_url": { + "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", + "type": "string" + }, + "pages": { + "additionalProperties": { + "properties": { + "first": { + "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", + "type": "integer" + }, + "list": { + "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", + "type": "string" + }, + "matching": { + "description": "Items in the time range (the whole list without one)", + "type": "integer" + }, + "next_offset": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "The offset that gives the next page of this range, or null at its end" + }, + "returned": { + "type": "integer" + }, + "total": { + "description": "Items in the whole list", + "type": "integer" + } + }, + "required": [ + "list", + "first", + "returned", + "matching", + "total", + "next_offset" + ], + "type": "object" + }, + "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", + "propertyNames": { + "enum": [ + "transcript", + "beats", + "faces", + "look", + "speech", + "sounds", + "speakers", + "voice", + "colour", + "picture" + ], + "type": "string" + }, + "type": "object" + }, + "probe": { + "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", + "properties": { + "format": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "streams": { + "items": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "format", + "streams" + ], + "type": "object" + }, + "sound_seconds": { + "description": "How long the sound runs; under duration_seconds, the end is soundless", + "type": "number" + }, + "sounds": { + "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", + "properties": { + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + }, + "peaks": { + "additionalProperties": { + "type": "number" + }, + "description": "Per sound reported: the highest probability it reaches anywhere", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "scores": { + "description": "Every probability, for your own cut-off or arithmetic", + "properties": { + "start_seconds": { + "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", + "type": "number" + }, + "step_seconds": { + "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", + "type": "number" + }, + "values": { + "additionalProperties": { + "items": { + "type": "number" + }, + "type": "array" + }, + "description": "Per sound, the highest probability in each step, 0 to 1", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "start_seconds", + "step_seconds", + "values" + ], + "type": "object" + }, + "spans": { + "additionalProperties": { + "items": { + "properties": { + "end": { + "description": "Where it ends, on the same clock", + "type": "number" + }, + "peak_probability": { + "type": "number" + }, + "peak_seconds": { + "description": "The middle of the most likely frame", + "type": "number" + }, + "start": { + "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", + "type": "number" + } + }, + "required": [ + "start", + "end", + "peak_seconds", + "peak_probability" + ], + "type": "object" + }, + "type": "array" + }, + "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "model", + "min_probability", + "spans", + "peaks" + ], + "type": "object" + }, + "speech": { + "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", + "properties": { + "frame_seconds": { + "description": "Each probability covers this long", + "type": "number" + }, + "from": { + "description": "Seconds where the first probability starts", + "type": "number" + }, + "model": { + "type": "string" + }, + "probabilities": { + "description": "Speech probability 0–1, one per frame, in order", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "model", + "frame_seconds", + "from", + "probabilities" + ], + "type": "object" + }, + "status": { + "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", + "enum": [ + "importing", + "processing", + "ready", + "failed" + ], + "type": "string" + }, + "transcript": { + "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", + "properties": { + "aligned_by": { + "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", + "type": [ + "string", + "null" + ] + }, + "from": { + "description": "The stretch transcribed, if not all", + "type": "number" + }, + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "text": { + "type": "string" + }, + "until": { + "type": [ + "number", + "null" + ] + }, + "words": { + "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", + "items": { + "properties": { + "confidence": { + "description": "0–1: how sure whisper is of the word itself", + "type": "number" + }, + "end": { + "type": "number" + }, + "speaker": { + "description": "Who says it, from the latest speakers run", + "type": [ + "number", + "null" + ] + }, + "start": { + "type": "number" + }, + "text": { + "type": "string" + }, + "timing_confidence": { + "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "text", + "start", + "end", + "confidence" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "model", + "language", + "text", + "words" + ], + "type": "object" + }, + "upload": { + "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", + "properties": { + "expires_at": { + "type": "string" + }, + "max_bytes": { + "type": "number" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "Signed: PUT the file here; no other credentials needed", + "type": "string" + } + }, + "required": [ + "upload_id", + "url", + "expires_at", + "max_bytes" + ], + "type": "object" + }, + "url": { + "description": "media.get: a signed download of the file itself, once it has one", + "type": "string" + }, + "warnings": { + "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", + "items": { + "type": "string" + }, + "type": "array" + }, + "width": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "media_id", + "name", + "status", + "kind", + "duration_seconds", + "fps", + "width", + "height", + "has_video", + "has_audio", + "codecs", + "conform", + "analysis" + ], + "type": "object" + }, + { + "properties": { + "hint": { + "type": "string" + }, + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "The media it is making; usable in edits once processing", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_uri": { + "type": "string" + } + }, + "required": [ + "job_id", + "status", + "status_uri", + "hint" + ], + "type": "object" + } +]
- Changed
create.stems1 field changed- changed
Output schema / anyOfPrevious value: -[ - { - "properties": { - "media_id": { - "description": "The media the stems came from", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "model": { - "type": "string" - }, - "stems": { - "additionalProperties": { - "properties": { - "analyses": { - "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", - "properties": { - "beats": { - "properties": { - "beats": { - "type": "number" - }, - "bpm": { - "type": "number" - } - }, - "required": [ - "bpm", - "beats" - ], - "type": "object" - }, - "colour": { - "properties": { - "frames": { - "type": "number" - }, - "regions": { - "items": { - "type": "string" - }, - "type": "array" - } - }, - "required": [ - "frames", - "regions" - ], - "type": "object" - }, - "faces": { - "properties": { - "frames_with_faces": { - "type": "number" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames_with_faces" - ], - "type": "object" - }, - "look": { - "description": "frames and questions: the latest run", - "properties": { - "frames": { - "type": "number" - }, - "questions": { - "items": { - "type": "string" - }, - "type": "array" - }, - "runs": { - "type": "number" - } - }, - "required": [ - "runs", - "frames", - "questions" - ], - "type": "object" - }, - "picture": { - "properties": { - "frames": { - "type": "number" - }, - "from": { - "type": "number" - } - }, - "required": [ - "frames", - "from" - ], - "type": "object" - }, - "sounds": { - "description": "heard: the sounds the latest run found at or above its min_probability", - "properties": { - "heard": { - "items": { - "type": "string" - }, - "type": "array" - }, - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "min_probability", - "heard" - ], - "type": "object" - }, - "speakers": { - "properties": { - "model": { - "type": "string" - }, - "speakers": { - "type": "number" - }, - "turns": { - "type": "number" - } - }, - "required": [ - "model", - "speakers", - "turns" - ], - "type": "object" - }, - "speech": { - "properties": { - "frame_seconds": { - "type": "number" - }, - "frames": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "frames", - "frame_seconds" - ], - "type": "object" - }, - "transcript": { - "properties": { - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "words": { - "description": "How many words; the words themselves are in transcript.words", - "type": "number" - } - }, - "required": [ - "model", - "language", - "words" - ], - "type": "object" - }, - "voice": { - "additionalProperties": {}, - "description": "frames, and the summary media.analyze voice gives", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "type": "object" - }, - "analysis": { - "anyOf": [ - { - "properties": { - "channels": { - "description": "Each channel at import", - "items": { - "properties": { - "channel": { - "exclusiveMinimum": 0, - "type": "integer" - }, - "noise_floor_dbfs": { - "type": [ - "number", - "null" - ] - }, - "peak_dbfs": { - "type": [ - "number", - "null" - ] - }, - "rms_dbfs": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "channel", - "rms_dbfs", - "peak_dbfs", - "noise_floor_dbfs" - ], - "type": "object" - }, - "type": "array" - }, - "loudness_lufs": { - "description": "Integrated loudness at import", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "loudness_lufs" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "beats": { - "properties": { - "beats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "bpm": { - "type": "number" - }, - "downbeats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "strengths": { - "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "bpm", - "beats", - "downbeats" - ], - "type": "object" - }, - "codecs": { - "properties": { - "audio": { - "type": [ - "string", - "null" - ] - }, - "video": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "video", - "audio" - ], - "type": "object" - }, - "conform": { - "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", - "items": { - "type": "string" - }, - "type": "array" - }, - "created_at": { - "description": "When it was imported or made", - "type": "string" - }, - "duration_seconds": { - "type": [ - "number", - "null" - ] - }, - "error": { - "properties": { - "code": { - "enum": [ - "bad_request", - "no_audio", - "not_found", - "expired", - "media_not_ready", - "media_offline", - "engine_error", - "render_failed", - "interrupted", - "stalled", - "cancelled", - "credential_required", - "credential_invalid", - "output_exists", - "path_not_allowed", - "limit_exceeded", - "quota_exceeded", - "subscription_required", - "email_unverified", - "service_unavailable", - "server_busy" - ], - "type": "string" - }, - "detail": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "hint": { - "type": "string" - }, - "message": { - "type": "string" - } - }, - "required": [ - "code", - "message" - ], - "type": "object" - }, - "expires_at": { - "description": "When url and page_url stop working", - "type": "string" - }, - "extension": { - "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", - "type": "string" - }, - "faces": { - "properties": { - "frames": { - "items": { - "properties": { - "faces": { - "items": { - "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", - "properties": { - "h": { - "type": "number" - }, - "score": { - "type": "number" - }, - "w": { - "type": "number" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "x", - "y", - "w", - "h", - "score" - ], - "type": "object" - }, - "type": "array" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "faces" - ], - "type": "object" - }, - "type": "array" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames" - ], - "type": "object" - }, - "fingerprint": { - "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", - "type": [ - "string", - "null" - ] - }, - "font_family": { - "description": "Fonts: the family; use font_media_id in a text style to set text in it", - "type": "string" - }, - "fps": { - "anyOf": [ - { - "properties": { - "den": { - "type": "number" - }, - "num": { - "type": "number" - } - }, - "required": [ - "num", - "den" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "has_audio": { - "type": "boolean" - }, - "has_video": { - "type": "boolean" - }, - "height": { - "type": [ - "number", - "null" - ] - }, - "job": { - "description": "media.get, while importing or being made: the job doing it", - "properties": { - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "progress": { - "type": "number" - }, - "seconds_left": { - "type": "number" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_message": { - "type": "string" - }, - "tool": { - "type": "string" - } - }, - "required": [ - "job_id", - "tool", - "status" - ], - "type": "object" - }, - "kind": { - "anyOf": [ - { - "enum": [ - "video", - "audio", - "image", - "font", - "lut" - ], - "type": "string" - }, - { - "type": "null" - } - ] - }, - "look": { - "properties": { - "runs": { - "items": { - "properties": { - "frames": { - "items": { - "properties": { - "answers": { - "additionalProperties": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "answers" - ], - "type": "object" - }, - "type": "array" - }, - "id": { - "type": "string" - }, - "model": { - "type": "string" - }, - "questions": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "state": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "with_previous": { - "type": "boolean" - } - }, - "required": [ - "id", - "model", - "questions", - "with_previous", - "frames" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "runs" - ], - "type": "object" - }, - "media_id": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "mime_type": { - "description": "Its MIME type, likewise", - "type": "string" - }, - "name": { - "type": "string" - }, - "origin": { - "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", - "properties": { - "attached": { - "description": "url: the file the user attached in the chat, passed as media.import file", - "type": "boolean" - }, - "file_name": { - "description": "upload: the name the file was sent with", - "type": "string" - }, - "frame": { - "description": "still: the frame", - "type": "number" - }, - "from": { - "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", - "enum": [ - "upload", - "url", - "path", - "render", - "created", - "still" - ], - "type": "string" - }, - "job_id": { - "description": "render: the export.video job; created: the create.* job", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "still: the video the frame is from", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "media_ids": { - "description": "created: the media it was made from (a cutout's source, a page's assets)", - "items": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "type": "array" - }, - "path": { - "description": "path: the file it was imported from", - "type": "string" - }, - "preset": { - "description": "render: the export preset", - "type": "string" - }, - "range": { - "description": "render: the frames exported, when not the whole timeline", - "properties": { - "end": { - "type": "number" - }, - "start": { - "type": "number" - } - }, - "required": [ - "start", - "end" - ], - "type": "object" - }, - "recipe": { - "description": "created: a hash of what it was made from; the same recipe makes the same media", - "type": "string" - }, - "sources": { - "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", - "items": { - "properties": { - "end": { - "type": "number" - }, - "kind": { - "enum": [ - "video", - "audio" - ], - "type": "string" - }, - "media_id": { - "description": "The media this stretch plays", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "source_in": { - "description": "Seconds into that media", - "type": "number" - }, - "source_out": { - "type": "number" - }, - "speed": { - "additionalProperties": {}, - "description": "The clip's speed change, as clip.set_speed set it", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "start": { - "description": "Seconds into this file", - "type": "number" - }, - "track_id": { - "description": "Track of the timeline; outer/inner inside a nested one", - "type": "string" - } - }, - "required": [ - "track_id", - "kind", - "start", - "end", - "media_id", - "source_in", - "source_out" - ], - "type": "object" - }, - "type": "array" - }, - "timeline_id": { - "description": "render: the timeline that was exported; still: the edit the frame is from", - "pattern": "^tl_[0-9a-f]{32}$", - "type": "string" - }, - "tool": { - "description": "created: the create.* tool that made it", - "type": "string" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "url: the address it was downloaded from, without its query string", - "type": "string" - }, - "via": { - "description": "upload: sent through a media.upload link, dropped on the account page, or a data: URI", - "enum": [ - "media.upload", - "account_page", - "data_uri" - ], - "type": "string" - } - }, - "required": [ - "from" - ], - "type": "object" - }, - "page_url": { - "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", - "type": "string" - }, - "pages": { - "additionalProperties": { - "properties": { - "first": { - "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", - "type": "integer" - }, - "list": { - "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", - "type": "string" - }, - "matching": { - "description": "Items in the time range (the whole list without one)", - "type": "integer" - }, - "next_offset": { - "anyOf": [ - { - "type": "integer" - }, - { - "type": "null" - } - ], - "description": "The offset that gives the next page of this range, or null at its end" - }, - "returned": { - "type": "integer" - }, - "total": { - "description": "Items in the whole list", - "type": "integer" - } - }, - "required": [ - "list", - "first", - "returned", - "matching", - "total", - "next_offset" - ], - "type": "object" - }, - "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", - "propertyNames": { - "enum": [ - "transcript", - "beats", - "faces", - "look", - "speech", - "sounds", - "speakers", - "voice", - "colour", - "picture" - ], - "type": "string" - }, - "type": "object" - }, - "probe": { - "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", - "properties": { - "format": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "streams": { - "items": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "format", - "streams" - ], - "type": "object" - }, - "sound_seconds": { - "description": "How long the sound runs; under duration_seconds, the end is soundless", - "type": "number" - }, - "sounds": { - "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", - "properties": { - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - }, - "peaks": { - "additionalProperties": { - "type": "number" - }, - "description": "Per sound reported: the highest probability it reaches anywhere", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "scores": { - "description": "Every probability, for your own cut-off or arithmetic", - "properties": { - "start_seconds": { - "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", - "type": "number" - }, - "step_seconds": { - "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", - "type": "number" - }, - "values": { - "additionalProperties": { - "items": { - "type": "number" - }, - "type": "array" - }, - "description": "Per sound, the highest probability in each step, 0 to 1", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "start_seconds", - "step_seconds", - "values" - ], - "type": "object" - }, - "spans": { - "additionalProperties": { - "items": { - "properties": { - "end": { - "description": "Where it ends, on the same clock", - "type": "number" - }, - "peak_probability": { - "type": "number" - }, - "peak_seconds": { - "description": "The middle of the most likely frame", - "type": "number" - }, - "start": { - "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", - "type": "number" - } - }, - "required": [ - "start", - "end", - "peak_seconds", - "peak_probability" - ], - "type": "object" - }, - "type": "array" - }, - "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "model", - "min_probability", - "spans", - "peaks" - ], - "type": "object" - }, - "speech": { - "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", - "properties": { - "frame_seconds": { - "description": "Each probability covers this long", - "type": "number" - }, - "from": { - "description": "Seconds where the first probability starts", - "type": "number" - }, - "model": { - "type": "string" - }, - "probabilities": { - "description": "Speech probability 0–1, one per frame, in order", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "model", - "frame_seconds", - "from", - "probabilities" - ], - "type": "object" - }, - "status": { - "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", - "enum": [ - "importing", - "processing", - "ready", - "failed" - ], - "type": "string" - }, - "transcript": { - "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", - "properties": { - "aligned_by": { - "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", - "type": [ - "string", - "null" - ] - }, - "from": { - "description": "The stretch transcribed, if not all", - "type": "number" - }, - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "text": { - "type": "string" - }, - "until": { - "type": [ - "number", - "null" - ] - }, - "words": { - "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", - "items": { - "properties": { - "confidence": { - "description": "0–1: how sure whisper is of the word itself", - "type": "number" - }, - "end": { - "type": "number" - }, - "speaker": { - "description": "Who says it, from the latest speakers run", - "type": [ - "number", - "null" - ] - }, - "start": { - "type": "number" - }, - "text": { - "type": "string" - }, - "timing_confidence": { - "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "text", - "start", - "end", - "confidence" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "model", - "language", - "text", - "words" - ], - "type": "object" - }, - "upload": { - "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", - "properties": { - "expires_at": { - "type": "string" - }, - "max_bytes": { - "type": "number" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "Signed: PUT the file here; no other credentials needed", - "type": "string" - } - }, - "required": [ - "upload_id", - "url", - "expires_at", - "max_bytes" - ], - "type": "object" - }, - "url": { - "description": "media.get: a signed download of the file itself, once it has one", - "type": "string" - }, - "warnings": { - "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", - "items": { - "type": "string" - }, - "type": "array" - }, - "width": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "media_id", - "name", - "status", - "kind", - "duration_seconds", - "fps", - "width", - "height", - "has_video", - "has_audio", - "codecs", - "conform", - "analysis" - ], - "type": "object" - }, - "description": "Per stem asked for: the new audio media (32-bit float WAV, 44.1 kHz stereo)", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "media_id", - "model", - "stems" - ], - "type": "object" - }, - { - "properties": { - "hint": { - "type": "string" - }, - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "The media it is making; usable in edits once processing", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_uri": { - "type": "string" - } - }, - "required": [ - "job_id", - "status", - "status_uri", - "hint" - ], - "type": "object" - } -]New value: +[ + { + "properties": { + "media_id": { + "description": "The media the stems came from", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "model": { + "type": "string" + }, + "stems": { + "additionalProperties": { + "properties": { + "analyses": { + "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", + "properties": { + "beats": { + "properties": { + "beats": { + "type": "number" + }, + "bpm": { + "type": "number" + } + }, + "required": [ + "bpm", + "beats" + ], + "type": "object" + }, + "colour": { + "properties": { + "frames": { + "type": "number" + }, + "regions": { + "items": { + "type": "string" + }, + "type": "array" + } + }, + "required": [ + "frames", + "regions" + ], + "type": "object" + }, + "faces": { + "properties": { + "frames_with_faces": { + "type": "number" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames_with_faces" + ], + "type": "object" + }, + "look": { + "description": "frames and questions: the latest run", + "properties": { + "frames": { + "type": "number" + }, + "questions": { + "items": { + "type": "string" + }, + "type": "array" + }, + "runs": { + "type": "number" + } + }, + "required": [ + "runs", + "frames", + "questions" + ], + "type": "object" + }, + "picture": { + "properties": { + "frames": { + "type": "number" + }, + "from": { + "type": "number" + } + }, + "required": [ + "frames", + "from" + ], + "type": "object" + }, + "sounds": { + "description": "heard: the sounds the latest run found at or above its min_probability", + "properties": { + "heard": { + "items": { + "type": "string" + }, + "type": "array" + }, + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "min_probability", + "heard" + ], + "type": "object" + }, + "speakers": { + "properties": { + "model": { + "type": "string" + }, + "speakers": { + "type": "number" + }, + "turns": { + "type": "number" + } + }, + "required": [ + "model", + "speakers", + "turns" + ], + "type": "object" + }, + "speech": { + "properties": { + "frame_seconds": { + "type": "number" + }, + "frames": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "frames", + "frame_seconds" + ], + "type": "object" + }, + "transcript": { + "properties": { + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "words": { + "description": "How many words; the words themselves are in transcript.words", + "type": "number" + } + }, + "required": [ + "model", + "language", + "words" + ], + "type": "object" + }, + "voice": { + "additionalProperties": {}, + "description": "frames, and the summary media.analyze voice gives", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "type": "object" + }, + "analysis": { + "anyOf": [ + { + "properties": { + "channels": { + "description": "Each channel at import", + "items": { + "properties": { + "channel": { + "exclusiveMinimum": 0, + "type": "integer" + }, + "noise_floor_dbfs": { + "type": [ + "number", + "null" + ] + }, + "peak_dbfs": { + "type": [ + "number", + "null" + ] + }, + "rms_dbfs": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "channel", + "rms_dbfs", + "peak_dbfs", + "noise_floor_dbfs" + ], + "type": "object" + }, + "type": "array" + }, + "loudness_lufs": { + "description": "Integrated loudness at import", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "loudness_lufs" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "beats": { + "properties": { + "beats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "bpm": { + "type": "number" + }, + "downbeats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "strengths": { + "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "bpm", + "beats", + "downbeats" + ], + "type": "object" + }, + "codecs": { + "properties": { + "audio": { + "type": [ + "string", + "null" + ] + }, + "video": { + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "video", + "audio" + ], + "type": "object" + }, + "conform": { + "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", + "items": { + "type": "string" + }, + "type": "array" + }, + "created_at": { + "description": "When it was imported or made", + "type": "string" + }, + "duration_seconds": { + "type": [ + "number", + "null" + ] + }, + "error": { + "properties": { + "code": { + "enum": [ + "bad_request", + "no_audio", + "not_found", + "expired", + "media_not_ready", + "media_offline", + "engine_error", + "render_failed", + "interrupted", + "stalled", + "cancelled", + "credential_required", + "credential_invalid", + "output_exists", + "path_not_allowed", + "limit_exceeded", + "quota_exceeded", + "subscription_required", + "email_unverified", + "service_unavailable", + "server_busy" + ], + "type": "string" + }, + "detail": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "hint": { + "type": "string" + }, + "message": { + "type": "string" + } + }, + "required": [ + "code", + "message" + ], + "type": "object" + }, + "expires_at": { + "description": "When url and page_url stop working", + "type": "string" + }, + "extension": { + "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", + "type": "string" + }, + "faces": { + "properties": { + "frames": { + "items": { + "properties": { + "faces": { + "items": { + "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", + "properties": { + "h": { + "type": "number" + }, + "score": { + "type": "number" + }, + "w": { + "type": "number" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "x", + "y", + "w", + "h", + "score" + ], + "type": "object" + }, + "type": "array" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "faces" + ], + "type": "object" + }, + "type": "array" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames" + ], + "type": "object" + }, + "fingerprint": { + "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", + "type": [ + "string", + "null" + ] + }, + "font_family": { + "description": "Fonts: the family; use font_media_id in a text style to set text in it", + "type": "string" + }, + "fps": { + "anyOf": [ + { + "properties": { + "den": { + "type": "number" + }, + "num": { + "type": "number" + } + }, + "required": [ + "num", + "den" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "has_audio": { + "type": "boolean" + }, + "has_video": { + "type": "boolean" + }, + "height": { + "type": [ + "number", + "null" + ] + }, + "job": { + "description": "media.get, while importing or being made: the job doing it", + "properties": { + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "progress": { + "type": "number" + }, + "seconds_left": { + "type": "number" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_message": { + "type": "string" + }, + "tool": { + "type": "string" + } + }, + "required": [ + "job_id", + "tool", + "status" + ], + "type": "object" + }, + "kind": { + "anyOf": [ + { + "enum": [ + "video", + "audio", + "image", + "font", + "lut" + ], + "type": "string" + }, + { + "type": "null" + } + ] + }, + "look": { + "properties": { + "runs": { + "items": { + "properties": { + "frames": { + "items": { + "properties": { + "answers": { + "additionalProperties": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "answers" + ], + "type": "object" + }, + "type": "array" + }, + "id": { + "type": "string" + }, + "model": { + "type": "string" + }, + "questions": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "state": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "with_previous": { + "type": "boolean" + } + }, + "required": [ + "id", + "model", + "questions", + "with_previous", + "frames" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "runs" + ], + "type": "object" + }, + "media_id": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "mime_type": { + "description": "Its MIME type, likewise", + "type": "string" + }, + "name": { + "type": "string" + }, + "origin": { + "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", + "properties": { + "attached": { + "description": "url: the file the user attached in the chat, passed as media.import file", + "type": "boolean" + }, + "file_name": { + "description": "upload: the name the file was sent with", + "type": "string" + }, + "frame": { + "description": "still: the frame", + "type": "number" + }, + "from": { + "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", + "enum": [ + "upload", + "url", + "path", + "render", + "created", + "still" + ], + "type": "string" + }, + "job_id": { + "description": "render: the export.video job; created: the create.* job", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "still: the video the frame is from", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "media_ids": { + "description": "created: the media it was made from (a cutout's source, a page's assets)", + "items": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "type": "array" + }, + "path": { + "description": "path: the file it was imported from", + "type": "string" + }, + "preset": { + "description": "render: the export preset", + "type": "string" + }, + "range": { + "description": "render: the frames exported, when not the whole timeline", + "properties": { + "end": { + "type": "number" + }, + "start": { + "type": "number" + } + }, + "required": [ + "start", + "end" + ], + "type": "object" + }, + "recipe": { + "description": "created: a hash of what it was made from; the same recipe makes the same media", + "type": "string" + }, + "sources": { + "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", + "items": { + "properties": { + "end": { + "type": "number" + }, + "kind": { + "enum": [ + "video", + "audio" + ], + "type": "string" + }, + "media_id": { + "description": "The media this stretch plays", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "source_in": { + "description": "Seconds into that media", + "type": "number" + }, + "source_out": { + "type": "number" + }, + "speed": { + "additionalProperties": {}, + "description": "The clip's speed change, as clip.set_speed set it", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "start": { + "description": "Seconds into this file", + "type": "number" + }, + "track_id": { + "description": "Track of the timeline; outer/inner inside a nested one", + "type": "string" + } + }, + "required": [ + "track_id", + "kind", + "start", + "end", + "media_id", + "source_in", + "source_out" + ], + "type": "object" + }, + "type": "array" + }, + "timeline_id": { + "description": "render: the timeline that was exported; still: the edit the frame is from", + "pattern": "^tl_[0-9a-f]{32}$", + "type": "string" + }, + "tool": { + "description": "created: the create.* tool that made it", + "type": "string" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "url: the address it was downloaded from, without its query string", + "type": "string" + }, + "via": { + "description": "upload: sent through a media.upload link, dropped on the cuttingfloor site, or a data: URI", + "enum": [ + "media.upload", + "account_page", + "data_uri" + ], + "type": "string" + } + }, + "required": [ + "from" + ], + "type": "object" + }, + "page_url": { + "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", + "type": "string" + }, + "pages": { + "additionalProperties": { + "properties": { + "first": { + "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", + "type": "integer" + }, + "list": { + "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", + "type": "string" + }, + "matching": { + "description": "Items in the time range (the whole list without one)", + "type": "integer" + }, + "next_offset": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "The offset that gives the next page of this range, or null at its end" + }, + "returned": { + "type": "integer" + }, + "total": { + "description": "Items in the whole list", + "type": "integer" + } + }, + "required": [ + "list", + "first", + "returned", + "matching", + "total", + "next_offset" + ], + "type": "object" + }, + "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", + "propertyNames": { + "enum": [ + "transcript", + "beats", + "faces", + "look", + "speech", + "sounds", + "speakers", + "voice", + "colour", + "picture" + ], + "type": "string" + }, + "type": "object" + }, + "probe": { + "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", + "properties": { + "format": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "streams": { + "items": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "format", + "streams" + ], + "type": "object" + }, + "sound_seconds": { + "description": "How long the sound runs; under duration_seconds, the end is soundless", + "type": "number" + }, + "sounds": { + "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", + "properties": { + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + }, + "peaks": { + "additionalProperties": { + "type": "number" + }, + "description": "Per sound reported: the highest probability it reaches anywhere", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "scores": { + "description": "Every probability, for your own cut-off or arithmetic", + "properties": { + "start_seconds": { + "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", + "type": "number" + }, + "step_seconds": { + "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", + "type": "number" + }, + "values": { + "additionalProperties": { + "items": { + "type": "number" + }, + "type": "array" + }, + "description": "Per sound, the highest probability in each step, 0 to 1", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "start_seconds", + "step_seconds", + "values" + ], + "type": "object" + }, + "spans": { + "additionalProperties": { + "items": { + "properties": { + "end": { + "description": "Where it ends, on the same clock", + "type": "number" + }, + "peak_probability": { + "type": "number" + }, + "peak_seconds": { + "description": "The middle of the most likely frame", + "type": "number" + }, + "start": { + "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", + "type": "number" + } + }, + "required": [ + "start", + "end", + "peak_seconds", + "peak_probability" + ], + "type": "object" + }, + "type": "array" + }, + "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "model", + "min_probability", + "spans", + "peaks" + ], + "type": "object" + }, + "speech": { + "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", + "properties": { + "frame_seconds": { + "description": "Each probability covers this long", + "type": "number" + }, + "from": { + "description": "Seconds where the first probability starts", + "type": "number" + }, + "model": { + "type": "string" + }, + "probabilities": { + "description": "Speech probability 0–1, one per frame, in order", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "model", + "frame_seconds", + "from", + "probabilities" + ], + "type": "object" + }, + "status": { + "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", + "enum": [ + "importing", + "processing", + "ready", + "failed" + ], + "type": "string" + }, + "transcript": { + "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", + "properties": { + "aligned_by": { + "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", + "type": [ + "string", + "null" + ] + }, + "from": { + "description": "The stretch transcribed, if not all", + "type": "number" + }, + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "text": { + "type": "string" + }, + "until": { + "type": [ + "number", + "null" + ] + }, + "words": { + "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", + "items": { + "properties": { + "confidence": { + "description": "0–1: how sure whisper is of the word itself", + "type": "number" + }, + "end": { + "type": "number" + }, + "speaker": { + "description": "Who says it, from the latest speakers run", + "type": [ + "number", + "null" + ] + }, + "start": { + "type": "number" + }, + "text": { + "type": "string" + }, + "timing_confidence": { + "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "text", + "start", + "end", + "confidence" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "model", + "language", + "text", + "words" + ], + "type": "object" + }, + "upload": { + "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", + "properties": { + "expires_at": { + "type": "string" + }, + "max_bytes": { + "type": "number" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "Signed: PUT the file here; no other credentials needed", + "type": "string" + } + }, + "required": [ + "upload_id", + "url", + "expires_at", + "max_bytes" + ], + "type": "object" + }, + "url": { + "description": "media.get: a signed download of the file itself, once it has one", + "type": "string" + }, + "warnings": { + "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", + "items": { + "type": "string" + }, + "type": "array" + }, + "width": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "media_id", + "name", + "status", + "kind", + "duration_seconds", + "fps", + "width", + "height", + "has_video", + "has_audio", + "codecs", + "conform", + "analysis" + ], + "type": "object" + }, + "description": "Per stem asked for: the new audio media (32-bit float WAV, 44.1 kHz stereo)", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "media_id", + "model", + "stems" + ], + "type": "object" + }, + { + "properties": { + "hint": { + "type": "string" + }, + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "The media it is making; usable in edits once processing", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_uri": { + "type": "string" + } + }, + "required": [ + "job_id", + "status", + "status_uri", + "hint" + ], + "type": "object" + } +]
- Changed
create.still1 field changed- changed
Output schema / anyOfPrevious value: -[ - { - "properties": { - "analyses": { - "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", - "properties": { - "beats": { - "properties": { - "beats": { - "type": "number" - }, - "bpm": { - "type": "number" - } - }, - "required": [ - "bpm", - "beats" - ], - "type": "object" - }, - "colour": { - "properties": { - "frames": { - "type": "number" - }, - "regions": { - "items": { - "type": "string" - }, - "type": "array" - } - }, - "required": [ - "frames", - "regions" - ], - "type": "object" - }, - "faces": { - "properties": { - "frames_with_faces": { - "type": "number" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames_with_faces" - ], - "type": "object" - }, - "look": { - "description": "frames and questions: the latest run", - "properties": { - "frames": { - "type": "number" - }, - "questions": { - "items": { - "type": "string" - }, - "type": "array" - }, - "runs": { - "type": "number" - } - }, - "required": [ - "runs", - "frames", - "questions" - ], - "type": "object" - }, - "picture": { - "properties": { - "frames": { - "type": "number" - }, - "from": { - "type": "number" - } - }, - "required": [ - "frames", - "from" - ], - "type": "object" - }, - "sounds": { - "description": "heard: the sounds the latest run found at or above its min_probability", - "properties": { - "heard": { - "items": { - "type": "string" - }, - "type": "array" - }, - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "min_probability", - "heard" - ], - "type": "object" - }, - "speakers": { - "properties": { - "model": { - "type": "string" - }, - "speakers": { - "type": "number" - }, - "turns": { - "type": "number" - } - }, - "required": [ - "model", - "speakers", - "turns" - ], - "type": "object" - }, - "speech": { - "properties": { - "frame_seconds": { - "type": "number" - }, - "frames": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "frames", - "frame_seconds" - ], - "type": "object" - }, - "transcript": { - "properties": { - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "words": { - "description": "How many words; the words themselves are in transcript.words", - "type": "number" - } - }, - "required": [ - "model", - "language", - "words" - ], - "type": "object" - }, - "voice": { - "additionalProperties": {}, - "description": "frames, and the summary media.analyze voice gives", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "type": "object" - }, - "analysis": { - "anyOf": [ - { - "properties": { - "channels": { - "description": "Each channel at import", - "items": { - "properties": { - "channel": { - "exclusiveMinimum": 0, - "type": "integer" - }, - "noise_floor_dbfs": { - "type": [ - "number", - "null" - ] - }, - "peak_dbfs": { - "type": [ - "number", - "null" - ] - }, - "rms_dbfs": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "channel", - "rms_dbfs", - "peak_dbfs", - "noise_floor_dbfs" - ], - "type": "object" - }, - "type": "array" - }, - "loudness_lufs": { - "description": "Integrated loudness at import", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "loudness_lufs" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "beats": { - "properties": { - "beats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "bpm": { - "type": "number" - }, - "downbeats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "strengths": { - "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "bpm", - "beats", - "downbeats" - ], - "type": "object" - }, - "codecs": { - "properties": { - "audio": { - "type": [ - "string", - "null" - ] - }, - "video": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "video", - "audio" - ], - "type": "object" - }, - "conform": { - "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", - "items": { - "type": "string" - }, - "type": "array" - }, - "created_at": { - "description": "When it was imported or made", - "type": "string" - }, - "duration_seconds": { - "type": [ - "number", - "null" - ] - }, - "error": { - "properties": { - "code": { - "enum": [ - "bad_request", - "no_audio", - "not_found", - "expired", - "media_not_ready", - "media_offline", - "engine_error", - "render_failed", - "interrupted", - "stalled", - "cancelled", - "credential_required", - "credential_invalid", - "output_exists", - "path_not_allowed", - "limit_exceeded", - "quota_exceeded", - "subscription_required", - "email_unverified", - "service_unavailable", - "server_busy" - ], - "type": "string" - }, - "detail": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "hint": { - "type": "string" - }, - "message": { - "type": "string" - } - }, - "required": [ - "code", - "message" - ], - "type": "object" - }, - "expires_at": { - "description": "When url and page_url stop working", - "type": "string" - }, - "extension": { - "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", - "type": "string" - }, - "faces": { - "properties": { - "frames": { - "items": { - "properties": { - "faces": { - "items": { - "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", - "properties": { - "h": { - "type": "number" - }, - "score": { - "type": "number" - }, - "w": { - "type": "number" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "x", - "y", - "w", - "h", - "score" - ], - "type": "object" - }, - "type": "array" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "faces" - ], - "type": "object" - }, - "type": "array" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames" - ], - "type": "object" - }, - "fingerprint": { - "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", - "type": [ - "string", - "null" - ] - }, - "font_family": { - "description": "Fonts: the family; use font_media_id in a text style to set text in it", - "type": "string" - }, - "fps": { - "anyOf": [ - { - "properties": { - "den": { - "type": "number" - }, - "num": { - "type": "number" - } - }, - "required": [ - "num", - "den" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "has_audio": { - "type": "boolean" - }, - "has_video": { - "type": "boolean" - }, - "height": { - "type": [ - "number", - "null" - ] - }, - "job": { - "description": "media.get, while importing or being made: the job doing it", - "properties": { - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "progress": { - "type": "number" - }, - "seconds_left": { - "type": "number" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_message": { - "type": "string" - }, - "tool": { - "type": "string" - } - }, - "required": [ - "job_id", - "tool", - "status" - ], - "type": "object" - }, - "kind": { - "anyOf": [ - { - "enum": [ - "video", - "audio", - "image", - "font", - "lut" - ], - "type": "string" - }, - { - "type": "null" - } - ] - }, - "look": { - "properties": { - "runs": { - "items": { - "properties": { - "frames": { - "items": { - "properties": { - "answers": { - "additionalProperties": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "answers" - ], - "type": "object" - }, - "type": "array" - }, - "id": { - "type": "string" - }, - "model": { - "type": "string" - }, - "questions": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "state": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "with_previous": { - "type": "boolean" - } - }, - "required": [ - "id", - "model", - "questions", - "with_previous", - "frames" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "runs" - ], - "type": "object" - }, - "media_id": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "mime_type": { - "description": "Its MIME type, likewise", - "type": "string" - }, - "name": { - "type": "string" - }, - "origin": { - "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", - "properties": { - "attached": { - "description": "url: the file the user attached in the chat, passed as media.import file", - "type": "boolean" - }, - "file_name": { - "description": "upload: the name the file was sent with", - "type": "string" - }, - "frame": { - "description": "still: the frame", - "type": "number" - }, - "from": { - "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", - "enum": [ - "upload", - "url", - "path", - "render", - "created", - "still" - ], - "type": "string" - }, - "job_id": { - "description": "render: the export.video job; created: the create.* job", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "still: the video the frame is from", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "media_ids": { - "description": "created: the media it was made from (a cutout's source, a page's assets)", - "items": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "type": "array" - }, - "path": { - "description": "path: the file it was imported from", - "type": "string" - }, - "preset": { - "description": "render: the export preset", - "type": "string" - }, - "range": { - "description": "render: the frames exported, when not the whole timeline", - "properties": { - "end": { - "type": "number" - }, - "start": { - "type": "number" - } - }, - "required": [ - "start", - "end" - ], - "type": "object" - }, - "recipe": { - "description": "created: a hash of what it was made from; the same recipe makes the same media", - "type": "string" - }, - "sources": { - "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", - "items": { - "properties": { - "end": { - "type": "number" - }, - "kind": { - "enum": [ - "video", - "audio" - ], - "type": "string" - }, - "media_id": { - "description": "The media this stretch plays", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "source_in": { - "description": "Seconds into that media", - "type": "number" - }, - "source_out": { - "type": "number" - }, - "speed": { - "additionalProperties": {}, - "description": "The clip's speed change, as clip.set_speed set it", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "start": { - "description": "Seconds into this file", - "type": "number" - }, - "track_id": { - "description": "Track of the timeline; outer/inner inside a nested one", - "type": "string" - } - }, - "required": [ - "track_id", - "kind", - "start", - "end", - "media_id", - "source_in", - "source_out" - ], - "type": "object" - }, - "type": "array" - }, - "timeline_id": { - "description": "render: the timeline that was exported; still: the edit the frame is from", - "pattern": "^tl_[0-9a-f]{32}$", - "type": "string" - }, - "tool": { - "description": "created: the create.* tool that made it", - "type": "string" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "url: the address it was downloaded from, without its query string", - "type": "string" - }, - "via": { - "description": "upload: sent through a media.upload link, dropped on the account page, or a data: URI", - "enum": [ - "media.upload", - "account_page", - "data_uri" - ], - "type": "string" - } - }, - "required": [ - "from" - ], - "type": "object" - }, - "page_url": { - "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", - "type": "string" - }, - "pages": { - "additionalProperties": { - "properties": { - "first": { - "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", - "type": "integer" - }, - "list": { - "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", - "type": "string" - }, - "matching": { - "description": "Items in the time range (the whole list without one)", - "type": "integer" - }, - "next_offset": { - "anyOf": [ - { - "type": "integer" - }, - { - "type": "null" - } - ], - "description": "The offset that gives the next page of this range, or null at its end" - }, - "returned": { - "type": "integer" - }, - "total": { - "description": "Items in the whole list", - "type": "integer" - } - }, - "required": [ - "list", - "first", - "returned", - "matching", - "total", - "next_offset" - ], - "type": "object" - }, - "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", - "propertyNames": { - "enum": [ - "transcript", - "beats", - "faces", - "look", - "speech", - "sounds", - "speakers", - "voice", - "colour", - "picture" - ], - "type": "string" - }, - "type": "object" - }, - "probe": { - "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", - "properties": { - "format": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "streams": { - "items": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "format", - "streams" - ], - "type": "object" - }, - "sound_seconds": { - "description": "How long the sound runs; under duration_seconds, the end is soundless", - "type": "number" - }, - "sounds": { - "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", - "properties": { - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - }, - "peaks": { - "additionalProperties": { - "type": "number" - }, - "description": "Per sound reported: the highest probability it reaches anywhere", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "scores": { - "description": "Every probability, for your own cut-off or arithmetic", - "properties": { - "start_seconds": { - "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", - "type": "number" - }, - "step_seconds": { - "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", - "type": "number" - }, - "values": { - "additionalProperties": { - "items": { - "type": "number" - }, - "type": "array" - }, - "description": "Per sound, the highest probability in each step, 0 to 1", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "start_seconds", - "step_seconds", - "values" - ], - "type": "object" - }, - "spans": { - "additionalProperties": { - "items": { - "properties": { - "end": { - "description": "Where it ends, on the same clock", - "type": "number" - }, - "peak_probability": { - "type": "number" - }, - "peak_seconds": { - "description": "The middle of the most likely frame", - "type": "number" - }, - "start": { - "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", - "type": "number" - } - }, - "required": [ - "start", - "end", - "peak_seconds", - "peak_probability" - ], - "type": "object" - }, - "type": "array" - }, - "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "model", - "min_probability", - "spans", - "peaks" - ], - "type": "object" - }, - "speech": { - "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", - "properties": { - "frame_seconds": { - "description": "Each probability covers this long", - "type": "number" - }, - "from": { - "description": "Seconds where the first probability starts", - "type": "number" - }, - "model": { - "type": "string" - }, - "probabilities": { - "description": "Speech probability 0–1, one per frame, in order", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "model", - "frame_seconds", - "from", - "probabilities" - ], - "type": "object" - }, - "status": { - "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", - "enum": [ - "importing", - "processing", - "ready", - "failed" - ], - "type": "string" - }, - "transcript": { - "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", - "properties": { - "aligned_by": { - "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", - "type": [ - "string", - "null" - ] - }, - "from": { - "description": "The stretch transcribed, if not all", - "type": "number" - }, - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "text": { - "type": "string" - }, - "until": { - "type": [ - "number", - "null" - ] - }, - "words": { - "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", - "items": { - "properties": { - "confidence": { - "description": "0–1: how sure whisper is of the word itself", - "type": "number" - }, - "end": { - "type": "number" - }, - "speaker": { - "description": "Who says it, from the latest speakers run", - "type": [ - "number", - "null" - ] - }, - "start": { - "type": "number" - }, - "text": { - "type": "string" - }, - "timing_confidence": { - "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "text", - "start", - "end", - "confidence" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "model", - "language", - "text", - "words" - ], - "type": "object" - }, - "upload": { - "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", - "properties": { - "expires_at": { - "type": "string" - }, - "max_bytes": { - "type": "number" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "Signed: PUT the file here; no other credentials needed", - "type": "string" - } - }, - "required": [ - "upload_id", - "url", - "expires_at", - "max_bytes" - ], - "type": "object" - }, - "url": { - "description": "media.get: a signed download of the file itself, once it has one", - "type": "string" - }, - "warnings": { - "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", - "items": { - "type": "string" - }, - "type": "array" - }, - "width": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "media_id", - "name", - "status", - "kind", - "duration_seconds", - "fps", - "width", - "height", - "has_video", - "has_audio", - "codecs", - "conform", - "analysis" - ], - "type": "object" - }, - { - "properties": { - "hint": { - "type": "string" - }, - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "The media it is making; usable in edits once processing", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_uri": { - "type": "string" - } - }, - "required": [ - "job_id", - "status", - "status_uri", - "hint" - ], - "type": "object" - } -]New value: +[ + { + "properties": { + "analyses": { + "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", + "properties": { + "beats": { + "properties": { + "beats": { + "type": "number" + }, + "bpm": { + "type": "number" + } + }, + "required": [ + "bpm", + "beats" + ], + "type": "object" + }, + "colour": { + "properties": { + "frames": { + "type": "number" + }, + "regions": { + "items": { + "type": "string" + }, + "type": "array" + } + }, + "required": [ + "frames", + "regions" + ], + "type": "object" + }, + "faces": { + "properties": { + "frames_with_faces": { + "type": "number" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames_with_faces" + ], + "type": "object" + }, + "look": { + "description": "frames and questions: the latest run", + "properties": { + "frames": { + "type": "number" + }, + "questions": { + "items": { + "type": "string" + }, + "type": "array" + }, + "runs": { + "type": "number" + } + }, + "required": [ + "runs", + "frames", + "questions" + ], + "type": "object" + }, + "picture": { + "properties": { + "frames": { + "type": "number" + }, + "from": { + "type": "number" + } + }, + "required": [ + "frames", + "from" + ], + "type": "object" + }, + "sounds": { + "description": "heard: the sounds the latest run found at or above its min_probability", + "properties": { + "heard": { + "items": { + "type": "string" + }, + "type": "array" + }, + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "min_probability", + "heard" + ], + "type": "object" + }, + "speakers": { + "properties": { + "model": { + "type": "string" + }, + "speakers": { + "type": "number" + }, + "turns": { + "type": "number" + } + }, + "required": [ + "model", + "speakers", + "turns" + ], + "type": "object" + }, + "speech": { + "properties": { + "frame_seconds": { + "type": "number" + }, + "frames": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "frames", + "frame_seconds" + ], + "type": "object" + }, + "transcript": { + "properties": { + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "words": { + "description": "How many words; the words themselves are in transcript.words", + "type": "number" + } + }, + "required": [ + "model", + "language", + "words" + ], + "type": "object" + }, + "voice": { + "additionalProperties": {}, + "description": "frames, and the summary media.analyze voice gives", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "type": "object" + }, + "analysis": { + "anyOf": [ + { + "properties": { + "channels": { + "description": "Each channel at import", + "items": { + "properties": { + "channel": { + "exclusiveMinimum": 0, + "type": "integer" + }, + "noise_floor_dbfs": { + "type": [ + "number", + "null" + ] + }, + "peak_dbfs": { + "type": [ + "number", + "null" + ] + }, + "rms_dbfs": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "channel", + "rms_dbfs", + "peak_dbfs", + "noise_floor_dbfs" + ], + "type": "object" + }, + "type": "array" + }, + "loudness_lufs": { + "description": "Integrated loudness at import", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "loudness_lufs" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "beats": { + "properties": { + "beats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "bpm": { + "type": "number" + }, + "downbeats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "strengths": { + "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "bpm", + "beats", + "downbeats" + ], + "type": "object" + }, + "codecs": { + "properties": { + "audio": { + "type": [ + "string", + "null" + ] + }, + "video": { + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "video", + "audio" + ], + "type": "object" + }, + "conform": { + "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", + "items": { + "type": "string" + }, + "type": "array" + }, + "created_at": { + "description": "When it was imported or made", + "type": "string" + }, + "duration_seconds": { + "type": [ + "number", + "null" + ] + }, + "error": { + "properties": { + "code": { + "enum": [ + "bad_request", + "no_audio", + "not_found", + "expired", + "media_not_ready", + "media_offline", + "engine_error", + "render_failed", + "interrupted", + "stalled", + "cancelled", + "credential_required", + "credential_invalid", + "output_exists", + "path_not_allowed", + "limit_exceeded", + "quota_exceeded", + "subscription_required", + "email_unverified", + "service_unavailable", + "server_busy" + ], + "type": "string" + }, + "detail": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "hint": { + "type": "string" + }, + "message": { + "type": "string" + } + }, + "required": [ + "code", + "message" + ], + "type": "object" + }, + "expires_at": { + "description": "When url and page_url stop working", + "type": "string" + }, + "extension": { + "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", + "type": "string" + }, + "faces": { + "properties": { + "frames": { + "items": { + "properties": { + "faces": { + "items": { + "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", + "properties": { + "h": { + "type": "number" + }, + "score": { + "type": "number" + }, + "w": { + "type": "number" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "x", + "y", + "w", + "h", + "score" + ], + "type": "object" + }, + "type": "array" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "faces" + ], + "type": "object" + }, + "type": "array" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames" + ], + "type": "object" + }, + "fingerprint": { + "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", + "type": [ + "string", + "null" + ] + }, + "font_family": { + "description": "Fonts: the family; use font_media_id in a text style to set text in it", + "type": "string" + }, + "fps": { + "anyOf": [ + { + "properties": { + "den": { + "type": "number" + }, + "num": { + "type": "number" + } + }, + "required": [ + "num", + "den" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "has_audio": { + "type": "boolean" + }, + "has_video": { + "type": "boolean" + }, + "height": { + "type": [ + "number", + "null" + ] + }, + "job": { + "description": "media.get, while importing or being made: the job doing it", + "properties": { + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "progress": { + "type": "number" + }, + "seconds_left": { + "type": "number" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_message": { + "type": "string" + }, + "tool": { + "type": "string" + } + }, + "required": [ + "job_id", + "tool", + "status" + ], + "type": "object" + }, + "kind": { + "anyOf": [ + { + "enum": [ + "video", + "audio", + "image", + "font", + "lut" + ], + "type": "string" + }, + { + "type": "null" + } + ] + }, + "look": { + "properties": { + "runs": { + "items": { + "properties": { + "frames": { + "items": { + "properties": { + "answers": { + "additionalProperties": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "answers" + ], + "type": "object" + }, + "type": "array" + }, + "id": { + "type": "string" + }, + "model": { + "type": "string" + }, + "questions": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "state": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "with_previous": { + "type": "boolean" + } + }, + "required": [ + "id", + "model", + "questions", + "with_previous", + "frames" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "runs" + ], + "type": "object" + }, + "media_id": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "mime_type": { + "description": "Its MIME type, likewise", + "type": "string" + }, + "name": { + "type": "string" + }, + "origin": { + "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", + "properties": { + "attached": { + "description": "url: the file the user attached in the chat, passed as media.import file", + "type": "boolean" + }, + "file_name": { + "description": "upload: the name the file was sent with", + "type": "string" + }, + "frame": { + "description": "still: the frame", + "type": "number" + }, + "from": { + "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", + "enum": [ + "upload", + "url", + "path", + "render", + "created", + "still" + ], + "type": "string" + }, + "job_id": { + "description": "render: the export.video job; created: the create.* job", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "still: the video the frame is from", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "media_ids": { + "description": "created: the media it was made from (a cutout's source, a page's assets)", + "items": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "type": "array" + }, + "path": { + "description": "path: the file it was imported from", + "type": "string" + }, + "preset": { + "description": "render: the export preset", + "type": "string" + }, + "range": { + "description": "render: the frames exported, when not the whole timeline", + "properties": { + "end": { + "type": "number" + }, + "start": { + "type": "number" + } + }, + "required": [ + "start", + "end" + ], + "type": "object" + }, + "recipe": { + "description": "created: a hash of what it was made from; the same recipe makes the same media", + "type": "string" + }, + "sources": { + "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", + "items": { + "properties": { + "end": { + "type": "number" + }, + "kind": { + "enum": [ + "video", + "audio" + ], + "type": "string" + }, + "media_id": { + "description": "The media this stretch plays", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "source_in": { + "description": "Seconds into that media", + "type": "number" + }, + "source_out": { + "type": "number" + }, + "speed": { + "additionalProperties": {}, + "description": "The clip's speed change, as clip.set_speed set it", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "start": { + "description": "Seconds into this file", + "type": "number" + }, + "track_id": { + "description": "Track of the timeline; outer/inner inside a nested one", + "type": "string" + } + }, + "required": [ + "track_id", + "kind", + "start", + "end", + "media_id", + "source_in", + "source_out" + ], + "type": "object" + }, + "type": "array" + }, + "timeline_id": { + "description": "render: the timeline that was exported; still: the edit the frame is from", + "pattern": "^tl_[0-9a-f]{32}$", + "type": "string" + }, + "tool": { + "description": "created: the create.* tool that made it", + "type": "string" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "url: the address it was downloaded from, without its query string", + "type": "string" + }, + "via": { + "description": "upload: sent through a media.upload link, dropped on the cuttingfloor site, or a data: URI", + "enum": [ + "media.upload", + "account_page", + "data_uri" + ], + "type": "string" + } + }, + "required": [ + "from" + ], + "type": "object" + }, + "page_url": { + "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", + "type": "string" + }, + "pages": { + "additionalProperties": { + "properties": { + "first": { + "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", + "type": "integer" + }, + "list": { + "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", + "type": "string" + }, + "matching": { + "description": "Items in the time range (the whole list without one)", + "type": "integer" + }, + "next_offset": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "The offset that gives the next page of this range, or null at its end" + }, + "returned": { + "type": "integer" + }, + "total": { + "description": "Items in the whole list", + "type": "integer" + } + }, + "required": [ + "list", + "first", + "returned", + "matching", + "total", + "next_offset" + ], + "type": "object" + }, + "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", + "propertyNames": { + "enum": [ + "transcript", + "beats", + "faces", + "look", + "speech", + "sounds", + "speakers", + "voice", + "colour", + "picture" + ], + "type": "string" + }, + "type": "object" + }, + "probe": { + "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", + "properties": { + "format": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "streams": { + "items": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "format", + "streams" + ], + "type": "object" + }, + "sound_seconds": { + "description": "How long the sound runs; under duration_seconds, the end is soundless", + "type": "number" + }, + "sounds": { + "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", + "properties": { + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + }, + "peaks": { + "additionalProperties": { + "type": "number" + }, + "description": "Per sound reported: the highest probability it reaches anywhere", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "scores": { + "description": "Every probability, for your own cut-off or arithmetic", + "properties": { + "start_seconds": { + "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", + "type": "number" + }, + "step_seconds": { + "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", + "type": "number" + }, + "values": { + "additionalProperties": { + "items": { + "type": "number" + }, + "type": "array" + }, + "description": "Per sound, the highest probability in each step, 0 to 1", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "start_seconds", + "step_seconds", + "values" + ], + "type": "object" + }, + "spans": { + "additionalProperties": { + "items": { + "properties": { + "end": { + "description": "Where it ends, on the same clock", + "type": "number" + }, + "peak_probability": { + "type": "number" + }, + "peak_seconds": { + "description": "The middle of the most likely frame", + "type": "number" + }, + "start": { + "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", + "type": "number" + } + }, + "required": [ + "start", + "end", + "peak_seconds", + "peak_probability" + ], + "type": "object" + }, + "type": "array" + }, + "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "model", + "min_probability", + "spans", + "peaks" + ], + "type": "object" + }, + "speech": { + "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", + "properties": { + "frame_seconds": { + "description": "Each probability covers this long", + "type": "number" + }, + "from": { + "description": "Seconds where the first probability starts", + "type": "number" + }, + "model": { + "type": "string" + }, + "probabilities": { + "description": "Speech probability 0–1, one per frame, in order", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "model", + "frame_seconds", + "from", + "probabilities" + ], + "type": "object" + }, + "status": { + "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", + "enum": [ + "importing", + "processing", + "ready", + "failed" + ], + "type": "string" + }, + "transcript": { + "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", + "properties": { + "aligned_by": { + "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", + "type": [ + "string", + "null" + ] + }, + "from": { + "description": "The stretch transcribed, if not all", + "type": "number" + }, + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "text": { + "type": "string" + }, + "until": { + "type": [ + "number", + "null" + ] + }, + "words": { + "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", + "items": { + "properties": { + "confidence": { + "description": "0–1: how sure whisper is of the word itself", + "type": "number" + }, + "end": { + "type": "number" + }, + "speaker": { + "description": "Who says it, from the latest speakers run", + "type": [ + "number", + "null" + ] + }, + "start": { + "type": "number" + }, + "text": { + "type": "string" + }, + "timing_confidence": { + "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "text", + "start", + "end", + "confidence" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "model", + "language", + "text", + "words" + ], + "type": "object" + }, + "upload": { + "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", + "properties": { + "expires_at": { + "type": "string" + }, + "max_bytes": { + "type": "number" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "Signed: PUT the file here; no other credentials needed", + "type": "string" + } + }, + "required": [ + "upload_id", + "url", + "expires_at", + "max_bytes" + ], + "type": "object" + }, + "url": { + "description": "media.get: a signed download of the file itself, once it has one", + "type": "string" + }, + "warnings": { + "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", + "items": { + "type": "string" + }, + "type": "array" + }, + "width": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "media_id", + "name", + "status", + "kind", + "duration_seconds", + "fps", + "width", + "height", + "has_video", + "has_audio", + "codecs", + "conform", + "analysis" + ], + "type": "object" + }, + { + "properties": { + "hint": { + "type": "string" + }, + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "The media it is making; usable in edits once processing", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_uri": { + "type": "string" + } + }, + "required": [ + "job_id", + "status", + "status_uri", + "hint" + ], + "type": "object" + } +]
- Changed
create.upscaled1 field changed- changed
Output schema / anyOfPrevious value: -[ - { - "properties": { - "analyses": { - "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", - "properties": { - "beats": { - "properties": { - "beats": { - "type": "number" - }, - "bpm": { - "type": "number" - } - }, - "required": [ - "bpm", - "beats" - ], - "type": "object" - }, - "colour": { - "properties": { - "frames": { - "type": "number" - }, - "regions": { - "items": { - "type": "string" - }, - "type": "array" - } - }, - "required": [ - "frames", - "regions" - ], - "type": "object" - }, - "faces": { - "properties": { - "frames_with_faces": { - "type": "number" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames_with_faces" - ], - "type": "object" - }, - "look": { - "description": "frames and questions: the latest run", - "properties": { - "frames": { - "type": "number" - }, - "questions": { - "items": { - "type": "string" - }, - "type": "array" - }, - "runs": { - "type": "number" - } - }, - "required": [ - "runs", - "frames", - "questions" - ], - "type": "object" - }, - "picture": { - "properties": { - "frames": { - "type": "number" - }, - "from": { - "type": "number" - } - }, - "required": [ - "frames", - "from" - ], - "type": "object" - }, - "sounds": { - "description": "heard: the sounds the latest run found at or above its min_probability", - "properties": { - "heard": { - "items": { - "type": "string" - }, - "type": "array" - }, - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "min_probability", - "heard" - ], - "type": "object" - }, - "speakers": { - "properties": { - "model": { - "type": "string" - }, - "speakers": { - "type": "number" - }, - "turns": { - "type": "number" - } - }, - "required": [ - "model", - "speakers", - "turns" - ], - "type": "object" - }, - "speech": { - "properties": { - "frame_seconds": { - "type": "number" - }, - "frames": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "frames", - "frame_seconds" - ], - "type": "object" - }, - "transcript": { - "properties": { - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "words": { - "description": "How many words; the words themselves are in transcript.words", - "type": "number" - } - }, - "required": [ - "model", - "language", - "words" - ], - "type": "object" - }, - "voice": { - "additionalProperties": {}, - "description": "frames, and the summary media.analyze voice gives", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "type": "object" - }, - "analysis": { - "anyOf": [ - { - "properties": { - "channels": { - "description": "Each channel at import", - "items": { - "properties": { - "channel": { - "exclusiveMinimum": 0, - "type": "integer" - }, - "noise_floor_dbfs": { - "type": [ - "number", - "null" - ] - }, - "peak_dbfs": { - "type": [ - "number", - "null" - ] - }, - "rms_dbfs": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "channel", - "rms_dbfs", - "peak_dbfs", - "noise_floor_dbfs" - ], - "type": "object" - }, - "type": "array" - }, - "loudness_lufs": { - "description": "Integrated loudness at import", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "loudness_lufs" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "beats": { - "properties": { - "beats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "bpm": { - "type": "number" - }, - "downbeats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "strengths": { - "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "bpm", - "beats", - "downbeats" - ], - "type": "object" - }, - "codecs": { - "properties": { - "audio": { - "type": [ - "string", - "null" - ] - }, - "video": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "video", - "audio" - ], - "type": "object" - }, - "conform": { - "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", - "items": { - "type": "string" - }, - "type": "array" - }, - "created_at": { - "description": "When it was imported or made", - "type": "string" - }, - "duration_seconds": { - "type": [ - "number", - "null" - ] - }, - "error": { - "properties": { - "code": { - "enum": [ - "bad_request", - "no_audio", - "not_found", - "expired", - "media_not_ready", - "media_offline", - "engine_error", - "render_failed", - "interrupted", - "stalled", - "cancelled", - "credential_required", - "credential_invalid", - "output_exists", - "path_not_allowed", - "limit_exceeded", - "quota_exceeded", - "subscription_required", - "email_unverified", - "service_unavailable", - "server_busy" - ], - "type": "string" - }, - "detail": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "hint": { - "type": "string" - }, - "message": { - "type": "string" - } - }, - "required": [ - "code", - "message" - ], - "type": "object" - }, - "expires_at": { - "description": "When url and page_url stop working", - "type": "string" - }, - "extension": { - "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", - "type": "string" - }, - "faces": { - "properties": { - "frames": { - "items": { - "properties": { - "faces": { - "items": { - "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", - "properties": { - "h": { - "type": "number" - }, - "score": { - "type": "number" - }, - "w": { - "type": "number" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "x", - "y", - "w", - "h", - "score" - ], - "type": "object" - }, - "type": "array" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "faces" - ], - "type": "object" - }, - "type": "array" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames" - ], - "type": "object" - }, - "fingerprint": { - "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", - "type": [ - "string", - "null" - ] - }, - "font_family": { - "description": "Fonts: the family; use font_media_id in a text style to set text in it", - "type": "string" - }, - "fps": { - "anyOf": [ - { - "properties": { - "den": { - "type": "number" - }, - "num": { - "type": "number" - } - }, - "required": [ - "num", - "den" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "has_audio": { - "type": "boolean" - }, - "has_video": { - "type": "boolean" - }, - "height": { - "type": [ - "number", - "null" - ] - }, - "job": { - "description": "media.get, while importing or being made: the job doing it", - "properties": { - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "progress": { - "type": "number" - }, - "seconds_left": { - "type": "number" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_message": { - "type": "string" - }, - "tool": { - "type": "string" - } - }, - "required": [ - "job_id", - "tool", - "status" - ], - "type": "object" - }, - "kind": { - "anyOf": [ - { - "enum": [ - "video", - "audio", - "image", - "font", - "lut" - ], - "type": "string" - }, - { - "type": "null" - } - ] - }, - "look": { - "properties": { - "runs": { - "items": { - "properties": { - "frames": { - "items": { - "properties": { - "answers": { - "additionalProperties": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "answers" - ], - "type": "object" - }, - "type": "array" - }, - "id": { - "type": "string" - }, - "model": { - "type": "string" - }, - "questions": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "state": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "with_previous": { - "type": "boolean" - } - }, - "required": [ - "id", - "model", - "questions", - "with_previous", - "frames" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "runs" - ], - "type": "object" - }, - "media_id": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "mime_type": { - "description": "Its MIME type, likewise", - "type": "string" - }, - "name": { - "type": "string" - }, - "origin": { - "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", - "properties": { - "attached": { - "description": "url: the file the user attached in the chat, passed as media.import file", - "type": "boolean" - }, - "file_name": { - "description": "upload: the name the file was sent with", - "type": "string" - }, - "frame": { - "description": "still: the frame", - "type": "number" - }, - "from": { - "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", - "enum": [ - "upload", - "url", - "path", - "render", - "created", - "still" - ], - "type": "string" - }, - "job_id": { - "description": "render: the export.video job; created: the create.* job", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "still: the video the frame is from", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "media_ids": { - "description": "created: the media it was made from (a cutout's source, a page's assets)", - "items": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "type": "array" - }, - "path": { - "description": "path: the file it was imported from", - "type": "string" - }, - "preset": { - "description": "render: the export preset", - "type": "string" - }, - "range": { - "description": "render: the frames exported, when not the whole timeline", - "properties": { - "end": { - "type": "number" - }, - "start": { - "type": "number" - } - }, - "required": [ - "start", - "end" - ], - "type": "object" - }, - "recipe": { - "description": "created: a hash of what it was made from; the same recipe makes the same media", - "type": "string" - }, - "sources": { - "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", - "items": { - "properties": { - "end": { - "type": "number" - }, - "kind": { - "enum": [ - "video", - "audio" - ], - "type": "string" - }, - "media_id": { - "description": "The media this stretch plays", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "source_in": { - "description": "Seconds into that media", - "type": "number" - }, - "source_out": { - "type": "number" - }, - "speed": { - "additionalProperties": {}, - "description": "The clip's speed change, as clip.set_speed set it", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "start": { - "description": "Seconds into this file", - "type": "number" - }, - "track_id": { - "description": "Track of the timeline; outer/inner inside a nested one", - "type": "string" - } - }, - "required": [ - "track_id", - "kind", - "start", - "end", - "media_id", - "source_in", - "source_out" - ], - "type": "object" - }, - "type": "array" - }, - "timeline_id": { - "description": "render: the timeline that was exported; still: the edit the frame is from", - "pattern": "^tl_[0-9a-f]{32}$", - "type": "string" - }, - "tool": { - "description": "created: the create.* tool that made it", - "type": "string" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "url: the address it was downloaded from, without its query string", - "type": "string" - }, - "via": { - "description": "upload: sent through a media.upload link, dropped on the account page, or a data: URI", - "enum": [ - "media.upload", - "account_page", - "data_uri" - ], - "type": "string" - } - }, - "required": [ - "from" - ], - "type": "object" - }, - "page_url": { - "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", - "type": "string" - }, - "pages": { - "additionalProperties": { - "properties": { - "first": { - "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", - "type": "integer" - }, - "list": { - "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", - "type": "string" - }, - "matching": { - "description": "Items in the time range (the whole list without one)", - "type": "integer" - }, - "next_offset": { - "anyOf": [ - { - "type": "integer" - }, - { - "type": "null" - } - ], - "description": "The offset that gives the next page of this range, or null at its end" - }, - "returned": { - "type": "integer" - }, - "total": { - "description": "Items in the whole list", - "type": "integer" - } - }, - "required": [ - "list", - "first", - "returned", - "matching", - "total", - "next_offset" - ], - "type": "object" - }, - "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", - "propertyNames": { - "enum": [ - "transcript", - "beats", - "faces", - "look", - "speech", - "sounds", - "speakers", - "voice", - "colour", - "picture" - ], - "type": "string" - }, - "type": "object" - }, - "probe": { - "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", - "properties": { - "format": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "streams": { - "items": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "format", - "streams" - ], - "type": "object" - }, - "sound_seconds": { - "description": "How long the sound runs; under duration_seconds, the end is soundless", - "type": "number" - }, - "sounds": { - "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", - "properties": { - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - }, - "peaks": { - "additionalProperties": { - "type": "number" - }, - "description": "Per sound reported: the highest probability it reaches anywhere", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "scores": { - "description": "Every probability, for your own cut-off or arithmetic", - "properties": { - "start_seconds": { - "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", - "type": "number" - }, - "step_seconds": { - "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", - "type": "number" - }, - "values": { - "additionalProperties": { - "items": { - "type": "number" - }, - "type": "array" - }, - "description": "Per sound, the highest probability in each step, 0 to 1", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "start_seconds", - "step_seconds", - "values" - ], - "type": "object" - }, - "spans": { - "additionalProperties": { - "items": { - "properties": { - "end": { - "description": "Where it ends, on the same clock", - "type": "number" - }, - "peak_probability": { - "type": "number" - }, - "peak_seconds": { - "description": "The middle of the most likely frame", - "type": "number" - }, - "start": { - "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", - "type": "number" - } - }, - "required": [ - "start", - "end", - "peak_seconds", - "peak_probability" - ], - "type": "object" - }, - "type": "array" - }, - "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "model", - "min_probability", - "spans", - "peaks" - ], - "type": "object" - }, - "speech": { - "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", - "properties": { - "frame_seconds": { - "description": "Each probability covers this long", - "type": "number" - }, - "from": { - "description": "Seconds where the first probability starts", - "type": "number" - }, - "model": { - "type": "string" - }, - "probabilities": { - "description": "Speech probability 0–1, one per frame, in order", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "model", - "frame_seconds", - "from", - "probabilities" - ], - "type": "object" - }, - "status": { - "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", - "enum": [ - "importing", - "processing", - "ready", - "failed" - ], - "type": "string" - }, - "transcript": { - "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", - "properties": { - "aligned_by": { - "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", - "type": [ - "string", - "null" - ] - }, - "from": { - "description": "The stretch transcribed, if not all", - "type": "number" - }, - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "text": { - "type": "string" - }, - "until": { - "type": [ - "number", - "null" - ] - }, - "words": { - "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", - "items": { - "properties": { - "confidence": { - "description": "0–1: how sure whisper is of the word itself", - "type": "number" - }, - "end": { - "type": "number" - }, - "speaker": { - "description": "Who says it, from the latest speakers run", - "type": [ - "number", - "null" - ] - }, - "start": { - "type": "number" - }, - "text": { - "type": "string" - }, - "timing_confidence": { - "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "text", - "start", - "end", - "confidence" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "model", - "language", - "text", - "words" - ], - "type": "object" - }, - "upload": { - "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", - "properties": { - "expires_at": { - "type": "string" - }, - "max_bytes": { - "type": "number" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "Signed: PUT the file here; no other credentials needed", - "type": "string" - } - }, - "required": [ - "upload_id", - "url", - "expires_at", - "max_bytes" - ], - "type": "object" - }, - "url": { - "description": "media.get: a signed download of the file itself, once it has one", - "type": "string" - }, - "warnings": { - "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", - "items": { - "type": "string" - }, - "type": "array" - }, - "width": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "media_id", - "name", - "status", - "kind", - "duration_seconds", - "fps", - "width", - "height", - "has_video", - "has_audio", - "codecs", - "conform", - "analysis" - ], - "type": "object" - }, - { - "properties": { - "hint": { - "type": "string" - }, - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "The media it is making; usable in edits once processing", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_uri": { - "type": "string" - } - }, - "required": [ - "job_id", - "status", - "status_uri", - "hint" - ], - "type": "object" - } -]New value: +[ + { + "properties": { + "analyses": { + "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", + "properties": { + "beats": { + "properties": { + "beats": { + "type": "number" + }, + "bpm": { + "type": "number" + } + }, + "required": [ + "bpm", + "beats" + ], + "type": "object" + }, + "colour": { + "properties": { + "frames": { + "type": "number" + }, + "regions": { + "items": { + "type": "string" + }, + "type": "array" + } + }, + "required": [ + "frames", + "regions" + ], + "type": "object" + }, + "faces": { + "properties": { + "frames_with_faces": { + "type": "number" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames_with_faces" + ], + "type": "object" + }, + "look": { + "description": "frames and questions: the latest run", + "properties": { + "frames": { + "type": "number" + }, + "questions": { + "items": { + "type": "string" + }, + "type": "array" + }, + "runs": { + "type": "number" + } + }, + "required": [ + "runs", + "frames", + "questions" + ], + "type": "object" + }, + "picture": { + "properties": { + "frames": { + "type": "number" + }, + "from": { + "type": "number" + } + }, + "required": [ + "frames", + "from" + ], + "type": "object" + }, + "sounds": { + "description": "heard: the sounds the latest run found at or above its min_probability", + "properties": { + "heard": { + "items": { + "type": "string" + }, + "type": "array" + }, + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "min_probability", + "heard" + ], + "type": "object" + }, + "speakers": { + "properties": { + "model": { + "type": "string" + }, + "speakers": { + "type": "number" + }, + "turns": { + "type": "number" + } + }, + "required": [ + "model", + "speakers", + "turns" + ], + "type": "object" + }, + "speech": { + "properties": { + "frame_seconds": { + "type": "number" + }, + "frames": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "frames", + "frame_seconds" + ], + "type": "object" + }, + "transcript": { + "properties": { + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "words": { + "description": "How many words; the words themselves are in transcript.words", + "type": "number" + } + }, + "required": [ + "model", + "language", + "words" + ], + "type": "object" + }, + "voice": { + "additionalProperties": {}, + "description": "frames, and the summary media.analyze voice gives", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "type": "object" + }, + "analysis": { + "anyOf": [ + { + "properties": { + "channels": { + "description": "Each channel at import", + "items": { + "properties": { + "channel": { + "exclusiveMinimum": 0, + "type": "integer" + }, + "noise_floor_dbfs": { + "type": [ + "number", + "null" + ] + }, + "peak_dbfs": { + "type": [ + "number", + "null" + ] + }, + "rms_dbfs": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "channel", + "rms_dbfs", + "peak_dbfs", + "noise_floor_dbfs" + ], + "type": "object" + }, + "type": "array" + }, + "loudness_lufs": { + "description": "Integrated loudness at import", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "loudness_lufs" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "beats": { + "properties": { + "beats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "bpm": { + "type": "number" + }, + "downbeats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "strengths": { + "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "bpm", + "beats", + "downbeats" + ], + "type": "object" + }, + "codecs": { + "properties": { + "audio": { + "type": [ + "string", + "null" + ] + }, + "video": { + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "video", + "audio" + ], + "type": "object" + }, + "conform": { + "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", + "items": { + "type": "string" + }, + "type": "array" + }, + "created_at": { + "description": "When it was imported or made", + "type": "string" + }, + "duration_seconds": { + "type": [ + "number", + "null" + ] + }, + "error": { + "properties": { + "code": { + "enum": [ + "bad_request", + "no_audio", + "not_found", + "expired", + "media_not_ready", + "media_offline", + "engine_error", + "render_failed", + "interrupted", + "stalled", + "cancelled", + "credential_required", + "credential_invalid", + "output_exists", + "path_not_allowed", + "limit_exceeded", + "quota_exceeded", + "subscription_required", + "email_unverified", + "service_unavailable", + "server_busy" + ], + "type": "string" + }, + "detail": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "hint": { + "type": "string" + }, + "message": { + "type": "string" + } + }, + "required": [ + "code", + "message" + ], + "type": "object" + }, + "expires_at": { + "description": "When url and page_url stop working", + "type": "string" + }, + "extension": { + "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", + "type": "string" + }, + "faces": { + "properties": { + "frames": { + "items": { + "properties": { + "faces": { + "items": { + "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", + "properties": { + "h": { + "type": "number" + }, + "score": { + "type": "number" + }, + "w": { + "type": "number" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "x", + "y", + "w", + "h", + "score" + ], + "type": "object" + }, + "type": "array" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "faces" + ], + "type": "object" + }, + "type": "array" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames" + ], + "type": "object" + }, + "fingerprint": { + "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", + "type": [ + "string", + "null" + ] + }, + "font_family": { + "description": "Fonts: the family; use font_media_id in a text style to set text in it", + "type": "string" + }, + "fps": { + "anyOf": [ + { + "properties": { + "den": { + "type": "number" + }, + "num": { + "type": "number" + } + }, + "required": [ + "num", + "den" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "has_audio": { + "type": "boolean" + }, + "has_video": { + "type": "boolean" + }, + "height": { + "type": [ + "number", + "null" + ] + }, + "job": { + "description": "media.get, while importing or being made: the job doing it", + "properties": { + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "progress": { + "type": "number" + }, + "seconds_left": { + "type": "number" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_message": { + "type": "string" + }, + "tool": { + "type": "string" + } + }, + "required": [ + "job_id", + "tool", + "status" + ], + "type": "object" + }, + "kind": { + "anyOf": [ + { + "enum": [ + "video", + "audio", + "image", + "font", + "lut" + ], + "type": "string" + }, + { + "type": "null" + } + ] + }, + "look": { + "properties": { + "runs": { + "items": { + "properties": { + "frames": { + "items": { + "properties": { + "answers": { + "additionalProperties": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "answers" + ], + "type": "object" + }, + "type": "array" + }, + "id": { + "type": "string" + }, + "model": { + "type": "string" + }, + "questions": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "state": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "with_previous": { + "type": "boolean" + } + }, + "required": [ + "id", + "model", + "questions", + "with_previous", + "frames" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "runs" + ], + "type": "object" + }, + "media_id": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "mime_type": { + "description": "Its MIME type, likewise", + "type": "string" + }, + "name": { + "type": "string" + }, + "origin": { + "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", + "properties": { + "attached": { + "description": "url: the file the user attached in the chat, passed as media.import file", + "type": "boolean" + }, + "file_name": { + "description": "upload: the name the file was sent with", + "type": "string" + }, + "frame": { + "description": "still: the frame", + "type": "number" + }, + "from": { + "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", + "enum": [ + "upload", + "url", + "path", + "render", + "created", + "still" + ], + "type": "string" + }, + "job_id": { + "description": "render: the export.video job; created: the create.* job", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "still: the video the frame is from", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "media_ids": { + "description": "created: the media it was made from (a cutout's source, a page's assets)", + "items": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "type": "array" + }, + "path": { + "description": "path: the file it was imported from", + "type": "string" + }, + "preset": { + "description": "render: the export preset", + "type": "string" + }, + "range": { + "description": "render: the frames exported, when not the whole timeline", + "properties": { + "end": { + "type": "number" + }, + "start": { + "type": "number" + } + }, + "required": [ + "start", + "end" + ], + "type": "object" + }, + "recipe": { + "description": "created: a hash of what it was made from; the same recipe makes the same media", + "type": "string" + }, + "sources": { + "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", + "items": { + "properties": { + "end": { + "type": "number" + }, + "kind": { + "enum": [ + "video", + "audio" + ], + "type": "string" + }, + "media_id": { + "description": "The media this stretch plays", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "source_in": { + "description": "Seconds into that media", + "type": "number" + }, + "source_out": { + "type": "number" + }, + "speed": { + "additionalProperties": {}, + "description": "The clip's speed change, as clip.set_speed set it", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "start": { + "description": "Seconds into this file", + "type": "number" + }, + "track_id": { + "description": "Track of the timeline; outer/inner inside a nested one", + "type": "string" + } + }, + "required": [ + "track_id", + "kind", + "start", + "end", + "media_id", + "source_in", + "source_out" + ], + "type": "object" + }, + "type": "array" + }, + "timeline_id": { + "description": "render: the timeline that was exported; still: the edit the frame is from", + "pattern": "^tl_[0-9a-f]{32}$", + "type": "string" + }, + "tool": { + "description": "created: the create.* tool that made it", + "type": "string" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "url: the address it was downloaded from, without its query string", + "type": "string" + }, + "via": { + "description": "upload: sent through a media.upload link, dropped on the cuttingfloor site, or a data: URI", + "enum": [ + "media.upload", + "account_page", + "data_uri" + ], + "type": "string" + } + }, + "required": [ + "from" + ], + "type": "object" + }, + "page_url": { + "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", + "type": "string" + }, + "pages": { + "additionalProperties": { + "properties": { + "first": { + "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", + "type": "integer" + }, + "list": { + "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", + "type": "string" + }, + "matching": { + "description": "Items in the time range (the whole list without one)", + "type": "integer" + }, + "next_offset": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "The offset that gives the next page of this range, or null at its end" + }, + "returned": { + "type": "integer" + }, + "total": { + "description": "Items in the whole list", + "type": "integer" + } + }, + "required": [ + "list", + "first", + "returned", + "matching", + "total", + "next_offset" + ], + "type": "object" + }, + "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", + "propertyNames": { + "enum": [ + "transcript", + "beats", + "faces", + "look", + "speech", + "sounds", + "speakers", + "voice", + "colour", + "picture" + ], + "type": "string" + }, + "type": "object" + }, + "probe": { + "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", + "properties": { + "format": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "streams": { + "items": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "format", + "streams" + ], + "type": "object" + }, + "sound_seconds": { + "description": "How long the sound runs; under duration_seconds, the end is soundless", + "type": "number" + }, + "sounds": { + "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", + "properties": { + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + }, + "peaks": { + "additionalProperties": { + "type": "number" + }, + "description": "Per sound reported: the highest probability it reaches anywhere", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "scores": { + "description": "Every probability, for your own cut-off or arithmetic", + "properties": { + "start_seconds": { + "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", + "type": "number" + }, + "step_seconds": { + "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", + "type": "number" + }, + "values": { + "additionalProperties": { + "items": { + "type": "number" + }, + "type": "array" + }, + "description": "Per sound, the highest probability in each step, 0 to 1", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "start_seconds", + "step_seconds", + "values" + ], + "type": "object" + }, + "spans": { + "additionalProperties": { + "items": { + "properties": { + "end": { + "description": "Where it ends, on the same clock", + "type": "number" + }, + "peak_probability": { + "type": "number" + }, + "peak_seconds": { + "description": "The middle of the most likely frame", + "type": "number" + }, + "start": { + "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", + "type": "number" + } + }, + "required": [ + "start", + "end", + "peak_seconds", + "peak_probability" + ], + "type": "object" + }, + "type": "array" + }, + "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "model", + "min_probability", + "spans", + "peaks" + ], + "type": "object" + }, + "speech": { + "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", + "properties": { + "frame_seconds": { + "description": "Each probability covers this long", + "type": "number" + }, + "from": { + "description": "Seconds where the first probability starts", + "type": "number" + }, + "model": { + "type": "string" + }, + "probabilities": { + "description": "Speech probability 0–1, one per frame, in order", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "model", + "frame_seconds", + "from", + "probabilities" + ], + "type": "object" + }, + "status": { + "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", + "enum": [ + "importing", + "processing", + "ready", + "failed" + ], + "type": "string" + }, + "transcript": { + "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", + "properties": { + "aligned_by": { + "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", + "type": [ + "string", + "null" + ] + }, + "from": { + "description": "The stretch transcribed, if not all", + "type": "number" + }, + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "text": { + "type": "string" + }, + "until": { + "type": [ + "number", + "null" + ] + }, + "words": { + "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", + "items": { + "properties": { + "confidence": { + "description": "0–1: how sure whisper is of the word itself", + "type": "number" + }, + "end": { + "type": "number" + }, + "speaker": { + "description": "Who says it, from the latest speakers run", + "type": [ + "number", + "null" + ] + }, + "start": { + "type": "number" + }, + "text": { + "type": "string" + }, + "timing_confidence": { + "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "text", + "start", + "end", + "confidence" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "model", + "language", + "text", + "words" + ], + "type": "object" + }, + "upload": { + "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", + "properties": { + "expires_at": { + "type": "string" + }, + "max_bytes": { + "type": "number" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "Signed: PUT the file here; no other credentials needed", + "type": "string" + } + }, + "required": [ + "upload_id", + "url", + "expires_at", + "max_bytes" + ], + "type": "object" + }, + "url": { + "description": "media.get: a signed download of the file itself, once it has one", + "type": "string" + }, + "warnings": { + "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", + "items": { + "type": "string" + }, + "type": "array" + }, + "width": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "media_id", + "name", + "status", + "kind", + "duration_seconds", + "fps", + "width", + "height", + "has_video", + "has_audio", + "codecs", + "conform", + "analysis" + ], + "type": "object" + }, + { + "properties": { + "hint": { + "type": "string" + }, + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "The media it is making; usable in edits once processing", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_uri": { + "type": "string" + } + }, + "required": [ + "job_id", + "status", + "status_uri", + "hint" + ], + "type": "object" + } +]
- Changed
create.voiceover1 field changed- changed
Output schema / anyOfPrevious value: -[ - { - "properties": { - "analyses": { - "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", - "properties": { - "beats": { - "properties": { - "beats": { - "type": "number" - }, - "bpm": { - "type": "number" - } - }, - "required": [ - "bpm", - "beats" - ], - "type": "object" - }, - "colour": { - "properties": { - "frames": { - "type": "number" - }, - "regions": { - "items": { - "type": "string" - }, - "type": "array" - } - }, - "required": [ - "frames", - "regions" - ], - "type": "object" - }, - "faces": { - "properties": { - "frames_with_faces": { - "type": "number" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames_with_faces" - ], - "type": "object" - }, - "look": { - "description": "frames and questions: the latest run", - "properties": { - "frames": { - "type": "number" - }, - "questions": { - "items": { - "type": "string" - }, - "type": "array" - }, - "runs": { - "type": "number" - } - }, - "required": [ - "runs", - "frames", - "questions" - ], - "type": "object" - }, - "picture": { - "properties": { - "frames": { - "type": "number" - }, - "from": { - "type": "number" - } - }, - "required": [ - "frames", - "from" - ], - "type": "object" - }, - "sounds": { - "description": "heard: the sounds the latest run found at or above its min_probability", - "properties": { - "heard": { - "items": { - "type": "string" - }, - "type": "array" - }, - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "min_probability", - "heard" - ], - "type": "object" - }, - "speakers": { - "properties": { - "model": { - "type": "string" - }, - "speakers": { - "type": "number" - }, - "turns": { - "type": "number" - } - }, - "required": [ - "model", - "speakers", - "turns" - ], - "type": "object" - }, - "speech": { - "properties": { - "frame_seconds": { - "type": "number" - }, - "frames": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "frames", - "frame_seconds" - ], - "type": "object" - }, - "transcript": { - "properties": { - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "words": { - "description": "How many words; the words themselves are in transcript.words", - "type": "number" - } - }, - "required": [ - "model", - "language", - "words" - ], - "type": "object" - }, - "voice": { - "additionalProperties": {}, - "description": "frames, and the summary media.analyze voice gives", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "type": "object" - }, - "analysis": { - "anyOf": [ - { - "properties": { - "channels": { - "description": "Each channel at import", - "items": { - "properties": { - "channel": { - "exclusiveMinimum": 0, - "type": "integer" - }, - "noise_floor_dbfs": { - "type": [ - "number", - "null" - ] - }, - "peak_dbfs": { - "type": [ - "number", - "null" - ] - }, - "rms_dbfs": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "channel", - "rms_dbfs", - "peak_dbfs", - "noise_floor_dbfs" - ], - "type": "object" - }, - "type": "array" - }, - "loudness_lufs": { - "description": "Integrated loudness at import", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "loudness_lufs" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "beats": { - "properties": { - "beats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "bpm": { - "type": "number" - }, - "downbeats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "strengths": { - "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "bpm", - "beats", - "downbeats" - ], - "type": "object" - }, - "codecs": { - "properties": { - "audio": { - "type": [ - "string", - "null" - ] - }, - "video": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "video", - "audio" - ], - "type": "object" - }, - "conform": { - "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", - "items": { - "type": "string" - }, - "type": "array" - }, - "created_at": { - "description": "When it was imported or made", - "type": "string" - }, - "duration_seconds": { - "type": [ - "number", - "null" - ] - }, - "error": { - "properties": { - "code": { - "enum": [ - "bad_request", - "no_audio", - "not_found", - "expired", - "media_not_ready", - "media_offline", - "engine_error", - "render_failed", - "interrupted", - "stalled", - "cancelled", - "credential_required", - "credential_invalid", - "output_exists", - "path_not_allowed", - "limit_exceeded", - "quota_exceeded", - "subscription_required", - "email_unverified", - "service_unavailable", - "server_busy" - ], - "type": "string" - }, - "detail": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "hint": { - "type": "string" - }, - "message": { - "type": "string" - } - }, - "required": [ - "code", - "message" - ], - "type": "object" - }, - "expires_at": { - "description": "When url and page_url stop working", - "type": "string" - }, - "extension": { - "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", - "type": "string" - }, - "faces": { - "properties": { - "frames": { - "items": { - "properties": { - "faces": { - "items": { - "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", - "properties": { - "h": { - "type": "number" - }, - "score": { - "type": "number" - }, - "w": { - "type": "number" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "x", - "y", - "w", - "h", - "score" - ], - "type": "object" - }, - "type": "array" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "faces" - ], - "type": "object" - }, - "type": "array" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames" - ], - "type": "object" - }, - "fingerprint": { - "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", - "type": [ - "string", - "null" - ] - }, - "font_family": { - "description": "Fonts: the family; use font_media_id in a text style to set text in it", - "type": "string" - }, - "fps": { - "anyOf": [ - { - "properties": { - "den": { - "type": "number" - }, - "num": { - "type": "number" - } - }, - "required": [ - "num", - "den" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "has_audio": { - "type": "boolean" - }, - "has_video": { - "type": "boolean" - }, - "height": { - "type": [ - "number", - "null" - ] - }, - "job": { - "description": "media.get, while importing or being made: the job doing it", - "properties": { - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "progress": { - "type": "number" - }, - "seconds_left": { - "type": "number" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_message": { - "type": "string" - }, - "tool": { - "type": "string" - } - }, - "required": [ - "job_id", - "tool", - "status" - ], - "type": "object" - }, - "kind": { - "anyOf": [ - { - "enum": [ - "video", - "audio", - "image", - "font", - "lut" - ], - "type": "string" - }, - { - "type": "null" - } - ] - }, - "look": { - "properties": { - "runs": { - "items": { - "properties": { - "frames": { - "items": { - "properties": { - "answers": { - "additionalProperties": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "answers" - ], - "type": "object" - }, - "type": "array" - }, - "id": { - "type": "string" - }, - "model": { - "type": "string" - }, - "questions": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "state": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "with_previous": { - "type": "boolean" - } - }, - "required": [ - "id", - "model", - "questions", - "with_previous", - "frames" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "runs" - ], - "type": "object" - }, - "media_id": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "mime_type": { - "description": "Its MIME type, likewise", - "type": "string" - }, - "name": { - "type": "string" - }, - "origin": { - "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", - "properties": { - "attached": { - "description": "url: the file the user attached in the chat, passed as media.import file", - "type": "boolean" - }, - "file_name": { - "description": "upload: the name the file was sent with", - "type": "string" - }, - "frame": { - "description": "still: the frame", - "type": "number" - }, - "from": { - "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", - "enum": [ - "upload", - "url", - "path", - "render", - "created", - "still" - ], - "type": "string" - }, - "job_id": { - "description": "render: the export.video job; created: the create.* job", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "still: the video the frame is from", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "media_ids": { - "description": "created: the media it was made from (a cutout's source, a page's assets)", - "items": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "type": "array" - }, - "path": { - "description": "path: the file it was imported from", - "type": "string" - }, - "preset": { - "description": "render: the export preset", - "type": "string" - }, - "range": { - "description": "render: the frames exported, when not the whole timeline", - "properties": { - "end": { - "type": "number" - }, - "start": { - "type": "number" - } - }, - "required": [ - "start", - "end" - ], - "type": "object" - }, - "recipe": { - "description": "created: a hash of what it was made from; the same recipe makes the same media", - "type": "string" - }, - "sources": { - "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", - "items": { - "properties": { - "end": { - "type": "number" - }, - "kind": { - "enum": [ - "video", - "audio" - ], - "type": "string" - }, - "media_id": { - "description": "The media this stretch plays", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "source_in": { - "description": "Seconds into that media", - "type": "number" - }, - "source_out": { - "type": "number" - }, - "speed": { - "additionalProperties": {}, - "description": "The clip's speed change, as clip.set_speed set it", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "start": { - "description": "Seconds into this file", - "type": "number" - }, - "track_id": { - "description": "Track of the timeline; outer/inner inside a nested one", - "type": "string" - } - }, - "required": [ - "track_id", - "kind", - "start", - "end", - "media_id", - "source_in", - "source_out" - ], - "type": "object" - }, - "type": "array" - }, - "timeline_id": { - "description": "render: the timeline that was exported; still: the edit the frame is from", - "pattern": "^tl_[0-9a-f]{32}$", - "type": "string" - }, - "tool": { - "description": "created: the create.* tool that made it", - "type": "string" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "url: the address it was downloaded from, without its query string", - "type": "string" - }, - "via": { - "description": "upload: sent through a media.upload link, dropped on the account page, or a data: URI", - "enum": [ - "media.upload", - "account_page", - "data_uri" - ], - "type": "string" - } - }, - "required": [ - "from" - ], - "type": "object" - }, - "page_url": { - "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", - "type": "string" - }, - "pages": { - "additionalProperties": { - "properties": { - "first": { - "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", - "type": "integer" - }, - "list": { - "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", - "type": "string" - }, - "matching": { - "description": "Items in the time range (the whole list without one)", - "type": "integer" - }, - "next_offset": { - "anyOf": [ - { - "type": "integer" - }, - { - "type": "null" - } - ], - "description": "The offset that gives the next page of this range, or null at its end" - }, - "returned": { - "type": "integer" - }, - "total": { - "description": "Items in the whole list", - "type": "integer" - } - }, - "required": [ - "list", - "first", - "returned", - "matching", - "total", - "next_offset" - ], - "type": "object" - }, - "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", - "propertyNames": { - "enum": [ - "transcript", - "beats", - "faces", - "look", - "speech", - "sounds", - "speakers", - "voice", - "colour", - "picture" - ], - "type": "string" - }, - "type": "object" - }, - "probe": { - "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", - "properties": { - "format": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "streams": { - "items": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "format", - "streams" - ], - "type": "object" - }, - "sound_seconds": { - "description": "How long the sound runs; under duration_seconds, the end is soundless", - "type": "number" - }, - "sounds": { - "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", - "properties": { - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - }, - "peaks": { - "additionalProperties": { - "type": "number" - }, - "description": "Per sound reported: the highest probability it reaches anywhere", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "scores": { - "description": "Every probability, for your own cut-off or arithmetic", - "properties": { - "start_seconds": { - "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", - "type": "number" - }, - "step_seconds": { - "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", - "type": "number" - }, - "values": { - "additionalProperties": { - "items": { - "type": "number" - }, - "type": "array" - }, - "description": "Per sound, the highest probability in each step, 0 to 1", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "start_seconds", - "step_seconds", - "values" - ], - "type": "object" - }, - "spans": { - "additionalProperties": { - "items": { - "properties": { - "end": { - "description": "Where it ends, on the same clock", - "type": "number" - }, - "peak_probability": { - "type": "number" - }, - "peak_seconds": { - "description": "The middle of the most likely frame", - "type": "number" - }, - "start": { - "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", - "type": "number" - } - }, - "required": [ - "start", - "end", - "peak_seconds", - "peak_probability" - ], - "type": "object" - }, - "type": "array" - }, - "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "model", - "min_probability", - "spans", - "peaks" - ], - "type": "object" - }, - "speech": { - "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", - "properties": { - "frame_seconds": { - "description": "Each probability covers this long", - "type": "number" - }, - "from": { - "description": "Seconds where the first probability starts", - "type": "number" - }, - "model": { - "type": "string" - }, - "probabilities": { - "description": "Speech probability 0–1, one per frame, in order", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "model", - "frame_seconds", - "from", - "probabilities" - ], - "type": "object" - }, - "status": { - "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", - "enum": [ - "importing", - "processing", - "ready", - "failed" - ], - "type": "string" - }, - "transcript": { - "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", - "properties": { - "aligned_by": { - "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", - "type": [ - "string", - "null" - ] - }, - "from": { - "description": "The stretch transcribed, if not all", - "type": "number" - }, - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "text": { - "type": "string" - }, - "until": { - "type": [ - "number", - "null" - ] - }, - "words": { - "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", - "items": { - "properties": { - "confidence": { - "description": "0–1: how sure whisper is of the word itself", - "type": "number" - }, - "end": { - "type": "number" - }, - "speaker": { - "description": "Who says it, from the latest speakers run", - "type": [ - "number", - "null" - ] - }, - "start": { - "type": "number" - }, - "text": { - "type": "string" - }, - "timing_confidence": { - "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "text", - "start", - "end", - "confidence" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "model", - "language", - "text", - "words" - ], - "type": "object" - }, - "upload": { - "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", - "properties": { - "expires_at": { - "type": "string" - }, - "max_bytes": { - "type": "number" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "Signed: PUT the file here; no other credentials needed", - "type": "string" - } - }, - "required": [ - "upload_id", - "url", - "expires_at", - "max_bytes" - ], - "type": "object" - }, - "url": { - "description": "media.get: a signed download of the file itself, once it has one", - "type": "string" - }, - "warnings": { - "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", - "items": { - "type": "string" - }, - "type": "array" - }, - "width": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "media_id", - "name", - "status", - "kind", - "duration_seconds", - "fps", - "width", - "height", - "has_video", - "has_audio", - "codecs", - "conform", - "analysis" - ], - "type": "object" - }, - { - "properties": { - "hint": { - "type": "string" - }, - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "The media it is making; usable in edits once processing", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_uri": { - "type": "string" - } - }, - "required": [ - "job_id", - "status", - "status_uri", - "hint" - ], - "type": "object" - } -]New value: +[ + { + "properties": { + "analyses": { + "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", + "properties": { + "beats": { + "properties": { + "beats": { + "type": "number" + }, + "bpm": { + "type": "number" + } + }, + "required": [ + "bpm", + "beats" + ], + "type": "object" + }, + "colour": { + "properties": { + "frames": { + "type": "number" + }, + "regions": { + "items": { + "type": "string" + }, + "type": "array" + } + }, + "required": [ + "frames", + "regions" + ], + "type": "object" + }, + "faces": { + "properties": { + "frames_with_faces": { + "type": "number" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames_with_faces" + ], + "type": "object" + }, + "look": { + "description": "frames and questions: the latest run", + "properties": { + "frames": { + "type": "number" + }, + "questions": { + "items": { + "type": "string" + }, + "type": "array" + }, + "runs": { + "type": "number" + } + }, + "required": [ + "runs", + "frames", + "questions" + ], + "type": "object" + }, + "picture": { + "properties": { + "frames": { + "type": "number" + }, + "from": { + "type": "number" + } + }, + "required": [ + "frames", + "from" + ], + "type": "object" + }, + "sounds": { + "description": "heard: the sounds the latest run found at or above its min_probability", + "properties": { + "heard": { + "items": { + "type": "string" + }, + "type": "array" + }, + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "min_probability", + "heard" + ], + "type": "object" + }, + "speakers": { + "properties": { + "model": { + "type": "string" + }, + "speakers": { + "type": "number" + }, + "turns": { + "type": "number" + } + }, + "required": [ + "model", + "speakers", + "turns" + ], + "type": "object" + }, + "speech": { + "properties": { + "frame_seconds": { + "type": "number" + }, + "frames": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "frames", + "frame_seconds" + ], + "type": "object" + }, + "transcript": { + "properties": { + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "words": { + "description": "How many words; the words themselves are in transcript.words", + "type": "number" + } + }, + "required": [ + "model", + "language", + "words" + ], + "type": "object" + }, + "voice": { + "additionalProperties": {}, + "description": "frames, and the summary media.analyze voice gives", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "type": "object" + }, + "analysis": { + "anyOf": [ + { + "properties": { + "channels": { + "description": "Each channel at import", + "items": { + "properties": { + "channel": { + "exclusiveMinimum": 0, + "type": "integer" + }, + "noise_floor_dbfs": { + "type": [ + "number", + "null" + ] + }, + "peak_dbfs": { + "type": [ + "number", + "null" + ] + }, + "rms_dbfs": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "channel", + "rms_dbfs", + "peak_dbfs", + "noise_floor_dbfs" + ], + "type": "object" + }, + "type": "array" + }, + "loudness_lufs": { + "description": "Integrated loudness at import", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "loudness_lufs" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "beats": { + "properties": { + "beats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "bpm": { + "type": "number" + }, + "downbeats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "strengths": { + "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "bpm", + "beats", + "downbeats" + ], + "type": "object" + }, + "codecs": { + "properties": { + "audio": { + "type": [ + "string", + "null" + ] + }, + "video": { + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "video", + "audio" + ], + "type": "object" + }, + "conform": { + "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", + "items": { + "type": "string" + }, + "type": "array" + }, + "created_at": { + "description": "When it was imported or made", + "type": "string" + }, + "duration_seconds": { + "type": [ + "number", + "null" + ] + }, + "error": { + "properties": { + "code": { + "enum": [ + "bad_request", + "no_audio", + "not_found", + "expired", + "media_not_ready", + "media_offline", + "engine_error", + "render_failed", + "interrupted", + "stalled", + "cancelled", + "credential_required", + "credential_invalid", + "output_exists", + "path_not_allowed", + "limit_exceeded", + "quota_exceeded", + "subscription_required", + "email_unverified", + "service_unavailable", + "server_busy" + ], + "type": "string" + }, + "detail": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "hint": { + "type": "string" + }, + "message": { + "type": "string" + } + }, + "required": [ + "code", + "message" + ], + "type": "object" + }, + "expires_at": { + "description": "When url and page_url stop working", + "type": "string" + }, + "extension": { + "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", + "type": "string" + }, + "faces": { + "properties": { + "frames": { + "items": { + "properties": { + "faces": { + "items": { + "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", + "properties": { + "h": { + "type": "number" + }, + "score": { + "type": "number" + }, + "w": { + "type": "number" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "x", + "y", + "w", + "h", + "score" + ], + "type": "object" + }, + "type": "array" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "faces" + ], + "type": "object" + }, + "type": "array" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames" + ], + "type": "object" + }, + "fingerprint": { + "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", + "type": [ + "string", + "null" + ] + }, + "font_family": { + "description": "Fonts: the family; use font_media_id in a text style to set text in it", + "type": "string" + }, + "fps": { + "anyOf": [ + { + "properties": { + "den": { + "type": "number" + }, + "num": { + "type": "number" + } + }, + "required": [ + "num", + "den" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "has_audio": { + "type": "boolean" + }, + "has_video": { + "type": "boolean" + }, + "height": { + "type": [ + "number", + "null" + ] + }, + "job": { + "description": "media.get, while importing or being made: the job doing it", + "properties": { + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "progress": { + "type": "number" + }, + "seconds_left": { + "type": "number" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_message": { + "type": "string" + }, + "tool": { + "type": "string" + } + }, + "required": [ + "job_id", + "tool", + "status" + ], + "type": "object" + }, + "kind": { + "anyOf": [ + { + "enum": [ + "video", + "audio", + "image", + "font", + "lut" + ], + "type": "string" + }, + { + "type": "null" + } + ] + }, + "look": { + "properties": { + "runs": { + "items": { + "properties": { + "frames": { + "items": { + "properties": { + "answers": { + "additionalProperties": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "answers" + ], + "type": "object" + }, + "type": "array" + }, + "id": { + "type": "string" + }, + "model": { + "type": "string" + }, + "questions": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "state": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "with_previous": { + "type": "boolean" + } + }, + "required": [ + "id", + "model", + "questions", + "with_previous", + "frames" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "runs" + ], + "type": "object" + }, + "media_id": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "mime_type": { + "description": "Its MIME type, likewise", + "type": "string" + }, + "name": { + "type": "string" + }, + "origin": { + "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", + "properties": { + "attached": { + "description": "url: the file the user attached in the chat, passed as media.import file", + "type": "boolean" + }, + "file_name": { + "description": "upload: the name the file was sent with", + "type": "string" + }, + "frame": { + "description": "still: the frame", + "type": "number" + }, + "from": { + "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", + "enum": [ + "upload", + "url", + "path", + "render", + "created", + "still" + ], + "type": "string" + }, + "job_id": { + "description": "render: the export.video job; created: the create.* job", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "still: the video the frame is from", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "media_ids": { + "description": "created: the media it was made from (a cutout's source, a page's assets)", + "items": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "type": "array" + }, + "path": { + "description": "path: the file it was imported from", + "type": "string" + }, + "preset": { + "description": "render: the export preset", + "type": "string" + }, + "range": { + "description": "render: the frames exported, when not the whole timeline", + "properties": { + "end": { + "type": "number" + }, + "start": { + "type": "number" + } + }, + "required": [ + "start", + "end" + ], + "type": "object" + }, + "recipe": { + "description": "created: a hash of what it was made from; the same recipe makes the same media", + "type": "string" + }, + "sources": { + "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", + "items": { + "properties": { + "end": { + "type": "number" + }, + "kind": { + "enum": [ + "video", + "audio" + ], + "type": "string" + }, + "media_id": { + "description": "The media this stretch plays", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "source_in": { + "description": "Seconds into that media", + "type": "number" + }, + "source_out": { + "type": "number" + }, + "speed": { + "additionalProperties": {}, + "description": "The clip's speed change, as clip.set_speed set it", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "start": { + "description": "Seconds into this file", + "type": "number" + }, + "track_id": { + "description": "Track of the timeline; outer/inner inside a nested one", + "type": "string" + } + }, + "required": [ + "track_id", + "kind", + "start", + "end", + "media_id", + "source_in", + "source_out" + ], + "type": "object" + }, + "type": "array" + }, + "timeline_id": { + "description": "render: the timeline that was exported; still: the edit the frame is from", + "pattern": "^tl_[0-9a-f]{32}$", + "type": "string" + }, + "tool": { + "description": "created: the create.* tool that made it", + "type": "string" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "url: the address it was downloaded from, without its query string", + "type": "string" + }, + "via": { + "description": "upload: sent through a media.upload link, dropped on the cuttingfloor site, or a data: URI", + "enum": [ + "media.upload", + "account_page", + "data_uri" + ], + "type": "string" + } + }, + "required": [ + "from" + ], + "type": "object" + }, + "page_url": { + "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", + "type": "string" + }, + "pages": { + "additionalProperties": { + "properties": { + "first": { + "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", + "type": "integer" + }, + "list": { + "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", + "type": "string" + }, + "matching": { + "description": "Items in the time range (the whole list without one)", + "type": "integer" + }, + "next_offset": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "The offset that gives the next page of this range, or null at its end" + }, + "returned": { + "type": "integer" + }, + "total": { + "description": "Items in the whole list", + "type": "integer" + } + }, + "required": [ + "list", + "first", + "returned", + "matching", + "total", + "next_offset" + ], + "type": "object" + }, + "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", + "propertyNames": { + "enum": [ + "transcript", + "beats", + "faces", + "look", + "speech", + "sounds", + "speakers", + "voice", + "colour", + "picture" + ], + "type": "string" + }, + "type": "object" + }, + "probe": { + "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", + "properties": { + "format": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "streams": { + "items": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "format", + "streams" + ], + "type": "object" + }, + "sound_seconds": { + "description": "How long the sound runs; under duration_seconds, the end is soundless", + "type": "number" + }, + "sounds": { + "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", + "properties": { + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + }, + "peaks": { + "additionalProperties": { + "type": "number" + }, + "description": "Per sound reported: the highest probability it reaches anywhere", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "scores": { + "description": "Every probability, for your own cut-off or arithmetic", + "properties": { + "start_seconds": { + "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", + "type": "number" + }, + "step_seconds": { + "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", + "type": "number" + }, + "values": { + "additionalProperties": { + "items": { + "type": "number" + }, + "type": "array" + }, + "description": "Per sound, the highest probability in each step, 0 to 1", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "start_seconds", + "step_seconds", + "values" + ], + "type": "object" + }, + "spans": { + "additionalProperties": { + "items": { + "properties": { + "end": { + "description": "Where it ends, on the same clock", + "type": "number" + }, + "peak_probability": { + "type": "number" + }, + "peak_seconds": { + "description": "The middle of the most likely frame", + "type": "number" + }, + "start": { + "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", + "type": "number" + } + }, + "required": [ + "start", + "end", + "peak_seconds", + "peak_probability" + ], + "type": "object" + }, + "type": "array" + }, + "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "model", + "min_probability", + "spans", + "peaks" + ], + "type": "object" + }, + "speech": { + "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", + "properties": { + "frame_seconds": { + "description": "Each probability covers this long", + "type": "number" + }, + "from": { + "description": "Seconds where the first probability starts", + "type": "number" + }, + "model": { + "type": "string" + }, + "probabilities": { + "description": "Speech probability 0–1, one per frame, in order", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "model", + "frame_seconds", + "from", + "probabilities" + ], + "type": "object" + }, + "status": { + "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", + "enum": [ + "importing", + "processing", + "ready", + "failed" + ], + "type": "string" + }, + "transcript": { + "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", + "properties": { + "aligned_by": { + "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", + "type": [ + "string", + "null" + ] + }, + "from": { + "description": "The stretch transcribed, if not all", + "type": "number" + }, + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "text": { + "type": "string" + }, + "until": { + "type": [ + "number", + "null" + ] + }, + "words": { + "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", + "items": { + "properties": { + "confidence": { + "description": "0–1: how sure whisper is of the word itself", + "type": "number" + }, + "end": { + "type": "number" + }, + "speaker": { + "description": "Who says it, from the latest speakers run", + "type": [ + "number", + "null" + ] + }, + "start": { + "type": "number" + }, + "text": { + "type": "string" + }, + "timing_confidence": { + "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "text", + "start", + "end", + "confidence" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "model", + "language", + "text", + "words" + ], + "type": "object" + }, + "upload": { + "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", + "properties": { + "expires_at": { + "type": "string" + }, + "max_bytes": { + "type": "number" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "Signed: PUT the file here; no other credentials needed", + "type": "string" + } + }, + "required": [ + "upload_id", + "url", + "expires_at", + "max_bytes" + ], + "type": "object" + }, + "url": { + "description": "media.get: a signed download of the file itself, once it has one", + "type": "string" + }, + "warnings": { + "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", + "items": { + "type": "string" + }, + "type": "array" + }, + "width": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "media_id", + "name", + "status", + "kind", + "duration_seconds", + "fps", + "width", + "height", + "has_video", + "has_audio", + "codecs", + "conform", + "analysis" + ], + "type": "object" + }, + { + "properties": { + "hint": { + "type": "string" + }, + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "The media it is making; usable in edits once processing", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_uri": { + "type": "string" + } + }, + "required": [ + "job_id", + "status", + "status_uri", + "hint" + ], + "type": "object" + } +]
- Changed
media.delete1 field changed- changed
Output schema / properties / storage / properties / max_bytes / descriptionPrevious value: -"The plan's cap; null for none"New value: +"This account's storage, in bytes; null for no limit"
- Changed
media.get1 field changed- changed
Output schema / properties / origin / properties / via / descriptionPrevious value: -"upload: sent through a media.upload link, dropped on the account page, or a data: URI"New value: +"upload: sent through a media.upload link, dropped on the cuttingfloor site, or a data: URI"
- Changed
media.import1 field changed- changed
Output schema / anyOfPrevious value: -[ - { - "properties": { - "analyses": { - "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", - "properties": { - "beats": { - "properties": { - "beats": { - "type": "number" - }, - "bpm": { - "type": "number" - } - }, - "required": [ - "bpm", - "beats" - ], - "type": "object" - }, - "colour": { - "properties": { - "frames": { - "type": "number" - }, - "regions": { - "items": { - "type": "string" - }, - "type": "array" - } - }, - "required": [ - "frames", - "regions" - ], - "type": "object" - }, - "faces": { - "properties": { - "frames_with_faces": { - "type": "number" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames_with_faces" - ], - "type": "object" - }, - "look": { - "description": "frames and questions: the latest run", - "properties": { - "frames": { - "type": "number" - }, - "questions": { - "items": { - "type": "string" - }, - "type": "array" - }, - "runs": { - "type": "number" - } - }, - "required": [ - "runs", - "frames", - "questions" - ], - "type": "object" - }, - "picture": { - "properties": { - "frames": { - "type": "number" - }, - "from": { - "type": "number" - } - }, - "required": [ - "frames", - "from" - ], - "type": "object" - }, - "sounds": { - "description": "heard: the sounds the latest run found at or above its min_probability", - "properties": { - "heard": { - "items": { - "type": "string" - }, - "type": "array" - }, - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "min_probability", - "heard" - ], - "type": "object" - }, - "speakers": { - "properties": { - "model": { - "type": "string" - }, - "speakers": { - "type": "number" - }, - "turns": { - "type": "number" - } - }, - "required": [ - "model", - "speakers", - "turns" - ], - "type": "object" - }, - "speech": { - "properties": { - "frame_seconds": { - "type": "number" - }, - "frames": { - "type": "number" - }, - "model": { - "type": "string" - } - }, - "required": [ - "model", - "frames", - "frame_seconds" - ], - "type": "object" - }, - "transcript": { - "properties": { - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "words": { - "description": "How many words; the words themselves are in transcript.words", - "type": "number" - } - }, - "required": [ - "model", - "language", - "words" - ], - "type": "object" - }, - "voice": { - "additionalProperties": {}, - "description": "frames, and the summary media.analyze voice gives", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "type": "object" - }, - "analysis": { - "anyOf": [ - { - "properties": { - "channels": { - "description": "Each channel at import", - "items": { - "properties": { - "channel": { - "exclusiveMinimum": 0, - "type": "integer" - }, - "noise_floor_dbfs": { - "type": [ - "number", - "null" - ] - }, - "peak_dbfs": { - "type": [ - "number", - "null" - ] - }, - "rms_dbfs": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "channel", - "rms_dbfs", - "peak_dbfs", - "noise_floor_dbfs" - ], - "type": "object" - }, - "type": "array" - }, - "loudness_lufs": { - "description": "Integrated loudness at import", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "loudness_lufs" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "beats": { - "properties": { - "beats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "bpm": { - "type": "number" - }, - "downbeats": { - "items": { - "type": "number" - }, - "type": "array" - }, - "strengths": { - "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "bpm", - "beats", - "downbeats" - ], - "type": "object" - }, - "codecs": { - "properties": { - "audio": { - "type": [ - "string", - "null" - ] - }, - "video": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "video", - "audio" - ], - "type": "object" - }, - "conform": { - "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", - "items": { - "type": "string" - }, - "type": "array" - }, - "created_at": { - "description": "When it was imported or made", - "type": "string" - }, - "duration_seconds": { - "type": [ - "number", - "null" - ] - }, - "error": { - "properties": { - "code": { - "enum": [ - "bad_request", - "no_audio", - "not_found", - "expired", - "media_not_ready", - "media_offline", - "engine_error", - "render_failed", - "interrupted", - "stalled", - "cancelled", - "credential_required", - "credential_invalid", - "output_exists", - "path_not_allowed", - "limit_exceeded", - "quota_exceeded", - "subscription_required", - "email_unverified", - "service_unavailable", - "server_busy" - ], - "type": "string" - }, - "detail": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "hint": { - "type": "string" - }, - "message": { - "type": "string" - } - }, - "required": [ - "code", - "message" - ], - "type": "object" - }, - "expires_at": { - "description": "When url and page_url stop working", - "type": "string" - }, - "extension": { - "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", - "type": "string" - }, - "faces": { - "properties": { - "frames": { - "items": { - "properties": { - "faces": { - "items": { - "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", - "properties": { - "h": { - "type": "number" - }, - "score": { - "type": "number" - }, - "w": { - "type": "number" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "x", - "y", - "w", - "h", - "score" - ], - "type": "object" - }, - "type": "array" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "faces" - ], - "type": "object" - }, - "type": "array" - }, - "rate": { - "type": "number" - } - }, - "required": [ - "rate", - "frames" - ], - "type": "object" - }, - "fingerprint": { - "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", - "type": [ - "string", - "null" - ] - }, - "font_family": { - "description": "Fonts: the family; use font_media_id in a text style to set text in it", - "type": "string" - }, - "fps": { - "anyOf": [ - { - "properties": { - "den": { - "type": "number" - }, - "num": { - "type": "number" - } - }, - "required": [ - "num", - "den" - ], - "type": "object" - }, - { - "type": "null" - } - ] - }, - "has_audio": { - "type": "boolean" - }, - "has_video": { - "type": "boolean" - }, - "height": { - "type": [ - "number", - "null" - ] - }, - "job": { - "description": "media.get, while importing or being made: the job doing it", - "properties": { - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "progress": { - "type": "number" - }, - "seconds_left": { - "type": "number" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_message": { - "type": "string" - }, - "tool": { - "type": "string" - } - }, - "required": [ - "job_id", - "tool", - "status" - ], - "type": "object" - }, - "kind": { - "anyOf": [ - { - "enum": [ - "video", - "audio", - "image", - "font", - "lut" - ], - "type": "string" - }, - { - "type": "null" - } - ] - }, - "look": { - "properties": { - "runs": { - "items": { - "properties": { - "frames": { - "items": { - "properties": { - "answers": { - "additionalProperties": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "time": { - "type": "number" - } - }, - "required": [ - "time", - "answers" - ], - "type": "object" - }, - "type": "array" - }, - "id": { - "type": "string" - }, - "model": { - "type": "string" - }, - "questions": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "state": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "with_previous": { - "type": "boolean" - } - }, - "required": [ - "id", - "model", - "questions", - "with_previous", - "frames" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "runs" - ], - "type": "object" - }, - "media_id": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "mime_type": { - "description": "Its MIME type, likewise", - "type": "string" - }, - "name": { - "type": "string" - }, - "origin": { - "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", - "properties": { - "attached": { - "description": "url: the file the user attached in the chat, passed as media.import file", - "type": "boolean" - }, - "file_name": { - "description": "upload: the name the file was sent with", - "type": "string" - }, - "frame": { - "description": "still: the frame", - "type": "number" - }, - "from": { - "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", - "enum": [ - "upload", - "url", - "path", - "render", - "created", - "still" - ], - "type": "string" - }, - "job_id": { - "description": "render: the export.video job; created: the create.* job", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "still: the video the frame is from", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "media_ids": { - "description": "created: the media it was made from (a cutout's source, a page's assets)", - "items": { - "description": "Media handle from media.import or a create.* tool", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "type": "array" - }, - "path": { - "description": "path: the file it was imported from", - "type": "string" - }, - "preset": { - "description": "render: the export preset", - "type": "string" - }, - "range": { - "description": "render: the frames exported, when not the whole timeline", - "properties": { - "end": { - "type": "number" - }, - "start": { - "type": "number" - } - }, - "required": [ - "start", - "end" - ], - "type": "object" - }, - "recipe": { - "description": "created: a hash of what it was made from; the same recipe makes the same media", - "type": "string" - }, - "sources": { - "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", - "items": { - "properties": { - "end": { - "type": "number" - }, - "kind": { - "enum": [ - "video", - "audio" - ], - "type": "string" - }, - "media_id": { - "description": "The media this stretch plays", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "source_in": { - "description": "Seconds into that media", - "type": "number" - }, - "source_out": { - "type": "number" - }, - "speed": { - "additionalProperties": {}, - "description": "The clip's speed change, as clip.set_speed set it", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "start": { - "description": "Seconds into this file", - "type": "number" - }, - "track_id": { - "description": "Track of the timeline; outer/inner inside a nested one", - "type": "string" - } - }, - "required": [ - "track_id", - "kind", - "start", - "end", - "media_id", - "source_in", - "source_out" - ], - "type": "object" - }, - "type": "array" - }, - "timeline_id": { - "description": "render: the timeline that was exported; still: the edit the frame is from", - "pattern": "^tl_[0-9a-f]{32}$", - "type": "string" - }, - "tool": { - "description": "created: the create.* tool that made it", - "type": "string" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "url: the address it was downloaded from, without its query string", - "type": "string" - }, - "via": { - "description": "upload: sent through a media.upload link, dropped on the account page, or a data: URI", - "enum": [ - "media.upload", - "account_page", - "data_uri" - ], - "type": "string" - } - }, - "required": [ - "from" - ], - "type": "object" - }, - "page_url": { - "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", - "type": "string" - }, - "pages": { - "additionalProperties": { - "properties": { - "first": { - "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", - "type": "integer" - }, - "list": { - "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", - "type": "string" - }, - "matching": { - "description": "Items in the time range (the whole list without one)", - "type": "integer" - }, - "next_offset": { - "anyOf": [ - { - "type": "integer" - }, - { - "type": "null" - } - ], - "description": "The offset that gives the next page of this range, or null at its end" - }, - "returned": { - "type": "integer" - }, - "total": { - "description": "Items in the whole list", - "type": "integer" - } - }, - "required": [ - "list", - "first", - "returned", - "matching", - "total", - "next_offset" - ], - "type": "object" - }, - "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", - "propertyNames": { - "enum": [ - "transcript", - "beats", - "faces", - "look", - "speech", - "sounds", - "speakers", - "voice", - "colour", - "picture" - ], - "type": "string" - }, - "type": "object" - }, - "probe": { - "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", - "properties": { - "format": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "streams": { - "items": { - "additionalProperties": {}, - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "format", - "streams" - ], - "type": "object" - }, - "sound_seconds": { - "description": "How long the sound runs; under duration_seconds, the end is soundless", - "type": "number" - }, - "sounds": { - "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", - "properties": { - "min_probability": { - "type": "number" - }, - "model": { - "type": "string" - }, - "peaks": { - "additionalProperties": { - "type": "number" - }, - "description": "Per sound reported: the highest probability it reaches anywhere", - "propertyNames": { - "type": "string" - }, - "type": "object" - }, - "scores": { - "description": "Every probability, for your own cut-off or arithmetic", - "properties": { - "start_seconds": { - "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", - "type": "number" - }, - "step_seconds": { - "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", - "type": "number" - }, - "values": { - "additionalProperties": { - "items": { - "type": "number" - }, - "type": "array" - }, - "description": "Per sound, the highest probability in each step, 0 to 1", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "start_seconds", - "step_seconds", - "values" - ], - "type": "object" - }, - "spans": { - "additionalProperties": { - "items": { - "properties": { - "end": { - "description": "Where it ends, on the same clock", - "type": "number" - }, - "peak_probability": { - "type": "number" - }, - "peak_seconds": { - "description": "The middle of the most likely frame", - "type": "number" - }, - "start": { - "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", - "type": "number" - } - }, - "required": [ - "start", - "end", - "peak_seconds", - "peak_probability" - ], - "type": "object" - }, - "type": "array" - }, - "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", - "propertyNames": { - "type": "string" - }, - "type": "object" - } - }, - "required": [ - "model", - "min_probability", - "spans", - "peaks" - ], - "type": "object" - }, - "speech": { - "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", - "properties": { - "frame_seconds": { - "description": "Each probability covers this long", - "type": "number" - }, - "from": { - "description": "Seconds where the first probability starts", - "type": "number" - }, - "model": { - "type": "string" - }, - "probabilities": { - "description": "Speech probability 0–1, one per frame, in order", - "items": { - "type": "number" - }, - "type": "array" - } - }, - "required": [ - "model", - "frame_seconds", - "from", - "probabilities" - ], - "type": "object" - }, - "status": { - "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", - "enum": [ - "importing", - "processing", - "ready", - "failed" - ], - "type": "string" - }, - "transcript": { - "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", - "properties": { - "aligned_by": { - "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", - "type": [ - "string", - "null" - ] - }, - "from": { - "description": "The stretch transcribed, if not all", - "type": "number" - }, - "language": { - "type": "string" - }, - "model": { - "type": "string" - }, - "text": { - "type": "string" - }, - "until": { - "type": [ - "number", - "null" - ] - }, - "words": { - "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", - "items": { - "properties": { - "confidence": { - "description": "0–1: how sure whisper is of the word itself", - "type": "number" - }, - "end": { - "type": "number" - }, - "speaker": { - "description": "Who says it, from the latest speakers run", - "type": [ - "number", - "null" - ] - }, - "start": { - "type": "number" - }, - "text": { - "type": "string" - }, - "timing_confidence": { - "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "text", - "start", - "end", - "confidence" - ], - "type": "object" - }, - "type": "array" - } - }, - "required": [ - "model", - "language", - "text", - "words" - ], - "type": "object" - }, - "upload": { - "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", - "properties": { - "expires_at": { - "type": "string" - }, - "max_bytes": { - "type": "number" - }, - "upload_id": { - "description": "Upload handle from media.upload", - "pattern": "^u_[0-9a-z]{26}$", - "type": "string" - }, - "url": { - "description": "Signed: PUT the file here; no other credentials needed", - "type": "string" - } - }, - "required": [ - "upload_id", - "url", - "expires_at", - "max_bytes" - ], - "type": "object" - }, - "url": { - "description": "media.get: a signed download of the file itself, once it has one", - "type": "string" - }, - "warnings": { - "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", - "items": { - "type": "string" - }, - "type": "array" - }, - "width": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "media_id", - "name", - "status", - "kind", - "duration_seconds", - "fps", - "width", - "height", - "has_video", - "has_audio", - "codecs", - "conform", - "analysis" - ], - "type": "object" - }, - { - "properties": { - "hint": { - "type": "string" - }, - "job_id": { - "description": "Job handle; the same value is the MCP task id", - "pattern": "^j_[0-9a-z]{26}$", - "type": "string" - }, - "media_id": { - "description": "The media it is making; usable in edits once processing", - "pattern": "^m_[0-9a-z]{26}$", - "type": "string" - }, - "status": { - "enum": [ - "working", - "completed", - "failed", - "cancelled" - ], - "type": "string" - }, - "status_uri": { - "type": "string" - } - }, - "required": [ - "job_id", - "status", - "status_uri", - "hint" - ], - "type": "object" - } -]New value: +[ + { + "properties": { + "analyses": { + "description": "Summaries of media.analyze results (counts, not data); include puts each in full at the top level", + "properties": { + "beats": { + "properties": { + "beats": { + "type": "number" + }, + "bpm": { + "type": "number" + } + }, + "required": [ + "bpm", + "beats" + ], + "type": "object" + }, + "colour": { + "properties": { + "frames": { + "type": "number" + }, + "regions": { + "items": { + "type": "string" + }, + "type": "array" + } + }, + "required": [ + "frames", + "regions" + ], + "type": "object" + }, + "faces": { + "properties": { + "frames_with_faces": { + "type": "number" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames_with_faces" + ], + "type": "object" + }, + "look": { + "description": "frames and questions: the latest run", + "properties": { + "frames": { + "type": "number" + }, + "questions": { + "items": { + "type": "string" + }, + "type": "array" + }, + "runs": { + "type": "number" + } + }, + "required": [ + "runs", + "frames", + "questions" + ], + "type": "object" + }, + "picture": { + "properties": { + "frames": { + "type": "number" + }, + "from": { + "type": "number" + } + }, + "required": [ + "frames", + "from" + ], + "type": "object" + }, + "sounds": { + "description": "heard: the sounds the latest run found at or above its min_probability", + "properties": { + "heard": { + "items": { + "type": "string" + }, + "type": "array" + }, + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "min_probability", + "heard" + ], + "type": "object" + }, + "speakers": { + "properties": { + "model": { + "type": "string" + }, + "speakers": { + "type": "number" + }, + "turns": { + "type": "number" + } + }, + "required": [ + "model", + "speakers", + "turns" + ], + "type": "object" + }, + "speech": { + "properties": { + "frame_seconds": { + "type": "number" + }, + "frames": { + "type": "number" + }, + "model": { + "type": "string" + } + }, + "required": [ + "model", + "frames", + "frame_seconds" + ], + "type": "object" + }, + "transcript": { + "properties": { + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "words": { + "description": "How many words; the words themselves are in transcript.words", + "type": "number" + } + }, + "required": [ + "model", + "language", + "words" + ], + "type": "object" + }, + "voice": { + "additionalProperties": {}, + "description": "frames, and the summary media.analyze voice gives", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "type": "object" + }, + "analysis": { + "anyOf": [ + { + "properties": { + "channels": { + "description": "Each channel at import", + "items": { + "properties": { + "channel": { + "exclusiveMinimum": 0, + "type": "integer" + }, + "noise_floor_dbfs": { + "type": [ + "number", + "null" + ] + }, + "peak_dbfs": { + "type": [ + "number", + "null" + ] + }, + "rms_dbfs": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "channel", + "rms_dbfs", + "peak_dbfs", + "noise_floor_dbfs" + ], + "type": "object" + }, + "type": "array" + }, + "loudness_lufs": { + "description": "Integrated loudness at import", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "loudness_lufs" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "beats": { + "properties": { + "beats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "bpm": { + "type": "number" + }, + "downbeats": { + "items": { + "type": "number" + }, + "type": "array" + }, + "strengths": { + "description": "Per beat: its onset strength, 0–1 of the strongest in the sound", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "bpm", + "beats", + "downbeats" + ], + "type": "object" + }, + "codecs": { + "properties": { + "audio": { + "type": [ + "string", + "null" + ] + }, + "video": { + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "video", + "audio" + ], + "type": "object" + }, + "conform": { + "description": "Why a mezzanine was made (vfr, rotation, hdr, codec); empty when not needed", + "items": { + "type": "string" + }, + "type": "array" + }, + "created_at": { + "description": "When it was imported or made", + "type": "string" + }, + "duration_seconds": { + "type": [ + "number", + "null" + ] + }, + "error": { + "properties": { + "code": { + "enum": [ + "bad_request", + "no_audio", + "not_found", + "expired", + "media_not_ready", + "media_offline", + "engine_error", + "render_failed", + "interrupted", + "stalled", + "cancelled", + "credential_required", + "credential_invalid", + "output_exists", + "path_not_allowed", + "limit_exceeded", + "quota_exceeded", + "subscription_required", + "email_unverified", + "service_unavailable", + "server_busy" + ], + "type": "string" + }, + "detail": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "hint": { + "type": "string" + }, + "message": { + "type": "string" + } + }, + "required": [ + "code", + "message" + ], + "type": "object" + }, + "expires_at": { + "description": "When url and page_url stop working", + "type": "string" + }, + "extension": { + "description": "The extension its format takes (.png, .mp4), from the file itself, not its name", + "type": "string" + }, + "faces": { + "properties": { + "frames": { + "items": { + "properties": { + "faces": { + "items": { + "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1", + "properties": { + "h": { + "type": "number" + }, + "score": { + "type": "number" + }, + "w": { + "type": "number" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "x", + "y", + "w", + "h", + "score" + ], + "type": "object" + }, + "type": "array" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "faces" + ], + "type": "object" + }, + "type": "array" + }, + "rate": { + "type": "number" + } + }, + "required": [ + "rate", + "frames" + ], + "type": "object" + }, + "fingerprint": { + "description": "Identity of the file (BLAKE3 of its size and first, middle and last 4 MB), as export.video gives", + "type": [ + "string", + "null" + ] + }, + "font_family": { + "description": "Fonts: the family; use font_media_id in a text style to set text in it", + "type": "string" + }, + "fps": { + "anyOf": [ + { + "properties": { + "den": { + "type": "number" + }, + "num": { + "type": "number" + } + }, + "required": [ + "num", + "den" + ], + "type": "object" + }, + { + "type": "null" + } + ] + }, + "has_audio": { + "type": "boolean" + }, + "has_video": { + "type": "boolean" + }, + "height": { + "type": [ + "number", + "null" + ] + }, + "job": { + "description": "media.get, while importing or being made: the job doing it", + "properties": { + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "progress": { + "type": "number" + }, + "seconds_left": { + "type": "number" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_message": { + "type": "string" + }, + "tool": { + "type": "string" + } + }, + "required": [ + "job_id", + "tool", + "status" + ], + "type": "object" + }, + "kind": { + "anyOf": [ + { + "enum": [ + "video", + "audio", + "image", + "font", + "lut" + ], + "type": "string" + }, + { + "type": "null" + } + ] + }, + "look": { + "properties": { + "runs": { + "items": { + "properties": { + "frames": { + "items": { + "properties": { + "answers": { + "additionalProperties": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "time": { + "type": "number" + } + }, + "required": [ + "time", + "answers" + ], + "type": "object" + }, + "type": "array" + }, + "id": { + "type": "string" + }, + "model": { + "type": "string" + }, + "questions": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "state": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "with_previous": { + "type": "boolean" + } + }, + "required": [ + "id", + "model", + "questions", + "with_previous", + "frames" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "runs" + ], + "type": "object" + }, + "media_id": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "mime_type": { + "description": "Its MIME type, likewise", + "type": "string" + }, + "name": { + "type": "string" + }, + "origin": { + "description": "Where it came from, to tell one version from another: the import source, the export of which timeline, or the create.* tool and the media it was made from", + "properties": { + "attached": { + "description": "url: the file the user attached in the chat, passed as media.import file", + "type": "boolean" + }, + "file_name": { + "description": "upload: the name the file was sent with", + "type": "string" + }, + "frame": { + "description": "still: the frame", + "type": "number" + }, + "from": { + "description": "upload: a file the user sent; url: downloaded; path: a media root; render: an export.video; created: a create.* tool; still: a frame of a video or an edit", + "enum": [ + "upload", + "url", + "path", + "render", + "created", + "still" + ], + "type": "string" + }, + "job_id": { + "description": "render: the export.video job; created: the create.* job", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "still: the video the frame is from", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "media_ids": { + "description": "created: the media it was made from (a cutout's source, a page's assets)", + "items": { + "description": "Media handle from media.import or a create.* tool", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "type": "array" + }, + "path": { + "description": "path: the file it was imported from", + "type": "string" + }, + "preset": { + "description": "render: the export preset", + "type": "string" + }, + "range": { + "description": "render: the frames exported, when not the whole timeline", + "properties": { + "end": { + "type": "number" + }, + "start": { + "type": "number" + } + }, + "required": [ + "start", + "end" + ], + "type": "object" + }, + "recipe": { + "description": "created: a hash of what it was made from; the same recipe makes the same media", + "type": "string" + }, + "sources": { + "description": "render: where each stretch of this file comes from, per track, read from the timeline when it was exported (kept after the timeline expires). With a speed curve, source_in and source_out are the clip's whole source range", + "items": { + "properties": { + "end": { + "type": "number" + }, + "kind": { + "enum": [ + "video", + "audio" + ], + "type": "string" + }, + "media_id": { + "description": "The media this stretch plays", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "source_in": { + "description": "Seconds into that media", + "type": "number" + }, + "source_out": { + "type": "number" + }, + "speed": { + "additionalProperties": {}, + "description": "The clip's speed change, as clip.set_speed set it", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "start": { + "description": "Seconds into this file", + "type": "number" + }, + "track_id": { + "description": "Track of the timeline; outer/inner inside a nested one", + "type": "string" + } + }, + "required": [ + "track_id", + "kind", + "start", + "end", + "media_id", + "source_in", + "source_out" + ], + "type": "object" + }, + "type": "array" + }, + "timeline_id": { + "description": "render: the timeline that was exported; still: the edit the frame is from", + "pattern": "^tl_[0-9a-f]{32}$", + "type": "string" + }, + "tool": { + "description": "created: the create.* tool that made it", + "type": "string" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "url: the address it was downloaded from, without its query string", + "type": "string" + }, + "via": { + "description": "upload: sent through a media.upload link, dropped on the cuttingfloor site, or a data: URI", + "enum": [ + "media.upload", + "account_page", + "data_uri" + ], + "type": "string" + } + }, + "required": [ + "from" + ], + "type": "object" + }, + "page_url": { + "description": "media.get: a page that plays or shows the file in any browser, with no sign-in, to give a person; works until expires_at", + "type": "string" + }, + "pages": { + "additionalProperties": { + "properties": { + "first": { + "description": "Index in the whole list of the first item returned (transcript: the word index clip.add source {words} takes)", + "type": "integer" + }, + "list": { + "description": "The list paged: transcript.words, beats.beats, faces.frames, speech.probabilities (one per frame_seconds from `from`), speakers.turns, voice.frames, colour.frames, picture.columns (every column alike), or look runs[].frames (each run is cut the same way; first and the counts are the latest run's)", + "type": "string" + }, + "matching": { + "description": "Items in the time range (the whole list without one)", + "type": "integer" + }, + "next_offset": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "The offset that gives the next page of this range, or null at its end" + }, + "returned": { + "type": "integer" + }, + "total": { + "description": "Items in the whole list", + "type": "integer" + } + }, + "required": [ + "list", + "first", + "returned", + "matching", + "total", + "next_offset" + ], + "type": "object" + }, + "description": "media.get with a range, offset or limit: per included analysis, which piece of its list came back", + "propertyNames": { + "enum": [ + "transcript", + "beats", + "faces", + "look", + "speech", + "sounds", + "speakers", + "voice", + "colour", + "picture" + ], + "type": "string" + }, + "type": "object" + }, + "probe": { + "description": "media.get: everything FFprobe read from the file, as it gives it: the format and every stream (codec, pix_fmt, color_space, color_primaries, color_transfer, color_range, bit depth, sample rate, channel layout, tags, side data). Not for fonts and LUTs", + "properties": { + "format": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "streams": { + "items": { + "additionalProperties": {}, + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "format", + "streams" + ], + "type": "object" + }, + "sound_seconds": { + "description": "How long the sound runs; under duration_seconds, the end is soundless", + "type": "number" + }, + "sounds": { + "description": "With include [\"sounds\"]: the latest media.analyze sounds run, every probability included", + "properties": { + "min_probability": { + "type": "number" + }, + "model": { + "type": "string" + }, + "peaks": { + "additionalProperties": { + "type": "number" + }, + "description": "Per sound reported: the highest probability it reaches anywhere", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "scores": { + "description": "Every probability, for your own cut-off or arithmetic", + "properties": { + "start_seconds": { + "description": "Where the first value starts: media seconds, or timeline seconds for a timeline", + "type": "number" + }, + "step_seconds": { + "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long", + "type": "number" + }, + "values": { + "additionalProperties": { + "items": { + "type": "number" + }, + "type": "array" + }, + "description": "Per sound, the highest probability in each step, 0 to 1", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "start_seconds", + "step_seconds", + "values" + ], + "type": "object" + }, + "spans": { + "additionalProperties": { + "items": { + "properties": { + "end": { + "description": "Where it ends, on the same clock", + "type": "number" + }, + "peak_probability": { + "type": "number" + }, + "peak_seconds": { + "description": "The middle of the most likely frame", + "type": "number" + }, + "start": { + "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline", + "type": "number" + } + }, + "required": [ + "start", + "end", + "peak_seconds", + "peak_probability" + ], + "type": "object" + }, + "type": "array" + }, + "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "model", + "min_probability", + "spans", + "peaks" + ], + "type": "object" + }, + "speech": { + "description": "With include [\"speech\"]: the speech probability of every 32 ms frame", + "properties": { + "frame_seconds": { + "description": "Each probability covers this long", + "type": "number" + }, + "from": { + "description": "Seconds where the first probability starts", + "type": "number" + }, + "model": { + "type": "string" + }, + "probabilities": { + "description": "Speech probability 0–1, one per frame, in order", + "items": { + "type": "number" + }, + "type": "array" + } + }, + "required": [ + "model", + "frame_seconds", + "from", + "probabilities" + ], + "type": "object" + }, + "status": { + "description": "importing: not probed yet; processing: usable in edits, proxy still rendering; ready; failed", + "enum": [ + "importing", + "processing", + "ready", + "failed" + ], + "type": "string" + }, + "transcript": { + "description": "With include [\"transcript\"]: every word as {text, start, end, confidence}", + "properties": { + "aligned_by": { + "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off", + "type": [ + "string", + "null" + ] + }, + "from": { + "description": "The stretch transcribed, if not all", + "type": "number" + }, + "language": { + "type": "string" + }, + "model": { + "type": "string" + }, + "text": { + "type": "string" + }, + "until": { + "type": [ + "number", + "null" + ] + }, + "words": { + "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when", + "items": { + "properties": { + "confidence": { + "description": "0–1: how sure whisper is of the word itself", + "type": "number" + }, + "end": { + "type": "number" + }, + "speaker": { + "description": "Who says it, from the latest speakers run", + "type": [ + "number", + "null" + ] + }, + "start": { + "type": "number" + }, + "text": { + "type": "string" + }, + "timing_confidence": { + "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours", + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "text", + "start", + "end", + "confidence" + ], + "type": "object" + }, + "type": "array" + } + }, + "required": [ + "model", + "language", + "text", + "words" + ], + "type": "object" + }, + "upload": { + "description": "media.import {source: {upload}}: where to send the file. It imports once the last byte arrives (the PUT answers with the job_id); media.get says where it stands", + "properties": { + "expires_at": { + "type": "string" + }, + "max_bytes": { + "type": "number" + }, + "upload_id": { + "description": "Upload handle from media.upload", + "pattern": "^u_[0-9a-z]{26}$", + "type": "string" + }, + "url": { + "description": "Signed: PUT the file here; no other credentials needed", + "type": "string" + } + }, + "required": [ + "upload_id", + "url", + "expires_at", + "max_bytes" + ], + "type": "object" + }, + "url": { + "description": "media.get: a signed download of the file itself, once it has one", + "type": "string" + }, + "warnings": { + "description": "create.motion and create.still: what went wrong in the page as it rendered (uncaught errors, console errors, files blocked or not found, tags never closed). It rendered anyway", + "items": { + "type": "string" + }, + "type": "array" + }, + "width": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "media_id", + "name", + "status", + "kind", + "duration_seconds", + "fps", + "width", + "height", + "has_video", + "has_audio", + "codecs", + "conform", + "analysis" + ], + "type": "object" + }, + { + "properties": { + "hint": { + "type": "string" + }, + "job_id": { + "description": "Job handle; the same value is the MCP task id", + "pattern": "^j_[0-9a-z]{26}$", + "type": "string" + }, + "media_id": { + "description": "The media it is making; usable in edits once processing", + "pattern": "^m_[0-9a-z]{26}$", + "type": "string" + }, + "status": { + "enum": [ + "working", + "completed", + "failed", + "cancelled" + ], + "type": "string" + }, + "status_uri": { + "type": "string" + } + }, + "required": [ + "job_id", + "status", + "status_uri", + "hint" + ], + "type": "object" + } +]
- Changed
media.list3 fields changed- changed
Output schema / properties / media / items / properties / origin / properties / via / descriptionPrevious value: -"upload: sent through a media.upload link, dropped on the account page, or a data: URI"New value: +"upload: sent through a media.upload link, dropped on the cuttingfloor site, or a data: URI" - changed
Output schema / properties / storage / properties / max_bytes / descriptionPrevious value: -"The plan's cap; null for none"New value: +"This account's storage, in bytes; null for no limit" - changed
Output schema / properties / uploads / descriptionPrevious value: -"Files sent to the server and not imported yet, newest first (up to 200): through media.upload, or dropped on the account page by the user"New value: +"Files sent to the server and not imported yet, newest first (up to 200): through media.upload, or dropped on the cuttingfloor site by the user"
1 tool update
- Removed
account.get
1 tool update
- Added
account.get
1 tool update
- Changed
skills.get3 fields changed- added
Output schema / properties / authorAdded value: +{ + "description": "Who wrote the skill; credit them when you name it", + "type": "string" +} - added
Output schema / properties / author_urlAdded value: +{ + "anyOf": [ + { + "format": "uri", + "type": "string" + }, + { + "type": "null" + } + ], + "description": "A link to the author or the work the skill came from; null when there is none" +} - changed
Output schema / requiredPrevious value: -[ - "slug", - "name", - "description", - "version", - "related", - "markdown" -]New value: +[ + "slug", + "name", + "description", + "version", + "author", + "author_url", + "related", + "markdown" +]
Related MCP Connectors
AI video editor your agent can drive: plan scenes, edit a multi-track timeline, and render MP4s.
The agentic video editor for ChatGPT and Claude. Ask in your chat and it cuts, captions, adds music, titles and motion graphics on a real timeline you can open and adjust.
FFmpeg as a service for AI agents: typed video editing tools, async jobs, downloadable outputs.
Compose, edit and render video with your AI agent. Turn footage into finished edits with content-aware reframing, word-timed captions and motion graphics. Hosted MCP with OAuth sign-in; cloud processing uses credits.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables AI agents to edit videos through natural language, providing tools for timeline editing, audio management, rendering, and more.2MIT
- FlicenseNot gradedqualityBmaintenanceEnables AI agents to read, edit, and export real video projects with a full multitrack timeline, effects, captions, and export capabilities.2-
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to autonomously edit raw video footage into publish-ready videos with millisecond-accurate cuts, Whisper transcription, karaoke captions, motion graphics, and high-CTR thumbnails.19 npm6MIT
- AlicenseNot gradedqualityBmaintenanceValmera — the world’s most advanced AI video editor. Edit recorded tutorials, podcasts, interviews and webinars through Claude, ChatGPT or another compatible MCP client. Cut footage, add captions, reframe, review a preview, revise specific sections and export MP4. Hosted editing with OAuth; an eligible paid Valmera plan is required.MIT
Glama MCP Gateway
Add one secure layer between your agents and this server.