Skip to main content
Glama
Sign In

cuttingfloor

by io.github.Mvp2o-ai
OverviewDiscussionsAdmin

Analyze media

media.analyze
DestructiveIdempotent

Find what is in imported media, or in the sound of an edit, as data for edits. transcript: every word with its time (whisper.cpp on the CPU; the model downloads once). For English the word edges are then aligned to the sound (wav2vec2 forced alignment): a word starts and ends within about 0.05 s of where its letters sound, and an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more; timing_confidence says how clearly each word was heard. Other languages keep whisper's own times, up to about a second off. Short ums and uhs are often left out of the words; kind speech still hears them. from_seconds and until_seconds transcribe just that stretch. beats: tempo, beat and downbeat times, and strengths: each beat's onset strength, 0–1 of the strongest (the hardest hits, the drop). faces: face boxes over time, e.g. to reframe to 9:16 with clip.set_transform keyframes. track: follow a region you give from a moment on, a box for each sampled time (found false where the subject was lost); to make a title, sticker, arrow or mask follow the subject, turn the boxes into clip.set_transform (or clip.set_mask) keyframes, no tracking of your own needed. look: your own typed questions about what is on screen, answered on every sampled frame (to find where something happens from plain words, media.seek). silence: stretches quieter than threshold_db for at least min_seconds. scenes: times the picture changes by more than threshold (0–100; hard cuts between ordinary shots score about 10 to 30, see threshold), with each cut's score. speech: the probability that someone is speaking, every 32 ms (Silero VAD), and the spans where it is at least your min_probability; it hears speech, not noise or music, so a cough or a door is not speech. loudness: integrated loudness (LUFS), true peak (dBTP), loudness range (LU), RMS level and noise floor (dBFS) over the whole media or from_seconds to until_seconds. To line up two recordings, media.correlate. sounds: what can be heard and when, every 40 ms, for 447 named sounds (video://catalog/sounds: Laughter, Applause, Cough, Sneeze, Music, Speech, Door, Footsteps, Beep…): give labels for the ones you want, or none for every sound that reaches your min_probability; you get each sound's spans and its probability in every frame (a run over the whole media is kept with it; one over from_seconds to until_seconds comes back only). To find a sound you describe in your own words, media.listen. speakers: who speaks when (pyannote and ERes2Net on the CPU, about 10 to 20 s a minute): turns numbered in the order voices are first heard; give speakers (how many) or cluster_threshold. Transcript words then carry their speaker. voice: pitch (YIN, Hz) and level (RMS dBFS) every 0.1 s, and a summary (pitch median and range, level, voiced seconds, words per second from the transcript), e.g. as judge.ask state. colour: per-channel mean and percentiles, luma, white point estimate and clipping (RGB 0–255) of the frame and regions you name, at sampled times. picture: FFmpeg signalstats of every frame over a stretch: luma, chroma, saturation, hue, and ydif, the change from the frame before (a flash or frozen frame shows there). With timeline_id instead of media_id, speech, loudness, silence, sounds, speakers, voice measure the edit's mix and faces, track, look, scenes, colour, picture its picture as it renders (titles, effects and grades included), over from_seconds to until_seconds (timeline seconds); they come back in the result. A source or range with no sound fails with no_audio, naming it. Times are media seconds (timeline seconds for a timeline); boxes are fractions of the picture, and a box you give takes units "pixels" too (picture_size gives the size). Results stay with the media (media.get include; the latest run of each kind) except track, silence, scenes and loudness, which come back in the result; speech over the whole media is kept (media.get include speech), so asking again with another min_probability or range answers from it. A result too long to return (a long transcript) comes back as its summary and a note: read it in pieces with media.get include and a time range or page. Long-running: returns a task, or a job_id for job.get.

HOW LOOK WORKS. TypeSafe Jev, a fast judgment model (preview), answers your questions about frames sampled from the media. Each sampled frame is its own request: Jev sees that one picture, named frame, and with with_previous also the frame sampled just before it, named previous, plus any state you give: text or JSON entries by name, the same on every frame (the script, what a term means). Every question is answered on every frame, independently and in parallel (about 0.15 s a frame), as typed data you then combine in your own code. Choose the type by the answer you need. noul: yes or no, answered as noul, the probability of yes from 0 to 1. choice: one of options you name, answered as choice plus probabilities and confidence. score: a degree on levels you order, answered as score (0 is the first level; it can land between levels) plus confidence. box: where one thing is, answered as box {x, y, w, h} in fractions of the picture plus confidence. Write each question about the picture as it looks. Name the picture in backticks ("in frame"), ask about one thing, word it so yes means the thing is there, and describe it as it appears: "Does frame show a human being, even if only their face fills the picture?", "Is one large word laid over frame, much bigger than any other text?". To compare, name both pictures and the thing that differs: "Is there a button in frame that is not in previous?". To locate, name the one thing so only it fits ("the 'Get started for free' button under the headline in frame") and, in the same call, a noul asking whether it is there; use the box on frames where that noul is high. Give a choice a none option for frames where nothing fits, and each score level a concrete situation. Read the answers this way: a noul of 0.5 or more reads as yes, and you can set your own cut-off; confidence tells you how far to trust a choice, a score or a box, and a loose box shows as low confidence. Work in passes: every_seconds 1 over the whole media, then at_seconds around the times where answers change. media.get scenes give when the picture cuts; look tells you what is on it. The result gives each question's spans (when a noul reads yes, which choice holds when); every frame's answers stay with the media (media.get include ['look'], the last 10 runs). Up to 600 frames and 32 questions a call. To find the moments that match a description, media.seek takes it in plain words and returns where it holds. For one judgment about text, a still or a single frame, judge.ask answers at once. Full guide to writing questions: https://docs.typesafe.ai/primitives and https://docs.typesafe.ai/images.

Input Schema

NameRequiredDescriptionDefault
kindYestranscript: words with times (whisper.cpp, then aligned to the sound); beats: tempo, beat times and the onset strength at each; faces: face boxes over time; track: follow a region; look: your own questions answered on sampled frames; silence: stretches quieter than threshold_db for min_seconds; scenes: times the picture changes by more than threshold; speech: where someone is speaking (Silero VAD probability every 32 ms, spans at or above min_probability); loudness: integrated loudness, true peak, loudness range, RMS and noise floor; sounds: when each of 447 named sounds (laughter, applause, cough, music, speech, door…) can be heard, every 40 ms; speakers: who speaks when; voice: pitch, level and pace over time; colour: colour statistics of frames and regions; picture: every frame's brightness, colour and change from the frame before
modelNotranscript: whisper size, on the server CPU. Bigger is more accurate and slower: small takes about 3 times as long as base, tiny about halfbase
stateNolook: text or JSON entries shown beside `frame` on every frame, by name, for questions to refer to (a script line, a definition)
labelsNosounds: the sounds to report, named as in video://catalog/sounds (e.g. ["Laughter", "Applause", "Cough"]). Default: every sound that reaches min_probability somewhere
regionNotrack: the box to follow (top-left x, y and size) and when
regionsNocolour: named boxes to measure besides the whole frame (frame), up to 16
languageNotranscript: e.g. 'en', or 'auto'auto
media_idNoAnalyze imported media (media seconds)
speakersNospeakers: how many people speak, when you know (or give cluster_threshold)
questionsNolook: id (yours, not shown to the model) to question; up to 32, all asked of every frame
thresholdNoscenes (required): how much the picture must change to count as a cut, 0–100 (FFmpeg scdet's score): the change from the frame before, in percent of full range, less the change just before it (so steady motion stays low). Hard cuts between ordinary shots score about 10 to 30, a cut between opposite pictures up to 100: start at FFmpeg's own default, 10, and raise it if you get too many
at_secondsNolook, colour: sample exactly these times instead
min_secondsNosilence (required): only stretches at least this long
percentilesNocolour: the percentiles to give of each channel
timeline_idNoAnalyze an edit as it plays (timeline seconds): its mix (speech, loudness, silence, sounds, speakers, voice) or its picture (faces, track, look, scenes, colour, picture)
from_secondsNoStart sampling or measuring here (any kind but beats, and faces and scenes on media). Default 0
threshold_dbNosilence (required): quieter than this counts as silence, dBFS
every_secondsNolook, colour: sample a frame this often. Default 1
until_secondsNoStop sampling or measuring here. Default: the end
with_previousNolook: also show the frame sampled before, as `previous`
idempotency_keyNo
min_probabilityNospeech (required): a 32 ms frame with at least this speech probability is speech; sounds (required): a 40 ms frame has the sound when its probability is at least this
cluster_thresholdNospeakers: when the count is unknown, how different two voices must be to count as two people; lower finds more speakers (try 0.5 to 0.9)

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed25 schema fields changed
    • changedInput schema / properties / at_seconds / description
      Previous value: -"look: sample exactly these times instead"New value: +"look, colour: sample exactly these times instead"
    • addedInput schema / properties / cluster_threshold
      Added value: +{
+  "description": "speakers: when the count is unknown, how different two voices must be to count as two people; lower finds more speakers (try 0.5 to 0.9)",
+  "exclusiveMinimum": 0,
+  "maximum": 10,
+  "type": "number"
+}
    • changedInput schema / properties / every_seconds / description
      Previous value: -"look: sample a frame this often. Default 1"New value: +"look, colour: sample a frame this often. Default 1"
    • changedInput schema / properties / from_seconds / description
      Previous value: -"look: start sampling here; speech, loudness, silence, sounds: measure from here. Default 0"New value: +"Start sampling or measuring here (any kind but beats, and faces and scenes on media). Default 0"
    • changedInput schema / properties / kind / description
      Previous value: -"transcript: words with times (whisper.cpp, then aligned to the sound); beats: tempo and beat times; faces: face boxes over time; track: follow a region; look: your own questions answered on sampled frames; silence: stretches quieter than threshold_db for min_seconds; scenes: times the picture changes by more than threshold; speech: where someone is speaking (Silero VAD probability every 32 ms, spans at or above min_probability); loudness: integrated loudness, true peak, loudness range, RMS and noise floor; sounds: when each of 447 named sounds (laughter, applause, cough, music, speech, door…) can be heard, every 40 ms"New value: +"transcript: words with times (whisper.cpp, then aligned to the sound); beats: tempo, beat times and the onset strength at each; faces: face boxes over time; track: follow a region; look: your own questions answered on sampled frames; silence: stretches quieter than threshold_db for min_seconds; scenes: times the picture changes by more than threshold; speech: where someone is speaking (Silero VAD probability every 32 ms, spans at or above min_probability); loudness: integrated loudness, true peak, loudness range, RMS and noise floor; sounds: when each of 447 named sounds (laughter, applause, cough, music, speech, door…) can be heard, every 40 ms; speakers: who speaks when; voice: pitch, level and pace over time; colour: colour statistics of frames and regions; picture: every frame's brightness, colour and change from the frame before"
    • changedInput schema / properties / kind / enum
      Previous value: -[
-  "transcript",
-  "beats",
-  "faces",
-  "track",
-  "look",
-  "silence",
-  "scenes",
-  "speech",
-  "loudness",
-  "sounds"
-]New value: +[
+  "transcript",
+  "beats",
+  "faces",
+  "track",
+  "look",
+  "silence",
+  "scenes",
+  "speech",
+  "loudness",
+  "sounds",
+  "speakers",
+  "voice",
+  "colour",
+  "picture"
+]
    • changedInput schema / properties / model / description
      Previous value: -"transcript: speed vs accuracy"New value: +"transcript: whisper size, on the server CPU. Bigger is more accurate and slower: small takes about 3 times as long as base, tiny about half"
    • addedInput schema / properties / percentiles
      Added value: +{
+  "default": [
+    5,
+    50,
+    95
+  ],
+  "description": "colour: the percentiles to give of each channel",
+  "items": {
+    "maximum": 100,
+    "minimum": 0,
+    "type": "number"
+  },
+  "maxItems": 20,
+  "minItems": 1,
+  "type": "array"
+}
    • changedInput schema / properties / region / description
      Previous value: -"track: the box to follow (top-left x, y and size, fractions of the picture) and when"New value: +"track: the box to follow (top-left x, y and size) and when"
    • addedInput schema / properties / region / properties / h / exclusiveMinimum
      Added value: +0
    • removedInput schema / properties / region / properties / h / maximum
      Removed value: -1
    • removedInput schema / properties / region / properties / h / minimum
      Removed value: -0.005
    • addedInput schema / properties / region / properties / units
      Added value: +{
+  "default": "fraction",
+  "description": "fraction (default): of the picture; pixels: of the picture at its own size",
+  "enum": [
+    "fraction",
+    "pixels"
+  ],
+  "type": "string"
+}
    • addedInput schema / properties / region / properties / w / exclusiveMinimum
      Added value: +0
    • removedInput schema / properties / region / properties / w / maximum
      Removed value: -1
    • removedInput schema / properties / region / properties / w / minimum
      Removed value: -0.005
    • removedInput schema / properties / region / properties / x / maximum
      Removed value: -1
    • removedInput schema / properties / region / properties / y / maximum
      Removed value: -1
    • changedInput schema / properties / region / required
      Previous value: -[
-  "at_seconds",
-  "x",
-  "y",
-  "w",
-  "h"
-]New value: +[
+  "x",
+  "y",
+  "w",
+  "h",
+  "at_seconds"
+]
    • addedInput schema / properties / regions
      Added value: +{
+  "additionalProperties": {
+    "properties": {
+      "h": {
+        "exclusiveMinimum": 0,
+        "type": "number"
+      },
+      "units": {
+        "default": "fraction",
+        "description": "fraction (default): of the picture; pixels: of the picture at its own size",
+        "enum": [
+          "fraction",
+          "pixels"
+        ],
+        "type": "string"
+      },
+      "w": {
+        "exclusiveMinimum": 0,
+        "type": "number"
+      },
+      "x": {
+        "minimum": 0,
+        "type": "number"
+      },
+      "y": {
+        "minimum": 0,
+        "type": "number"
+      }
+    },
+    "required": [
+      "x",
+      "y",
+      "w",
+      "h"
+    ],
+    "type": "object"
+  },
+  "description": "colour: named boxes to measure besides the whole frame (frame), up to 16",
+  "propertyNames": {
+    "pattern": "^[A-Za-z][A-Za-z0-9_]{0,63}$",
+    "type": "string"
+  },
+  "type": "object"
+}
    • addedInput schema / properties / speakers
      Added value: +{
+  "description": "speakers: how many people speak, when you know (or give cluster_threshold)",
+  "maximum": 20,
+  "minimum": 1,
+  "type": "integer"
+}
    • changedInput schema / properties / threshold / description
      Previous value: -"scenes (required): how much the picture must change to count as a cut, 0–100"New value: +"scenes (required): how much the picture must change to count as a cut, 0–100 (FFmpeg scdet's score): the change from the frame before, in percent of full range, less the change just before it (so steady motion stays low). Hard cuts between ordinary shots score about 10 to 30, a cut between opposite pictures up to 100: start at FFmpeg's own default, 10, and raise it if you get too many"
    • changedInput schema / properties / timeline_id / description
      Previous value: -"Analyze an edit's mix as it plays (timeline seconds): speech, loudness, silence, sounds"New value: +"Analyze an edit as it plays (timeline seconds): its mix (speech, loudness, silence, sounds, speakers, voice) or its picture (faces, track, look, scenes, colour, picture)"
    • changedInput schema / properties / until_seconds / description
      Previous value: -"track, look, speech, loudness, silence, sounds: stop here. Default: the end"New value: +"Stop sampling or measuring here. Default: the end"
    • changedOutput schema / anyOf
      Previous value: -[
-  {
-    "properties": {
-      "beats": {
-        "properties": {
-          "beats": {
-            "items": {
-              "type": "number"
-            },
-            "type": "array"
-          },
-          "bpm": {
-            "type": "number"
-          },
-          "downbeats": {
-            "items": {
-              "type": "number"
-            },
-            "type": "array"
-          }
-        },
-        "required": [
-          "bpm",
-          "beats",
-          "downbeats"
-        ],
-        "type": "object"
-      },
-      "faces": {
-        "properties": {
-          "frames": {
-            "items": {
-              "properties": {
-                "faces": {
-                  "items": {
-                    "properties": {
-                      "h": {
-                        "type": "number"
-                      },
-                      "score": {
-                        "type": "number"
-                      },
-                      "w": {
-                        "type": "number"
-                      },
-                      "x": {
-                        "type": "number"
-                      },
-                      "y": {
-                        "type": "number"
-                      }
-                    },
-                    "required": [
-                      "x",
-                      "y",
-                      "w",
-                      "h",
-                      "score"
-                    ],
-                    "type": "object"
-                  },
-                  "type": "array"
-                },
-                "time": {
-                  "type": "number"
-                }
-              },
-              "required": [
-                "time",
-                "faces"
-              ],
-              "type": "object"
-            },
-            "type": "array"
-          },
-          "rate": {
-            "type": "number"
-          }
-        },
-        "required": [
-          "rate",
-          "frames"
-        ],
-        "type": "object"
-      },
-      "frames": {
-        "description": "look: frames asked about",
-        "type": "number"
-      },
-      "kind": {
-        "enum": [
-          "transcript",
-          "beats",
-          "faces",
-          "track",
-          "look",
-          "silence",
-          "scenes",
-          "speech",
-          "loudness",
-          "sounds"
-        ],
-        "type": "string"
-      },
-      "look": {
-        "additionalProperties": {},
-        "description": "look: this run (id, model, questions, frames with answers)",
-        "propertyNames": {
-          "type": "string"
-        },
-        "type": "object"
-      },
-      "loudness": {
-        "properties": {
-          "from": {
-            "description": "Seconds measured from",
-            "type": "number"
-          },
-          "integrated_lufs": {
-            "description": "EBU R128; null below its -70 LUFS gate",
-            "type": [
-              "number",
-              "null"
-            ]
-          },
-          "loudness_range_lu": {
-            "description": "EBU R128 loudness range (LRA)",
-            "type": [
-              "number",
-              "null"
-            ]
-          },
-          "noise_floor_dbfs": {
-            "description": "Level of the quietest moments (FFmpeg astats noise floor): room tone or hiss",
-            "type": [
-              "number",
-              "null"
-            ]
-          },
-          "rms_dbfs": {
-            "description": "RMS level over the whole stretch",
-            "type": [
-              "number",
-              "null"
-            ]
-          },
-          "true_peak_dbtp": {
-            "type": [
-              "number",
-              "null"
-            ]
-          },
-          "until": {
-            "description": "Seconds measured until; null: the end",
-            "type": [
-              "number",
-              "null"
-            ]
-          }
-        },
-        "required": [
-          "integrated_lufs",
-          "true_peak_dbtp",
-          "loudness_range_lu",
-          "rms_dbfs",
-          "noise_floor_dbfs",
-          "from",
-          "until"
-        ],
-        "type": "object"
-      },
-      "media_id": {
-        "description": "Media handle from media.import or a create.* tool",
-        "pattern": "^m_[0-9a-z]{26}$",
-        "type": "string"
-      },
-      "model": {
-        "description": "look: the Jev model",
-        "type": "string"
-      },
-      "note": {
-        "description": "In place of the analysis when it is too long to return: where to read it",
-        "type": "string"
-      },
-      "scenes": {
-        "description": "scenes: cut times, media seconds",
-        "items": {
-          "type": "number"
-        },
-        "type": "array"
-      },
-      "silences": {
-        "description": "silence: media seconds",
-        "items": {
-          "properties": {
-            "end": {
-              "type": "number"
-            },
-            "start": {
-              "type": "number"
-            }
-          },
-          "required": [
-            "start",
-            "end"
-          ],
-          "type": "object"
-        },
-        "type": "array"
-      },
-      "sounds": {
-        "properties": {
-          "min_probability": {
-            "type": "number"
-          },
-          "model": {
-            "type": "string"
-          },
-          "peaks": {
-            "additionalProperties": {
-              "type": "number"
-            },
-            "description": "Per sound reported: the highest probability it reaches anywhere",
-            "propertyNames": {
-              "type": "string"
-            },
-            "type": "object"
-          },
-          "scores": {
-            "description": "Every probability, for your own cut-off or arithmetic",
-            "properties": {
-              "start_seconds": {
-                "description": "Where the first value starts: media seconds, or timeline seconds for a timeline",
-                "type": "number"
-              },
-              "step_seconds": {
-                "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long",
-                "type": "number"
-              },
-              "values": {
-                "additionalProperties": {
-                  "items": {
-                    "type": "number"
-                  },
-                  "type": "array"
-                },
-                "description": "Per sound, the highest probability in each step, 0 to 1",
-                "propertyNames": {
-                  "type": "string"
-                },
-                "type": "object"
-              }
-            },
-            "required": [
-              "start_seconds",
-              "step_seconds",
-              "values"
-            ],
-            "type": "object"
-          },
-          "spans": {
-            "additionalProperties": {
-              "items": {
-                "properties": {
-                  "end": {
-                    "description": "Where it ends, on the same clock",
-                    "type": "number"
-                  },
-                  "peak_probability": {
-                    "type": "number"
-                  },
-                  "peak_seconds": {
-                    "description": "The middle of the most likely frame",
-                    "type": "number"
-                  },
-                  "start": {
-                    "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline",
-                    "type": "number"
-                  }
-                },
-                "required": [
-                  "start",
-                  "end",
-                  "peak_seconds",
-                  "peak_probability"
-                ],
-                "type": "object"
-              },
-              "type": "array"
-            },
-            "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is",
-            "propertyNames": {
-              "type": "string"
-            },
-            "type": "object"
-          }
-        },
-        "required": [
-          "model",
-          "min_probability",
-          "spans",
-          "peaks"
-        ],
-        "type": "object"
-      },
-      "speech": {
-        "properties": {
-          "frame_seconds": {
-            "type": "number"
-          },
-          "from": {
-            "type": "number"
-          },
-          "min_probability": {
-            "type": "number"
-          },
-          "model": {
-            "type": "string"
-          },
-          "probabilities": {
-            "description": "Every frame, when short enough to return; media: also media.get include speech",
-            "items": {
-              "type": "number"
-            },
-            "type": "array"
-          },
-          "spans": {
-            "description": "Runs of frames at or above min_probability, seconds",
-            "items": {
-              "properties": {
-                "end": {
-                  "type": "number"
-                },
-                "start": {
-                  "type": "number"
-                }
-              },
-              "required": [
-                "start",
-                "end"
-              ],
-              "type": "object"
-            },
-            "type": "array"
-          }
-        },
-        "required": [
-          "model",
-          "min_probability",
-          "spans",
-          "frame_seconds",
-          "from"
-        ],
-        "type": "object"
-      },
-      "summary": {
-        "additionalProperties": {},
-        "description": "The summary media.get gives under analyses; look: where each answer holds",
-        "propertyNames": {
-          "type": "string"
-        },
-        "type": "object"
-      },
-      "timeline_id": {
-        "description": "Timeline handle; every edit returns a new one",
-        "pattern": "^tl_[0-9a-f]{32}$",
-        "type": "string"
-      },
-      "track": {
-        "properties": {
-          "points": {
-            "items": {
-              "properties": {
-                "found": {
-                  "type": "boolean"
-                },
-                "h": {
-                  "type": "number"
-                },
-                "time": {
-                  "type": "number"
-                },
-                "w": {
-                  "type": "number"
-                },
-                "x": {
-                  "type": "number"
-                },
-                "y": {
-                  "type": "number"
-                }
-              },
-              "required": [
-                "time",
-                "x",
-                "y",
-                "w",
-                "h",
-                "found"
-              ],
-              "type": "object"
-            },
-            "type": "array"
-          },
-          "rate": {
-            "type": "number"
-          }
-        },
-        "required": [
-          "rate",
-          "points"
-        ],
-        "type": "object"
-      },
-      "transcript": {
-        "properties": {
-          "aligned_by": {
-            "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off",
-            "type": [
-              "string",
-              "null"
-            ]
-          },
-          "language": {
-            "type": "string"
-          },
-          "model": {
-            "type": "string"
-          },
-          "text": {
-            "type": "string"
-          },
-          "words": {
-            "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech",
-            "items": {
-              "properties": {
-                "confidence": {
-                  "description": "0–1: how sure whisper is of the word itself",
-                  "type": "number"
-                },
-                "end": {
-                  "type": "number"
-                },
-                "start": {
-                  "type": "number"
-                },
-                "text": {
-                  "type": "string"
-                },
-                "timing_confidence": {
-                  "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours",
-                  "type": [
-                    "number",
-                    "null"
-                  ]
-                }
-              },
-              "required": [
-                "text",
-                "start",
-                "end",
-                "confidence"
-              ],
-              "type": "object"
-            },
-            "type": "array"
-          }
-        },
-        "required": [
-          "model",
-          "language",
-          "text",
-          "words"
-        ],
-        "type": "object"
-      }
-    },
-    "required": [
-      "kind"
-    ],
-    "type": "object"
-  },
-  {
-    "properties": {
-      "hint": {
-        "type": "string"
-      },
-      "job_id": {
-        "description": "Job handle; the same value is the MCP task id",
-        "pattern": "^j_[0-9a-z]{26}$",
-        "type": "string"
-      },
-      "media_id": {
-        "description": "The media it is making; usable in edits once processing",
-        "pattern": "^m_[0-9a-z]{26}$",
-        "type": "string"
-      },
-      "status": {
-        "enum": [
-          "working",
-          "completed",
-          "failed",
-          "cancelled"
-        ],
-        "type": "string"
-      },
-      "status_uri": {
-        "type": "string"
-      }
-    },
-    "required": [
-      "job_id",
-      "status",
-      "status_uri",
-      "hint"
-    ],
-    "type": "object"
-  }
-]New value: +[
+  {
+    "properties": {
+      "beats": {
+        "properties": {
+          "beats": {
+            "items": {
+              "type": "number"
+            },
+            "type": "array"
+          },
+          "bpm": {
+            "type": "number"
+          },
+          "downbeats": {
+            "items": {
+              "type": "number"
+            },
+            "type": "array"
+          },
+          "strengths": {
+            "description": "Per beat: its onset strength, 0–1 of the strongest in the sound",
+            "items": {
+              "type": "number"
+            },
+            "type": "array"
+          }
+        },
+        "required": [
+          "bpm",
+          "beats",
+          "downbeats"
+        ],
+        "type": "object"
+      },
+      "colour": {
+        "properties": {
+          "frames": {
+            "items": {
+              "properties": {
+                "regions": {
+                  "additionalProperties": {
+                    "properties": {
+                      "b": {
+                        "properties": {
+                          "mean": {
+                            "type": "number"
+                          },
+                          "percentiles": {
+                            "additionalProperties": {
+                              "type": "number"
+                            },
+                            "description": "Per percentile asked (\"5\", \"50\", \"95\"): the value",
+                            "propertyNames": {
+                              "type": "string"
+                            },
+                            "type": "object"
+                          }
+                        },
+                        "required": [
+                          "mean",
+                          "percentiles"
+                        ],
+                        "type": "object"
+                      },
+                      "clipped_high": {
+                        "description": "Share of pixels with a channel at 255",
+                        "type": "number"
+                      },
+                      "clipped_low": {
+                        "description": "Share of pixels black in every channel",
+                        "type": "number"
+                      },
+                      "g": {
+                        "properties": {
+                          "mean": {
+                            "type": "number"
+                          },
+                          "percentiles": {
+                            "additionalProperties": {
+                              "type": "number"
+                            },
+                            "description": "Per percentile asked (\"5\", \"50\", \"95\"): the value",
+                            "propertyNames": {
+                              "type": "string"
+                            },
+                            "type": "object"
+                          }
+                        },
+                        "required": [
+                          "mean",
+                          "percentiles"
+                        ],
+                        "type": "object"
+                      },
+                      "luma": {
+                        "description": "Rec. 709 luma of the encoded values: 0.2126 R + 0.7152 G + 0.0722 B",
+                        "properties": {
+                          "mean": {
+                            "type": "number"
+                          },
+                          "percentiles": {
+                            "additionalProperties": {
+                              "type": "number"
+                            },
+                            "description": "Per percentile asked (\"5\", \"50\", \"95\"): the value",
+                            "propertyNames": {
+                              "type": "string"
+                            },
+                            "type": "object"
+                          }
+                        },
+                        "required": [
+                          "mean",
+                          "percentiles"
+                        ],
+                        "type": "object"
+                      },
+                      "pixels": {
+                        "description": "The region measured, in pixels of the picture",
+                        "properties": {
+                          "h": {
+                            "type": "number"
+                          },
+                          "w": {
+                            "type": "number"
+                          },
+                          "x": {
+                            "type": "number"
+                          },
+                          "y": {
+                            "type": "number"
+                          }
+                        },
+                        "required": [
+                          "x",
+                          "y",
+                          "w",
+                          "h"
+                        ],
+                        "type": "object"
+                      },
+                      "r": {
+                        "properties": {
+                          "mean": {
+                            "type": "number"
+                          },
+                          "percentiles": {
+                            "additionalProperties": {
+                              "type": "number"
+                            },
+                            "description": "Per percentile asked (\"5\", \"50\", \"95\"): the value",
+                            "propertyNames": {
+                              "type": "string"
+                            },
+                            "type": "object"
+                          }
+                        },
+                        "required": [
+                          "mean",
+                          "percentiles"
+                        ],
+                        "type": "object"
+                      },
+                      "white_point": {
+                        "description": "Mean colour of the brightest 1% of its pixels, an estimate of the white point",
+                        "properties": {
+                          "b": {
+                            "type": "number"
+                          },
+                          "g": {
+                            "type": "number"
+                          },
+                          "r": {
+                            "type": "number"
+                          }
+                        },
+                        "required": [
+                          "r",
+                          "g",
+                          "b"
+                        ],
+                        "type": "object"
+                      }
+                    },
+                    "required": [
+                      "pixels",
+                      "r",
+                      "g",
+                      "b",
+                      "luma",
+                      "white_point",
+                      "clipped_high",
+                      "clipped_low"
+                    ],
+                    "type": "object"
+                  },
+                  "description": "frame: the whole picture; and each region you named",
+                  "propertyNames": {
+                    "type": "string"
+                  },
+                  "type": "object"
+                },
+                "time": {
+                  "type": "number"
+                }
+              },
+              "required": [
+                "time",
+                "regions"
+              ],
+              "type": "object"
+            },
+            "type": "array"
+          },
+          "percentiles": {
+            "items": {
+              "type": "number"
+            },
+            "type": "array"
+          }
+        },
+        "required": [
+          "percentiles",
+          "frames"
+        ],
+        "type": "object"
+      },
+      "faces": {
+        "properties": {
+          "frames": {
+            "items": {
+              "properties": {
+                "faces": {
+                  "items": {
+                    "description": "Top-left corner x, y and size w, h, fractions of the picture (centre: x + w/2, y + h/2); score 0–1",
+                    "properties": {
+                      "h": {
+                        "type": "number"
+                      },
+                      "score": {
+                        "type": "number"
+                      },
+                      "w": {
+                        "type": "number"
+                      },
+                      "x": {
+                        "type": "number"
+                      },
+                      "y": {
+                        "type": "number"
+                      }
+                    },
+                    "required": [
+                      "x",
+                      "y",
+                      "w",
+                      "h",
+                      "score"
+                    ],
+                    "type": "object"
+                  },
+                  "type": "array"
+                },
+                "time": {
+                  "type": "number"
+                }
+              },
+              "required": [
+                "time",
+                "faces"
+              ],
+              "type": "object"
+            },
+            "type": "array"
+          },
+          "rate": {
+            "type": "number"
+          }
+        },
+        "required": [
+          "rate",
+          "frames"
+        ],
+        "type": "object"
+      },
+      "frames": {
+        "description": "look: frames asked about",
+        "type": "number"
+      },
+      "kind": {
+        "enum": [
+          "transcript",
+          "beats",
+          "faces",
+          "track",
+          "look",
+          "silence",
+          "scenes",
+          "speech",
+          "loudness",
+          "sounds",
+          "speakers",
+          "voice",
+          "colour",
+          "picture"
+        ],
+        "type": "string"
+      },
+      "look": {
+        "additionalProperties": {},
+        "description": "look: this run (id, model, questions, frames with answers)",
+        "propertyNames": {
+          "type": "string"
+        },
+        "type": "object"
+      },
+      "loudness": {
+        "properties": {
+          "channels": {
+            "description": "Each channel on its own, in stream order (1 is left in stereo)",
+            "items": {
+              "properties": {
+                "channel": {
+                  "exclusiveMinimum": 0,
+                  "type": "integer"
+                },
+                "noise_floor_dbfs": {
+                  "type": [
+                    "number",
+                    "null"
+                  ]
+                },
+                "peak_dbfs": {
+                  "type": [
+                    "number",
+                    "null"
+                  ]
+                },
+                "rms_dbfs": {
+                  "type": [
+                    "number",
+                    "null"
+                  ]
+                }
+              },
+              "required": [
+                "channel",
+                "rms_dbfs",
+                "peak_dbfs",
+                "noise_floor_dbfs"
+              ],
+              "type": "object"
+            },
+            "type": "array"
+          },
+          "from": {
+            "description": "Seconds measured from",
+            "type": "number"
+          },
+          "integrated_lufs": {
+            "description": "EBU R128; null below its -70 LUFS gate",
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "loudness_range_lu": {
+            "description": "EBU R128 loudness range (LRA)",
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "noise_floor_dbfs": {
+            "description": "Level of the quietest moments (FFmpeg astats noise floor): room tone or hiss",
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "rms_dbfs": {
+            "description": "RMS level over the whole stretch",
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "true_peak_dbtp": {
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "until": {
+            "description": "Seconds measured until; null: the end",
+            "type": [
+              "number",
+              "null"
+            ]
+          }
+        },
+        "required": [
+          "integrated_lufs",
+          "true_peak_dbtp",
+          "loudness_range_lu",
+          "rms_dbfs",
+          "noise_floor_dbfs",
+          "channels",
+          "from",
+          "until"
+        ],
+        "type": "object"
+      },
+      "media_id": {
+        "description": "Media handle from media.import or a create.* tool",
+        "pattern": "^m_[0-9a-z]{26}$",
+        "type": "string"
+      },
+      "model": {
+        "description": "look: the Jev model",
+        "type": "string"
+      },
+      "note": {
+        "description": "In place of the analysis when it is too long to return: where to read it",
+        "type": "string"
+      },
+      "picture": {
+        "properties": {
+          "columns": {
+            "additionalProperties": {
+              "items": {
+                "type": "number"
+              },
+              "type": "array"
+            },
+            "description": "One number per frame in each, in order: time, then FFmpeg signalstats in the picture's own bit depth (0–255 for 8-bit): ymin, ylow (10th percentile), yavg, yhigh (90th), ymax, the same for u and v, satmin…satmax, huemed, hueavg, ydif, udif, vdif (mean change from the frame before), and the bits each plane uses (ybitdepth…)",
+            "propertyNames": {
+              "type": "string"
+            },
+            "type": "object"
+          },
+          "from": {
+            "type": "number"
+          },
+          "method": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "method",
+          "from",
+          "columns"
+        ],
+        "type": "object"
+      },
+      "picture_size": {
+        "description": "faces, track, colour: the picture measured, in pixels (a box times it is pixels)",
+        "properties": {
+          "height": {
+            "type": "number"
+          },
+          "width": {
+            "type": "number"
+          }
+        },
+        "required": [
+          "width",
+          "height"
+        ],
+        "type": "object"
+      },
+      "scene_scores": {
+        "description": "scenes: how much the picture changed at each cut, on the threshold scale",
+        "items": {
+          "type": "number"
+        },
+        "type": "array"
+      },
+      "scenes": {
+        "description": "scenes: cut times, media seconds",
+        "items": {
+          "type": "number"
+        },
+        "type": "array"
+      },
+      "silences": {
+        "description": "silence: media seconds; a stretch after the sound stops (see sound_ends) counts as silent",
+        "items": {
+          "properties": {
+            "end": {
+              "type": "number"
+            },
+            "start": {
+              "type": "number"
+            }
+          },
+          "required": [
+            "start",
+            "end"
+          ],
+          "type": "object"
+        },
+        "type": "array"
+      },
+      "sound_ends": {
+        "description": "silence: where the sound stream stops, when it stops before the media ends: past it there is no sound at all",
+        "type": "number"
+      },
+      "sounds": {
+        "properties": {
+          "min_probability": {
+            "type": "number"
+          },
+          "model": {
+            "type": "string"
+          },
+          "peaks": {
+            "additionalProperties": {
+              "type": "number"
+            },
+            "description": "Per sound reported: the highest probability it reaches anywhere",
+            "propertyNames": {
+              "type": "string"
+            },
+            "type": "object"
+          },
+          "scores": {
+            "description": "Every probability, for your own cut-off or arithmetic",
+            "properties": {
+              "start_seconds": {
+                "description": "Where the first value starts: media seconds, or timeline seconds for a timeline",
+                "type": "number"
+              },
+              "step_seconds": {
+                "description": "Seconds each value covers: 0.04 (every frame) unless asked or too long",
+                "type": "number"
+              },
+              "values": {
+                "additionalProperties": {
+                  "items": {
+                    "type": "number"
+                  },
+                  "type": "array"
+                },
+                "description": "Per sound, the highest probability in each step, 0 to 1",
+                "propertyNames": {
+                  "type": "string"
+                },
+                "type": "object"
+              }
+            },
+            "required": [
+              "start_seconds",
+              "step_seconds",
+              "values"
+            ],
+            "type": "object"
+          },
+          "spans": {
+            "additionalProperties": {
+              "items": {
+                "properties": {
+                  "end": {
+                    "description": "Where it ends, on the same clock",
+                    "type": "number"
+                  },
+                  "peak_probability": {
+                    "type": "number"
+                  },
+                  "peak_seconds": {
+                    "description": "The middle of the most likely frame",
+                    "type": "number"
+                  },
+                  "start": {
+                    "description": "Where the run of matching 40 ms frames starts: media seconds, or timeline seconds for a timeline",
+                    "type": "number"
+                  }
+                },
+                "required": [
+                  "start",
+                  "end",
+                  "peak_seconds",
+                  "peak_probability"
+                ],
+                "type": "object"
+              },
+              "type": "array"
+            },
+            "description": "Per sound: the runs of 40 ms frames at or above min_probability, in time order; empty when it never is",
+            "propertyNames": {
+              "type": "string"
+            },
+            "type": "object"
+          }
+        },
+        "required": [
+          "model",
+          "min_probability",
+          "spans",
+          "peaks"
+        ],
+        "type": "object"
+      },
+      "speakers": {
+        "properties": {
+          "cluster_threshold": {
+            "description": "The cut-off you gave, or null",
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "from": {
+            "description": "The stretch diarised, when not the whole media",
+            "type": "number"
+          },
+          "model": {
+            "type": "string"
+          },
+          "speakers": {
+            "description": "How many speakers were found",
+            "type": "number"
+          },
+          "speakers_asked": {
+            "description": "The count you gave, or null",
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "turns": {
+            "description": "Each stretch of one voice, by start; two voices at once overlap",
+            "items": {
+              "properties": {
+                "confidence": {
+                  "description": "0–1: how close this turn's voice sits to its speaker's other turns",
+                  "type": [
+                    "number",
+                    "null"
+                  ]
+                },
+                "end": {
+                  "type": "number"
+                },
+                "speaker": {
+                  "description": "0 is the first voice heard, 1 the next new one, …",
+                  "type": "number"
+                },
+                "start": {
+                  "type": "number"
+                }
+              },
+              "required": [
+                "start",
+                "end",
+                "speaker",
+                "confidence"
+              ],
+              "type": "object"
+            },
+            "type": "array"
+          },
+          "until": {
+            "type": [
+              "number",
+              "null"
+            ]
+          }
+        },
+        "required": [
+          "model",
+          "speakers_asked",
+          "cluster_threshold",
+          "speakers",
+          "turns"
+        ],
+        "type": "object"
+      },
+      "speech": {
+        "properties": {
+          "frame_seconds": {
+            "type": "number"
+          },
+          "from": {
+            "type": "number"
+          },
+          "min_probability": {
+            "type": "number"
+          },
+          "model": {
+            "type": "string"
+          },
+          "probabilities": {
+            "description": "Every frame, when short enough to return; media: also media.get include speech",
+            "items": {
+              "type": "number"
+            },
+            "type": "array"
+          },
+          "spans": {
+            "description": "Runs of frames at or above min_probability, seconds",
+            "items": {
+              "properties": {
+                "end": {
+                  "type": "number"
+                },
+                "start": {
+                  "type": "number"
+                }
+              },
+              "required": [
+                "start",
+                "end"
+              ],
+              "type": "object"
+            },
+            "type": "array"
+          }
+        },
+        "required": [
+          "model",
+          "min_probability",
+          "spans",
+          "frame_seconds",
+          "from"
+        ],
+        "type": "object"
+      },
+      "summary": {
+        "additionalProperties": {},
+        "description": "The summary media.get gives under analyses; look: where each answer holds",
+        "propertyNames": {
+          "type": "string"
+        },
+        "type": "object"
+      },
+      "timeline_id": {
+        "description": "Timeline handle; every edit returns a new one",
+        "pattern": "^tl_[0-9a-f]{32}$",
+        "type": "string"
+      },
+      "track": {
+        "properties": {
+          "points": {
+            "items": {
+              "properties": {
+                "found": {
+                  "type": "boolean"
+                },
+                "h": {
+                  "type": "number"
+                },
+                "time": {
+                  "type": "number"
+                },
+                "w": {
+                  "type": "number"
+                },
+                "x": {
+                  "type": "number"
+                },
+                "y": {
+                  "type": "number"
+                }
+              },
+              "required": [
+                "time",
+                "x",
+                "y",
+                "w",
+                "h",
+                "found"
+              ],
+              "type": "object"
+            },
+            "type": "array"
+          },
+          "rate": {
+            "type": "number"
+          }
+        },
+        "required": [
+          "rate",
+          "points"
+        ],
+        "type": "object"
+      },
+      "transcript": {
+        "properties": {
+          "aligned_by": {
+            "description": "The model that moved word edges to where the letters sound (English); null: the times are whisper's own, up to about a second off",
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "from": {
+            "description": "The stretch transcribed, if not all",
+            "type": "number"
+          },
+          "language": {
+            "type": "string"
+          },
+          "model": {
+            "type": "string"
+          },
+          "text": {
+            "type": "string"
+          },
+          "until": {
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "words": {
+            "description": "Media seconds. Aligned words start and end within about 0.05 s of where their letters sound; an end on a soft consonant (s, t, k, f) or a breath can ring on up to about 0.1 s more. Short ums are often missing from the words; kind speech finds them as speech. A word \"-\" (or one starting with it) is whisper marking a new speaker, as subtitles do: a hint only; kind speakers measures who speaks when",
+            "items": {
+              "properties": {
+                "confidence": {
+                  "description": "0–1: how sure whisper is of the word itself",
+                  "type": "number"
+                },
+                "end": {
+                  "type": "number"
+                },
+                "speaker": {
+                  "description": "Who says it, from the latest speakers run",
+                  "type": [
+                    "number",
+                    "null"
+                  ]
+                },
+                "start": {
+                  "type": "number"
+                },
+                "text": {
+                  "type": "string"
+                },
+                "timing_confidence": {
+                  "description": "0–1: how clearly the letters were heard between start and end; null: not aligned (digits, symbols, other languages), times fitted between neighbours",
+                  "type": [
+                    "number",
+                    "null"
+                  ]
+                }
+              },
+              "required": [
+                "text",
+                "start",
+                "end",
+                "confidence"
+              ],
+              "type": "object"
+            },
+            "type": "array"
+          }
+        },
+        "required": [
+          "model",
+          "language",
+          "text",
+          "words"
+        ],
+        "type": "object"
+      },
+      "voice": {
+        "properties": {
+          "frame_seconds": {
+            "type": "number"
+          },
+          "frames": {
+            "items": {
+              "properties": {
+                "level_db": {
+                  "description": "RMS, dBFS; null: digital silence",
+                  "type": [
+                    "number",
+                    "null"
+                  ]
+                },
+                "pitch_hz": {
+                  "description": "Median of its voiced 10 ms steps; null: none",
+                  "type": [
+                    "number",
+                    "null"
+                  ]
+                },
+                "time": {
+                  "type": "number"
+                },
+                "voiced": {
+                  "description": "Share of it with a clear pitch, 0–1",
+                  "type": "number"
+                }
+              },
+              "required": [
+                "time",
+                "pitch_hz",
+                "voiced",
+                "level_db"
+              ],
+              "type": "object"
+            },
+            "type": "array"
+          },
+          "from": {
+            "type": "number"
+          },
+          "method": {
+            "type": "string"
+          },
+          "summary": {
+            "properties": {
+              "level_db": {
+                "properties": {
+                  "p10": {
+                    "type": [
+                      "number",
+                      "null"
+                    ]
+                  },
+                  "p90": {
+                    "type": [
+                      "number",
+                      "null"
+                    ]
+                  },
+                  "rms": {
+                    "type": [
+                      "number",
+                      "null"
+                    ]
+                  }
+                },
+                "required": [
+                  "rms",
+                  "p10",
+                  "p90"
+                ],
+                "type": "object"
+              },
+              "pitch_hz": {
+                "anyOf": [
+                  {
+                    "properties": {
+                      "median": {
+                        "type": "number"
+                      },
+                      "p10": {
+                        "type": "number"
+                      },
+                      "p90": {
+                        "type": "number"
+                      }
+                    },
+                    "required": [
+                      "median",
+                      "p10",
+                      "p90"
+                    ],
+                    "type": "object"
+                  },
+                  {
+                    "type": "null"
+                  }
+                ],
+                "description": "Over the voiced time"
+              },
+              "pitch_range_semitones": {
+                "description": "p10 to p90: how much the voice moves",
+                "type": [
+                  "number",
+                  "null"
+                ]
+              },
+              "seconds": {
+                "type": "number"
+              },
+              "voiced_seconds": {
+                "description": "Time with a clear pitch: voice, not noise or silence",
+                "type": "number"
+              },
+              "words": {
+                "description": "Transcript words in the stretch; null: no transcript",
+                "type": [
+                  "number",
+                  "null"
+                ]
+              },
+              "words_per_second": {
+                "type": [
+                  "number",
+                  "null"
+                ]
+              }
+            },
+            "required": [
+              "seconds",
+              "voiced_seconds",
+              "pitch_hz",
+              "pitch_range_semitones",
+              "level_db",
+              "words",
+              "words_per_second"
+            ],
+            "type": "object"
+          }
+        },
+        "required": [
+          "method",
+          "frame_seconds",
+          "from",
+          "summary",
+          "frames"
+        ],
+        "type": "object"
+      }
+    },
+    "required": [
+      "kind"
+    ],
+    "type": "object"
+  },
+  {
+    "properties": {
+      "hint": {
+        "type": "string"
+      },
+      "job_id": {
+        "description": "Job handle; the same value is the MCP task id",
+        "pattern": "^j_[0-9a-z]{26}$",
+        "type": "string"
+      },
+      "media_id": {
+        "description": "The media it is making; usable in edits once processing",
+        "pattern": "^m_[0-9a-z]{26}$",
+        "type": "string"
+      },
+      "status": {
+        "enum": [
+          "working",
+          "completed",
+          "failed",
+          "cancelled"
+        ],
+        "type": "string"
+      },
+      "status_uri": {
+        "type": "string"
+      }
+    },
+    "required": [
+      "job_id",
+      "status",
+      "status_uri",
+      "hint"
+    ],
+    "type": "object"
+  }
+]
  2. First observed

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the supplied annotations the description discloses persistence rules (results kept with the media via media.get include, except track, silence, scenes and loudness which return inline), truncation behavior (long transcripts return as summary plus a note), failure modes ("fails with no_audio, naming it"), execution model (returns a task or job_id for job.get), cost (speakers ~10-20 s per minute of media), and hard limits (600 frames, 32 questions, last 10 look runs). These are exactly the behavioral traits the annotations cannot express.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The initial purpose sentence is well front-loaded, but the body is a single semicolon-chained run-on enumerating 14 kinds, followed by a very long "HOW LOOK WORKS" tutorial with external doc links. Given the 14-kind surface area most content earns its place, but the presentation is hard to scan and it repeats the enum descriptions verbatim from the schema instead of trusting them.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, 96% schema coverage and rich annotations, the description fills the remaining gaps an agent needs: which kinds persist vs return, which parameters are required per kind, timeline vs media semantics, and the sampling/limit constraints. Nothing material is missing for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 96%, so the schema already carries most parameter documentation, and much of the description's per-kind parameter prose duplicates the enum and property descriptions. It still adds real interpretive value: calibrating threshold against FFmpeg scdet scores ("hard cuts between ordinary shots score about 10 to 30… start at 10"), guidance on cluster_threshold ranges, and the fractional-vs-pixel box units convention.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a concrete verb-and-resource statement ("Find what is in imported media, or in the sound of an edit, as data for edits") and then enumerates all 14 `kind` values with what each returns. It actively distinguishes itself from media.seek, media.listen, media.correlate, judge.ask and media.get, so an agent can route without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit routing is given per sibling: "to find where something happens from plain words, media.seek"; "To find a sound you describe in your own words, media.listen"; "To line up two recordings, media.correlate"; "For one judgment about text, a still or a single frame, judge.ask answers at once." It also states media_id vs timeline_id selection and which kinds work on which, plus mode-specific exclusions (from_seconds not valid for beats).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All cuttingfloor toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources