Skip to main content
Glama

Generate music for a video

video_to_music

Generate an original, licensed music score for a video by analyzing its pacing, motion, and emotion, aligning beats to cut points, and matching its exact duration. Cleared for commercial use.

Instructions

Generate an original score for a video: Sonilo analyzes the video's pacing, motion, and emotion, aligns transitions and beat drops to its cut points, and matches the video's duration exactly. Provide either a local video file path or a publicly accessible video URL. Generated tracks are fully licensed (music licensed via Shutterstock) and cleared for commercial use on social, brand content, and advertising.

⚠️ COST WARNING: This tool makes an API call to Sonilo which may incur charges. Only use when explicitly requested by the user.

Args: video_path (str, optional): Absolute local path, or relative to SONILO_MCP_BASE_PATH. Supports .mp4/.mov/.avi/.wmv/.webm/.mkv. Subject to the account's max upload size (typically 300 MB). Maximum video duration is 360 seconds (6 minutes). video_url (str, optional): HTTPS URL to a video file. prompt (str, optional): Style hint for the generated music. preserve_speech (bool, optional): Keep the source speech from the video audible in the result. When set, in addition to the generated music you also get a 'vocals' speech stem and a ready-to-use 'mux' (speech+music, already mixed). Defaults to False. output_format (str, optional): 'm4a' (default), 'wav', or 'mp3' (320 kbps). ducking (bool, optional): Duck the generated music under the source voice at finalize time. Default-OFF server-side: leave unset to skip it, pass True to run it, which adds a ducked track alongside the unchanged clean audio. Free, best-effort. variants_num (int, optional): 1-10, default 1. Generate this many distinct music variants in one request — each is its own creative direction with its own title, not just a re-roll. Cost scales linearly with N. Values above 1 are never covered by the free trial — even a trial account is billed for variants beyond the first. prompt_influence (float, optional): 0-1, API default 0.5. Sets how strongly the generated music follows the prompt; omit it unless the user asks for stricter or looser prompt adherence. Free. stems (bool, optional): Default false. Pass stems=true to also get the generated music split into four separate instrument tracks — drums, bass, vocals and other — at no extra charge; it adds a few minutes to the wait (this tool then waits up to 75 minutes), so set it only when the user asks for stems or separated tracks. It splits the generated music, never the video's own audio. If a stems result comes back with stems_error instead of (or alongside a partial) stems — surfaced here as a note in the returned text — only the separation failed or was skipped: the music itself generated fine, the saved audio files are valid, and the user was not charged for the separation — report it as a missing extra, not as a failed generation. Any of preserve_speech/a non-m4a output_format/ducking/variants_num>1/stems makes this tool internally use the backend's async generation mode (submit + poll) instead of streaming — the call takes longer but the tool still waits for completion. Subject to the same 360-second video duration cap as the plain case. output_directory (str, optional): Where to save the resulting audio file(s). Defaults to SONILO_MCP_BASE_PATH.

Exactly one of video_path and video_url must be provided.

Returns: Plain case (no async-triggering param set): one TextContent per generated audio stream, unchanged from before. preserve_speech=True: one TextContent per generated audio stream, plus one for the speech ('vocals') file, plus one per mux stream (speech+music mixed — this is the ready-to-use combined result). ducking=True: also one TextContent per ducked stream (music lowered under the source voice), when the backend rendered one. variants_num > 1: one audio TextContent per variant (suffixed -0, -1, ...), each labeled with its title when the backend provides one. stems=True: also one TextContent per saved stem file (drums/bass/vocals/other, per stream), or a note carrying stems_error when the free separation failed or was skipped. Each TextContent's label says which kind of file it is.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "video_to_music"
  }
}

Input Schema

NameRequiredDescriptionDefault
stemsNo
promptNo
duckingNo
video_urlNo
video_pathNo
variants_numNo
output_formatNo
preserve_speechNo
output_directoryNo
prompt_influenceNo

Output Schema

NameRequiredDescriptionDefault
resultYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed6 schema fields changedv0.28.0
    • addedInput schema / properties / ducking
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "boolean"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Ducking"
      +}
    • addedInput schema / properties / output_format
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "string"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Output Format"
      +}
    • addedInput schema / properties / preserve_speech
      Added value: +{
      +  "default": false,
      +  "title": "Preserve Speech",
      +  "type": "boolean"
      +}
    • addedInput schema / properties / prompt_influence
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "number"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Prompt Influence"
      +}
    • addedInput schema / properties / stems
      Added value: +{
      +  "default": false,
      +  "title": "Stems",
      +  "type": "boolean"
      +}
    • addedInput schema / properties / variants_num
      Added value: +{
      +  "default": 1,
      +  "title": "Variants Num",
      +  "type": "integer"
      +}
  2. First observedv0.4.0

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes far beyond the annotations: discloses cost/billing behavior (variants above 1 never covered by free trial, cost scales linearly), internal async-mode switching conditions, a 360s duration cap and ~300 MB upload limit, and how to interpret a stems_error partial result. Annotations only give the generic openWorld/non-idempotent profile; the description supplies the operational and financial risk details.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads purpose then the cost warning, so the highest-stakes information is early. It is long and repeats the 360-second cap across two paragraphs, and the returns block is somewhat verbose, but almost every sentence carries operational value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-param, expensive, open-world generation tool with an output schema, this is thorough: it covers licensing, cost, duration/size limits, async latency (up to 75 min), error semantics, and result labeling. An agent has everything needed to decide and invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must carry all meaning — and it does, documenting every one of the 10 params: defaults, valid formats, ranges (variants_num 1-10, prompt_influence 0-1), file extensions, path resolution against SONILO_MCP_BASE_PATH, and non-obvious semantics (ducking is default-OFF server-side, stems splits generated music not source audio).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Generate an original score for a video') and immediately describes the mechanism (pacing, motion, emotion analysis, beat alignment, exact duration match). This clearly differentiates it from text_to_music and video_to_sfx siblings because the input medium (video) and output (score) are both explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides an explicit when-not: 'Only use when explicitly requested by the user' backed by a cost warning, which is exactly the gating condition an agent needs. It also clarifies the input requirement ('Exactly one of video_path and video_url'). It does not name sibling alternatives (e.g., use text_to_music when there is no video), so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.