Clueso

Analyze audio

analyze_audio

Read-only

Analyze an audio file. Modes:

transcript: Speech-to-text with word-level timestamps — returns {language, full_text, words} where each word is {word, start_ms, end_ms, punctuated_word}. ⚠ MILLISECONDS here; get_clip's voiceover_words gives the same words in SECONDS as {word, start, end}. Use for caption timing, script editing, or word-level visual sync. Works on narration and on sung lyrics: the provider is picked from the content, and if it returns nothing the other one is tried automatically. So an empty words means the audio genuinely has no intelligible speech, not that the wrong engine ran.
silences: returns {silences, total_silence_ms}, each span {start_ms, end_ms, peak_db}. Use for auto-trim, smart-split, or pause-aware editing.
beats: returns {tempo_bpm, beats_ms} — beats_ms is an array of MILLISECOND positions. Empty for non-music. Use to align animations/zooms to a beat.
features: returns {duration_ms, peak_db, rms_db, lufs, has_music, has_speech, speech_share} — loudness plus two INDEPENDENT hints — has_music (beats hold a steady period) and has_speech (at least 40% of the audible seconds sound like talking; speech_share is that share, 0-1). Instrumentals read false; so do sung vocals and narration under a loud bed, so when the words matter, use transcript. Cheap dispatcher — call once to pick a mode, or for duration_ms to pass as add_audio's source_duration.
energy: loudness over time — {samples_per_second, start_ms, db}, db one RMS dBFS value per 1/samples_per_second s from start_ms (default 4 a second; energy_options.samples_per_second 1-20). Read builds, dips and hits from it.
sections: how a track is built — {loud_db, sections: [{label, start_ms, end_ms, db, onsets_per_s}], drops: [{t_ms, rise_db, quiet_start_ms?, quiet_end_ms?}]}, label intro | build | drop | main | breakdown | outro; a drop's t_ms is the hit (within ~10 ms), quiet_* the dip just before it. Set animation cues on beats (beats mode) or drops.

Source: provide exactly one of mcp_upload_id (from upload tool) or file_url (already-public URL, e.g. a find(type='music') result or Clueso CDN file). Optional time range: range_start_ms / range_end_ms crops the analysis window.

Input Schema

TableJSON Schema

Name	Required	Description
`mode`	Yes	Which analysis to run.
`file_url`	No	Public http(s) URL to the audio. Use when the file is already accessible (e.g. find(type='music') result, Clueso CDN). Validated against private/loopback ranges.
`range_end_ms`	No	End of analysis window in milliseconds. Default: whole file.
`mcp_upload_id`	No	MCP upload ID from the upload_file tool. Use this OR file_url, not both.
`energy_options`	No	Options for mode='energy'.
`range_start_ms`	No	Start of analysis window in milliseconds. Default 0 (beginning).
`silences_options`	No	Options for mode='silences'.
`transcript_options`	No	Options for mode='transcript'.

Schema Changelog

Changes observed during successful MCP inspections.

Changed2 schema fields changedOct 12, 2026

addedInput schema / properties / energy_options

Added value: +{
+  "description": "Options for mode='energy'.",
+  "properties": {
+    "samples_per_second": {
+      "description": "Loudness values per second. Default 4.",
+      "maximum": 20,
+      "minimum": 1,
+      "type": "number"
+    }
+  },
+  "type": "object"
+}

changedInput schema / properties / mode / enum

Previous value: -[
-  "transcript",
-  "silences",
-  "beats",
-  "features"
-]New value: +[
+  "transcript",
+  "silences",
+  "beats",
+  "features",
+  "energy",
+  "sections"
+]

Changed4 schema fields changedSep 22, 2026

removedInput schema / properties / context

Removed value: -{
-  "description": "Explain in 15-25 words, in third person, why this tool is called and how it supports the user's goal. For analytics only. You MUST describe only the abstract purpose of the tool call. NEVER include, repeat, paraphrase, or infer personal, sensitive, or identifying information from the user request or tool results, including names, emails, phone numbers, IPs, IDs, or credentials. You MUST generalize specific entities into roles such as \"a user\", \"the customer\", or \"an account\". Example: \"Retrieving a customer's recent orders to investigate a billing issue and help support determine the appropriate resolution.\"",
-  "type": "string"
-}

removedInput schema / properties / conversation_id

Removed value: -{
-  "description": "Echo the conversation_id from the server's previous response. The server provides it on the first call — never invent one, and do not issue parallel tool calls until you have it.",
-  "type": "string"
-}

removedInput schema / properties / llm_model

Removed value: -{
-  "description": "The exact model identifier you (the assistant) are running as, taken from your system prompt or environment (e.g. \"claude-opus-4-8\", \"gpt-5.2\"). Used for analytics only. If you do not know your model identifier with certainty, pass \"unknown\" — never guess.",
-  "type": "string"
-}

changedInput schema / required

Previous value: -[
-  "mode",
-  "context",
-  "llm_model"
-]New value: +[
+  "mode"
+]

Changed4 schema fields changedSep 22, 2026

addedInput schema / properties / context

Added value: +{
+  "description": "Explain in 15-25 words, in third person, why this tool is called and how it supports the user's goal. For analytics only. You MUST describe only the abstract purpose of the tool call. NEVER include, repeat, paraphrase, or infer personal, sensitive, or identifying information from the user request or tool results, including names, emails, phone numbers, IPs, IDs, or credentials. You MUST generalize specific entities into roles such as \"a user\", \"the customer\", or \"an account\". Example: \"Retrieving a customer's recent orders to investigate a billing issue and help support determine the appropriate resolution.\"",
+  "type": "string"
+}

addedInput schema / properties / conversation_id

Added value: +{
+  "description": "Echo the conversation_id from the server's previous response. The server provides it on the first call — never invent one, and do not issue parallel tool calls until you have it.",
+  "type": "string"
+}

addedInput schema / properties / llm_model

Added value: +{
+  "description": "The exact model identifier you (the assistant) are running as, taken from your system prompt or environment (e.g. \"claude-opus-4-8\", \"gpt-5.2\"). Used for analytics only. If you do not know your model identifier with certainty, pass \"unknown\" — never guess.",
+  "type": "string"
+}

changedInput schema / required

Previous value: -[
-  "mode"
-]New value: +[
+  "mode",
+  "context",
+  "llm_model"
+]

Changed7 schema fields changedSep 17, 2026
- changedInput schema / $schema
```
Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
```
- removedInput schema / additionalProperties
```
Removed value: -false
```
- addedInput schema / properties / range_end_ms / maximum
```
Added value: +9007199254740991
```
- addedInput schema / properties / range_start_ms / maximum
```
Added value: +9007199254740991
```
- removedInput schema / properties / silences_options / additionalProperties
```
Removed value: -false
```
- addedInput schema / properties / silences_options / properties / min_pause_ms / maximum
```
Added value: +9007199254740991
```
- removedInput schema / properties / transcript_options / additionalProperties
```
Removed value: -false
```
First observedAug 11, 2026

TDQS

A4.9/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Despite readOnlyHint/destructiveHint already covering the safety profile, the description adds units (ms vs get_clip's seconds), provider fallback behavior for transcript, the guarantee that empty words means no real speech, beat-empty-means-no-music, drop timing accuracy (~10 ms), and that features is cheap. This is unusually rich behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Long but tightly organized: a mode-indexed bullet list, front-loaded return shape then use case for each entry, and a trailing source/time-range note. Every sentence carries distinct, non-redundant information given the six-mode surface.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description fully compensates by documenting each mode's return object and field semantics (including ms vs seconds pitfalls). Source exclusivity and cropping are covered, so nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds meaning beyond the schema: the exactly-one-of mcp_upload_id/file_url constraint, and that range_start_ms/range_end_ms crop the analysis window. It also flags the energy_options samples_per_second range and the beat/drop semantics of returned values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Analyze an audio file') and then enumerates six distinct modes with their return shapes and use cases, so an agent knows exactly what each invocation produces. It actively differentiates from siblings by naming get_clip's voiceover_words and add_audio's source_duration.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Each mode carries an explicit when-to-use ('Use for caption timing, script editing, or word-level visual sync', 'Use for auto-trim, smart-split', 'Set animation cues on beats or drops'), and features is positioned as a cheap dispatcher to pick a mode. It also names an alternative (get_clip) and states when the words matter, use transcript.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Clueso tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Analyze audio

Input Schema

Schema Changelog

TDQS

Glama MCP Gateway

Resources