Skip to main content
Glama

Clueso

by io.clueso
OverviewDiscussionsAdmin

Analyze audio

analyze_audio
Read-only

Analyze an audio file. Modes:

  • transcript: Speech-to-text with word-level timestamps — returns {language, full_text, words} where each word is {word, start_ms, end_ms, punctuated_word}. ⚠ MILLISECONDS here; get_clip's voiceover_words gives the same words in SECONDS as {word, start, end}. Use for caption timing, script editing, or word-level visual sync. Works on narration and on sung lyrics: the provider is picked from the content, and if it returns nothing the other one is tried automatically. So an empty words means the audio genuinely has no intelligible speech, not that the wrong engine ran.

  • silences: returns {silences, total_silence_ms}, each span {start_ms, end_ms, peak_db}. Use for auto-trim, smart-split, or pause-aware editing.

  • beats: returns {tempo_bpm, beats_ms} — beats_ms is an array of MILLISECOND positions. Empty for non-music. Use to align animations/zooms to a beat.

  • features: returns {duration_ms, peak_db, rms_db, lufs, has_music, has_speech} — loudness plus two INDEPENDENT hints — has_music (beats hold a steady period) and has_speech (there is audible content above a loudness floor; it is not a speech detector). Both can be true at once, e.g. narration over a music bed. Cheap dispatcher — call once to pick a mode, or for duration_ms to pass as add_audio's source_duration. When the words matter, trust transcript rather than these flags.

Source: provide exactly one of mcp_upload_id (from upload tool) or file_url (already-public URL, e.g. a find(type='music') result or Clueso CDN file). Optional time range: range_start_ms / range_end_ms crops the analysis window.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
modeYesWhich analysis to run.
file_urlNoPublic http(s) URL to the audio. Use when the file is already accessible (e.g. find(type='music') result, Clueso CDN). Validated against private/loopback ranges.
range_end_msNoEnd of analysis window in milliseconds. Default: whole file.
mcp_upload_idNoMCP upload ID from the upload_file tool. Use this OR file_url, not both.
range_start_msNoStart of analysis window in milliseconds. Default 0 (beginning).
silences_optionsNoOptions for mode='silences'.
transcript_optionsNoOptions for mode='transcript'.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed4 schema fields changed
    • removedInput schema / properties / context
      Removed value: -{
-  "description": "Explain in 15-25 words, in third person, why this tool is called and how it supports the user's goal. For analytics only. You MUST describe only the abstract purpose of the tool call. NEVER include, repeat, paraphrase, or infer personal, sensitive, or identifying information from the user request or tool results, including names, emails, phone numbers, IPs, IDs, or credentials. You MUST generalize specific entities into roles such as \"a user\", \"the customer\", or \"an account\". Example: \"Retrieving a customer's recent orders to investigate a billing issue and help support determine the appropriate resolution.\"",
-  "type": "string"
-}
    • removedInput schema / properties / conversation_id
      Removed value: -{
-  "description": "Echo the conversation_id from the server's previous response. The server provides it on the first call — never invent one, and do not issue parallel tool calls until you have it.",
-  "type": "string"
-}
    • removedInput schema / properties / llm_model
      Removed value: -{
-  "description": "The exact model identifier you (the assistant) are running as, taken from your system prompt or environment (e.g. \"claude-opus-4-8\", \"gpt-5.2\"). Used for analytics only. If you do not know your model identifier with certainty, pass \"unknown\" — never guess.",
-  "type": "string"
-}
    • changedInput schema / required
      Previous value: -[
-  "mode",
-  "context",
-  "llm_model"
-]New value: +[
+  "mode"
+]
  2. Changed4 schema fields changed
    • addedInput schema / properties / context
      Added value: +{
+  "description": "Explain in 15-25 words, in third person, why this tool is called and how it supports the user's goal. For analytics only. You MUST describe only the abstract purpose of the tool call. NEVER include, repeat, paraphrase, or infer personal, sensitive, or identifying information from the user request or tool results, including names, emails, phone numbers, IPs, IDs, or credentials. You MUST generalize specific entities into roles such as \"a user\", \"the customer\", or \"an account\". Example: \"Retrieving a customer's recent orders to investigate a billing issue and help support determine the appropriate resolution.\"",
+  "type": "string"
+}
    • addedInput schema / properties / conversation_id
      Added value: +{
+  "description": "Echo the conversation_id from the server's previous response. The server provides it on the first call — never invent one, and do not issue parallel tool calls until you have it.",
+  "type": "string"
+}
    • addedInput schema / properties / llm_model
      Added value: +{
+  "description": "The exact model identifier you (the assistant) are running as, taken from your system prompt or environment (e.g. \"claude-opus-4-8\", \"gpt-5.2\"). Used for analytics only. If you do not know your model identifier with certainty, pass \"unknown\" — never guess.",
+  "type": "string"
+}
    • changedInput schema / required
      Previous value: -[
-  "mode"
-]New value: +[
+  "mode",
+  "context",
+  "llm_model"
+]
  3. Changed7 schema fields changed
    • changedInput schema / $schema
      Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • removedInput schema / additionalProperties
      Removed value: -false
    • addedInput schema / properties / range_end_ms / maximum
      Added value: +9007199254740991
    • addedInput schema / properties / range_start_ms / maximum
      Added value: +9007199254740991
    • removedInput schema / properties / silences_options / additionalProperties
      Removed value: -false
    • addedInput schema / properties / silences_options / properties / min_pause_ms / maximum
      Added value: +9007199254740991
    • removedInput schema / properties / transcript_options / additionalProperties
      Removed value: -false
  4. First observed

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With readOnlyHint=true and destructiveHint=false, the description adds substantial behavioral context: the unit warning (ms vs seconds vs get_clip's voiceover_words), the automatic provider fallback for transcript, the independence of has_music/has_speech flags, and the meaning of an empty words array. These details go far beyond the annotations and prevent misinterpretation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long, but the complexity of the tool (4 modes, 7 params, nested options) justifies the length. It is well structured with bolded mode names, bullet-like formatting, and warnings placed prominently. A slight deduction for density—some agents might be overwhelmed, but every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity and the absence of an output schema, the description is remarkably complete. It details all return structures, unit conventions, fallback behavior, and usage scenarios, and even cross-references sibling tools (get_clip, add_audio). An agent can invoke it correctly with no further information.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description clarifies how each parameter behaves in practice: mode meaning and return shapes, source exclusivity, the effect of range_start_ms/range_end_ms on the analysis window, and the purpose of silences_options and transcript_options. It also explains the unit conversions and fallback logic, adding meaning that the schema alone does not convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Analyze an audio file') and enumerates four distinct modes with concrete use cases, distinguishing it clearly from sibling tools like add_audio (which adds rather than reads) and get_clip (which returns clip metadata, not analysis). The mode list and return-formats leave no ambiguity about what the tool accomplishes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit when-to-use guidance for each mode: transcript for caption timing/script editing, silences for auto-trim, beats for alignment, features for a cheap dispatcher or duration retrieval. It also tells the agent when to trust transcript over the feature flags and how to choose the source parameter. No exclusions are needed since this is a specialized analysis tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Clueso toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources