tool_clone_voice_from_audio
Create a reusable voice profile from a 5–30 second speech sample. Provide a local WAV file or a YouTube URL clip with an exact transcript to save the voice for later text-to-speech synthesis.
Instructions
Save a voice profile from a reference audio (5-30s WAV ideal).
Two input modes (EITHER ref_audio_path OR audio_url must be provided, not both): A) Local file: pass ref_audio_path='/path/to/sample.wav'. B) URL clip: pass audio_url='https://youtu.be/...', ts=30, tf=45. yt-dlp downloads the audio and ffmpeg slices [ts, tf] into a 24kHz mono 16-bit PCM WAV, which is then used as the reference.
Workflow:
Pick a clean 5-30s speech sample (local file OR URL clip).
Write the exact transcript in ref_text.
Pick a memorable voice_name (used later as voice_name= in synthesize_speech).
Returns a dict with 'status', 'voice_name', 'profile_path', 'ref_audio_path', and (for URL mode) 'source_url', 'ts', 'tf', 'clip_duration_s'.
Common errors:
both ref_audio_path and audio_url provided: pick one.
voice_exists: pass overwrite=True or pick a different voice_name.
bad local path: check the path exists and is readable.
bad URL: yt-dlp/ffmpeg failure will be reported in the raised RuntimeError.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| tf | No | Mode B only: end second of the URL clip (default: end of stream). Example: ts=30, tf=45 yields a 15-second clip from 30s to 45s. Voice cloning needs >=3s; 5-30s is the ideal range. Ignored if audio_url is not provided. | |
| ts | No | Mode B only: start second of the URL clip (default 0). Example: ts=30 starts at 30s into the video. Combined with tf, defines the [ts, tf] clip window. | |
| language | No | Language of the reference audio. 'auto' (default, OmniVoice detects), or one of: 'English', 'Italian', 'French', 'German', 'Spanish', etc. | auto |
| ref_text | Yes | Exact transcript of what is spoken in the reference audio, including punctuation. OmniVoice uses this for prosody matching. Wrong transcripts degrade clone quality. Required in BOTH input modes (local file or URL clip). | |
| audio_url | No | Mode B: YouTube (or any yt-dlp-supported) URL. Combined with ts/tf it downloads + slices a clip into a temp file and uses that as the reference. Example: 'https://www.youtube.com/watch?v=21X5lGlDOfg'. SUGGESTED DURATION: 5-30 seconds ideal for voice cloning. Pipeline: yt-dlp download -> ffmpeg slice [ts, tf] -> 24kHz mono 16-bit PCM WAV. Use EITHER this OR ref_audio_path, not both. | |
| overwrite | No | If True, replace an existing profile with the same voice_name. Default: False. | |
| voice_name | Yes | Unique identifier for this voice (alphanumeric + underscores, 1-64 chars). Will be slugified. Use a memorable name like 'claudia_asmr' or 'nasa_male_v1'. Used in subsequent synthesize_speech(voice_name=...) calls. | |
| description | No | Optional human-readable description (e.g. 'ASMR whisper, female, CC0 from archive.org' or 'NASA mission audio, male, PD US Gov'). | |
| ref_audio_path | No | Mode A: absolute path to the reference audio file (WAV, 16-bit PCM, 24kHz ideal). 5-30 seconds of clear, single-speaker speech works best. Example: '/home/jagones/Repositories/VoiceStudio/inputs/asmr_sample.wav'. Use EITHER this OR audio_url, not both. Omit both to get a validation error. |