# VoiceCard

**Clone your voice and turn it into a shareable waveform _video_, all driven by MCP. No GUI. Apple Silicon only.**

Voice cloning is now commodity. The differentiator is _delivery_: a short MP4 with a
beautiful audio-reactive waveform plays inline in WhatsApp, iMessage, Signal, and Slack,
where raw audio files stall, fail to autoplay, or won't cross iOS/Android. Every clip
you send is also an ad for the tool. That is the whole idea.

The front end **is** the MCP. You talk to it from Claude Code, Cursor, or any
MCP-aware agent:

> "Clone my voice from `~/Desktop/me.mov` (here's the transcript), then make a
> 15-second Miami-palette clip saying happy birthday, with my photo in the center."

## How it works

Fully standalone. Speech is generated locally by
[`mlx-audio`](https://github.com/Blaizzy/mlx-audio) (Qwen3-TTS via MLX), the waveform
video is rendered with MLX / Pillow, and muxed with `ffmpeg`. No cloud, no Voicebox,
no desktop app.

Cloning is zero-shot in-context learning: it trims your reference to ~12 seconds and
conditions on it. There is no training step and no separate model per voice.

## Requirements

- Apple Silicon Mac (arm64). Enforced at startup.
- Nothing else to install by hand. `ffmpeg` ships bundled (static binary via
  `imageio-ffmpeg`); a system `ffmpeg` on PATH is used automatically if present.

### Models

VoiceCard uses two models, downloaded **once** by `voicecard-provision` (see Install)
into the shared Hugging Face cache (`~/.cache/huggingface/hub/`):

- **Qwen3-TTS** (~4 GB) — speech synthesis.
- **Whisper** (`large-v3-turbo`, ~1.5 GB) — auto-transcribes references and times captions.

**No Hugging Face account or token is required** — both are public. Anonymous downloads
are rate-limited, so if provisioning is slow you can set a free token first (optional):

```bash
export HF_TOKEN=hf_xxx      # optional — faster/de-throttled provisioning download
```

After provisioning, the MCP server runs **offline** and loads the cached models into
memory in a few seconds on first use — it never downloads. (It has to: mlx-audio's loader
makes a blocking network check that deadlocks against the server's async event loop, so
the download is deliberately kept out-of-band in `voicecard-provision`.)

## Install

Needs [`uv`](https://docs.astral.sh/uv/) (`brew install uv`, or the one-line installer
from that page). Nothing else — ffmpeg is bundled; models are fetched by the one-time
`voicecard-provision` step below.

**1. Install, straight from GitHub:**

```bash
uv tool install git+https://github.com/JasonMakes801/voicecard-mcp
```

**2. Provision once** (downloads the ~5 GB of models, verifies ffmpeg, vets with a real
load — this is the only step that touches the network, with progress in your terminal):

```bash
voicecard-provision
```

**3. Register it with Claude Code** (the `voicecard` command is now on your PATH):

```bash
claude mcp add -s user voicecard -- voicecard
```

That's it. Ask your agent to `clone_voice` and `make_clip`.

> The MCP server runs **offline** — it never downloads. It loads the provisioned models
> into memory on first use (a few seconds) and warms them lazily. `check_environment()`
> reports readiness; `warm_up()` loads them explicitly. If models are missing it tells you
> to run `voicecard-provision`. (Provisioning is separate on purpose: model loading
> deadlocks against a live event loop if it tries to hit the network, so the server must
> stay offline and downloads happen out-of-band.)

<details>
<summary>Alternative: from a local clone (for development)</summary>

```bash
git clone https://github.com/JasonMakes801/voicecard-mcp
cd voicecard-mcp
uv tool install .                      # or: uv run voicecard

# register the in-place checkout
claude mcp add -s user voicecard -- uv --directory "$PWD" run voicecard
```
</details>

## Tools

| Tool | What it does |
| --- | --- |
| `check_environment()` | Fast readiness probe (no heavy work): is ffmpeg present, are the models downloaded and loaded (warm), is it `ready`. Call first. |
| `warm_up()` | Load the models into memory now (a few seconds). Optional — the first `make_clip` warms lazily anyway, but this front-loads the wait. |
| `list_voices()` | List stored voices, each paired with its `ref_text`, `ref_audio`, and companion `avatar` (or `null`). |
| `palettes()` | Render the palettes as an **inline swatch image** (shows in the chat, nothing to open) plus the list of names to pass to `make_clip`. |
| `clone_voice(name, source, transcript?, photo?)` | Clone from an **audio or video** file. Trims the reference to ~12s and stores a profile. `transcript` is **optional** — omit it and VoiceCard auto-transcribes the reference (Whisper). `photo` is an optional companion avatar, **copied** into the profile so voice + face travel together. |
| `speak(text, voice)` | Generate speech in a cloned voice; returns a wav path. |
| `make_clip(text, voice, palette?, photo?, label?, aspect?, layout?, arc?, captions?)` | Generate speech **and render the shareable waveform mp4**. Captions (pop-up bubbles timed to delivery) are **on by default**; avatar falls back to the voice's stored `photo`. Returns the mp4 path. |

**Palettes:** `Noir` (default), `70s Gold`, `Miami`, `Aurora`, `Sunset`, `Mint`, `Blueprint`
**Aspect:** `square` (1080², default), `portrait` (9:16)
**Layout:** `linear` (default — circular photo + compact gradient waveform below, Dynamic-Island style) · `circle` (centered audio-reactive spectrum ring)
**Arc** (circle layout only): `full` (default, mirrored ring) · `top` (spikes over the top half)

## Config (env)

| Var | Default |
| --- | --- |
| `VOICECARD_OUT` | `~/VoiceCard` (profiles + rendered clips) |
| `VOICECARD_MODEL` | `mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16` |
| `VOICECARD_WHISPER` | `mlx-community/whisper-large-v3-turbo` |
| `VOICECARD_FFMPEG` | override the ffmpeg binary (else system PATH, else bundled) |
| `VOICECARD_URL` | QR target printed on each clip |
| `VOICECARD_TAGLINE` | tagline printed on each clip |

## Testing

```bash
uv pip install -e ".[test]"

pytest tests/test_fast.py tests/test_mcp.py -q      # fast: no model, no network (~1s)
VOICECARD_RUN_MODEL_TESTS=1 pytest tests/test_integration.py -q   # real Qwen+Whisper (downloads GBs)
bash scripts/virgin_install_test.sh                 # fresh-venv install + smoke, from scratch
```

- **Fast** (`test_fast.py`, `test_mcp.py`): palette integrity, caption alignment
  (input words, STT timings), arg validation, and the MCP protocol surface.
- **Integration** (`test_integration.py`, gated): clone → speak → make_clip end to end,
  asserting the mp4 has audio+video and non-zero duration.
- **Virgin install** (`scripts/virgin_install_test.sh`): installs into a throwaway venv
  and verifies imports, the bundled-ffmpeg fallback (no brew), the console script, and
  the no-model suites against the *installed* package.

## Notes

- Output is stereo, `+faststart`, H.264 High profile — tuned to autoplay inline in
  messengers.
- The waveform is stylized to always look alive, not a diagnostic spectrum analyzer.
- Core ML / ANE offload of the autoregressive decoder is **not** planned (dynamic
  shapes + KV cache + sequential decode are a poor ANE fit). A future Core ML
  **vocoder** offload is the only piece worth exploring.

## Roadmap

- **Captions without Whisper (harvest Qwen's own alignment).** Qwen3-TTS runs at
  12 Hz and already knows word timing internally via its cross-attention (a monotonic
  text-token → audio-frame ridge). Today we recover timing by running Whisper on the
  output — robust but a whole second model. Instead, patch mlx-audio's decode to surface
  the cross-attention for the *target* region (front-cut the ICL reference), extract a
  monotonic alignment, and map text tokens → words. This would move Whisper off the
  per-clip hot path: combined with passing `transcript` at clone time, Whisper would
  never run (or download) for a normal user. Risk: mlx-audio's internal decode is not a
  stable API, so **pin the version and add a drift test** (Qwen-derived timings vs
  Whisper on a fixture; fail if they diverge). Note: there are no phonemes to align —
  only BPE text tokens → audio frames.

## License

MIT. You are responsible for only cloning voices you own or are authorized to use.