# Lyric Studio

A tool for building lyric videos: a background image + an audio track, with
lyric lines that appear and disappear at exact timestamps you choose, in any
real Google Font.

It has two parts that share the same project:

- **The editor UI** — a web app you open in your browser to upload media,
  type lyric lines with timing, pick a font/color, and render.
- **The MCP server** — the same project, exposed as tools so an AI assistant
  (Claude Desktop, Claude Code, etc.) can add/edit lyric lines and trigger a
  render on your behalf.

Because they share the same `projects/default/project.json`, lines added by
the AI show up in the browser UI instantly (the server watches the file and
pushes an update over Server-Sent Events — no polling), and vice versa.

## Requirements

- **Node.js** 18+
- **ffmpeg** and **ffprobe** on your `PATH` (`ffmpeg -version` to check)
- Internet access (to download Google Fonts on first use of each font)

## Setup

```bash
npm install
```

## 1. Run the editor UI

```bash
npm run start
```

Then open **http://localhost:4790**.

In the UI:
1. Add an audio file.
2. Add one or more background images. For each one, type how long it should
   play in the "until" box (e.g. `3:00`) before clicking **Add image to
   sequence** — images play back-to-back in the order you add them. Leave
   "until" blank on the last image to have it run to the end of the song.
   Example for a 4-minute song with 2 images: add image 1 with `until: 3:00`,
   then image 2 with `until` left blank (it'll auto-run to 4:00).
3. Pick a Google Font, size, color, outline, and position (top/middle/bottom).
4. Add lyric lines: text + start time + end time (`3:05`, `185`, or `1:03:05`
   all work). **Drag a lyric block** on the timeline to shift its timing, or
   **drag its left/right edge** to resize just the start or end — both commit
   automatically when you release.
5. Scrub the timeline to preview which line and which image show at any
   given moment.
6. Click **Render video** — the MP4 appears with a download link when done.

## 2. Run the MCP server (for AI control)

The MCP server talks over stdio, so you point your AI tool's config at it
rather than running it manually yourself.

**Claude Desktop / Claude Code** — add to your MCP config
(`claude_desktop_config.json` or equivalent):

```json
{
  "mcpServers": {
    "lyric-studio": {
      "command": "node",
      "args": ["/absolute/path/to/lyric-video-mcp/mcp-server.js"]
    }
  }
}
```

Restart the client, and the AI will have these tools available:

| Tool | What it does |
|---|---|
| `set_audio` | Set the audio track (absolute file path) |
| `add_image` | Add an image to the visual sequence, with a `until` end time (defaults to end of song if omitted), and an optional transition in from the previous image |
| `update_image` | Change an existing image's start/until time |
| `remove_image` | Delete an image from the sequence |
| `list_images` | See the current image sequence with timing |
| `clear_images` | Wipe the image sequence |
| `set_image_transition` | Set or clear the transition an image blends in with |
| `list_transition_types` | See available transition types (fade, wipes, slides, etc.) |
| `set_style` | Set font (any Google Fonts name), size, color, outline, position, text fade |
| `list_google_fonts` | See a curated list of good font choices |
| `add_lyric_line` | Add one lyric line with a start/end time |
| `update_lyric_line` | Edit an existing line's text or timing |
| `remove_lyric_line` | Delete a line |
| `clear_lyric_lines` | Wipe all lines |
| `list_lyric_lines` | See all current lines |
| `get_project_status` | See the full project state |
| `render_video` | Render the final MP4 |

### Multiple images, one per section of the song

Images play back-to-back in the order you add them — each one runs from
where the previous one ended until the `until` time you give it:

> Set the audio to /Users/me/song.mp3 (it's 4 minutes long). Add
> /Users/me/cover1.jpg with until "3:00". Then add /Users/me/cover2.jpg with
> no "until" — it should run to the end.

That gives you: cover1.jpg for 0:00–3:00, cover2.jpg for 3:00–4:00. Add as
many images as you like this way — each new one just needs its own `until`.
Rendering fails with a clear error if the images don't fully cover the audio
length, so you'll know if a gap needs filling.

### Transitions between images

Every image (except the first) can blend in from the one before it instead
of a hard cut. Pick any built-in type — `fade`, `dissolve`, `wipeleft`,
`slideup`, `circleopen`, `pixelize`, and more (`list_transition_types` for
the full set) — plus a duration in seconds. These are powered by ffmpeg's
`xfade` filter, so there's nothing to download; the blend is generated on
the fly and your lyric timing stays exactly in sync (it borrows a little
display time from each neighboring image rather than shifting the timeline).

> Add /Users/me/cover2.jpg with until "4:00", transition_type "fade",
> transition_duration 0.6.

### Text fade

Off by default (an instant hard cut, same as it's always been) — captions
only fade in/out if you turn it on. Set `textFadeMs` via `set_style` (or
the "Text fade in/out" field in the Type panel), e.g. `150` for a 150ms
fade each way. This is separate from the image transitions above (it uses
ASS subtitle fades, not ffmpeg's `xfade`) since captions are independent
text events rather than a chain of video clips.

### Example prompt to the AI once connected

> Use lyric-studio: set the image to /Users/me/cover.jpg and the audio to
> /Users/me/raikan-cinta.mp3. Use the font "Poppins" in white with a black
> outline, bottom position. Add this lyric line: "Mari mawar raikan cinta,
> tanpa cintanya kita tiada, mawar cinta mawar ku abadi, cinta pemilik segala
> jadi" from 3:05 to 3:35. Then render it.

## How rendering works

Lyric lines are converted into an `.ass` subtitle file and burned into the
video with ffmpeg/libass, using the actual Google Font TTF (downloaded and
cached in `fonts-cache/`) — not a generic system font substitute.

## Project structure

```
shared/projectStore.js   core logic — used by BOTH the UI server and MCP server
shared/fonts.js          Google Fonts catalog + real TTF downloader
shared/timeUtils.js      "3:05" / "185" / "1:03:05" time parsing
ui-server.js             Express server + REST API for the browser UI
mcp-server.js            MCP server (stdio) for AI control
public/                  browser UI (index.html, style.css, app.js)
projects/default/        this project's media, generated .ass file, project.json
fonts-cache/             downloaded Google Font TTF files (cached after first use)
public/output/           rendered MP4s
```

## Notes

- This is a single-project tool by design (there's one active project,
  `default`) — simplest fit for "one song, one video" use. If you want
  multiple concurrent projects, every function in `shared/projectStore.js`
  already takes a `projectId`, so it's a small change to thread a different
  id through both servers.
- Font files are cached after first download, so re-rendering with the same
  font doesn't hit the network again.