# Diorama

> Turn any live webpage into an animated 3D video scene via Google Chrome and Model Context Protocol (MCP).

Diorama allows AI agents and developers to inspect a live webpage in Google Chrome, select DOM elements to detach into floating 3D layers, compose cinematic camera and layer keyframe animations, and render deterministic 4K/60fps MP4 videos via WebCodecs.

---

## Architecture Overview

```
┌─────────────────────���───────────────────────────────────────┐
│                        AI AGENT                             │
│   (Claude Code, DeepSeek Harness, Cursor, Windsurf, Codex)  │
└──────────────────────────────┬──────────────────────────────┘
                               │ MCP Stdio Protocol
┌──────────────────────────────▼──────────────────────────────┐
│                    @diorama/bridge                          │
│               Local Node.js MCP Server                      │
│            WebSocket Hub (ws://127.0.0.1:47831)             │
└───────────────┬─────────────────────────────┬───────────────┘
                │ ws                          │ ws
┌───────────────▼───────────────┐ ┌───────────▼───────────────┐
│       Chrome Background       │ │       Diorama Studio      │
│        Service Worker         │ │        WebGL / Tab        │
│  - Tab & DOM discovery        │ │  - Three.js 3D Viewport   │
│  - CDP Native screenshot/crop │ │  - Timeline & Keyframes   │
│  - Cluster extraction (Zap)   │ │  - WebCodecs MP4 Muxer    │
└───────────────────────────────┘ └───────────────────────────┘
```

---

## Agent Quickstart & Installation Manual

*This manual is specifically structured with deterministic steps and completion criteria for AI agents.*

### Prerequisites

| Requirement | Minimum Version | Purpose |
| :--- | :--- | :--- |
| **Node.js** | `>= 18.0.0` (v20+ recommended) | Build tooling & MCP bridge runtime |
| **npm** | `>= 9.0.0` | Package management |
| **Google Chrome / Chromium** | `>= 120.0` (Canary/Dev recommended for experimental APIs) | Host browser for extension & Studio |
| **GPU Acceleration** | WebGL 2.0 active | 3D rendering canvas (`chrome://gpu`) |
| **WebCodecs** | H.264 (`VideoEncoder`) | Deterministic MP4 export |

---

### Phase 1: Build the Chrome Extension

Run from the repository root:

```bash
# 1. Install dependencies
npm install

# 2. Build extension bundles (studio, content script, background service worker)
npm run build
```

**Completion criterion:**
Verify that the `dist/` directory contains:
- `dist/manifest.json`
- `dist/background.js`
- `dist/content.js`
- `dist/studio.html`
- `dist/assets/`

---

### Phase 2: Build the MCP Bridge Server

The MCP bridge bridges your AI agent to the Chrome extension over WebSockets.

```bash
# 1. Move to bridge directory and install dependencies
cd bridge
npm install

# 2. Compile TypeScript
npm run build
cd ..
```

**Completion criterion:**
Verify that `bridge/dist/cli.js` exists and is executable:
```bash
test -f bridge/dist/cli.js && echo "Bridge build SUCCESS"
```

---

### Phase 3: Configure Chrome & Load Extension

#### 1. Chrome Flags & Runtime Requirements

Diorama uses two rendering strategies:
1. **Native CDP Raster (Primary & Default)**: Uses Chrome DevTools Protocol (`chrome.debugger`) to capture pixel-accurate raster textures. Works on all modern Chrome versions (120+).
2. **HTML-in-Canvas Fallback (Experimental)**: Uses the WICG HTML-in-Canvas specification (`canvas.captureElementImage`, `ctx.drawElementImage`).
   - Required only if CDP rastering is unavailable or blocked.
   - To enable in Chromium / Chrome Canary (147+):
     - **Via Chrome Flags UI:** Open `chrome://flags/#canvas-draw-element` and set to **Enabled**.
     - **Via CLI Launch:**
       ```bash
       google-chrome --enable-blink-features=HTMLInCanvas --enable-features=CanvasDrawElement
       # macOS:
       /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --enable-blink-features=HTMLInCanvas --enable-features=CanvasDrawElement
       ```

> **CRITICAL BROWSER RULE:** Chromium suppresses painting on occluded or background tabs. **The Chrome window and the Diorama Studio tab must remain visible (not minimized) during capture and video export.**

#### 2. Load Unpacked Extension

1. Open Google Chrome and navigate to `chrome://extensions`.
2. Toggle **Developer mode** ON (top right switch).
3. Click **Load unpacked** (top left).
4. Select the absolute path to the `dist/` folder inside this repository.
5. Note the Extension ID generated by Chrome (e.g. `pjbbmbniadijhcaldfgocmiifmadnbll`).

---

### Phase 4: Configure the MCP Server in Your Agent

Add the `diorama` MCP server configuration to your AI agent environment. Replace `<ABSOLUTE_PATH_TO_DIORAMA>` with your actual repository path.

#### Claude Desktop (`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS)
```json
{
  "mcpServers": {
    "diorama": {
      "command": "node",
      "args": ["<ABSOLUTE_PATH_TO_DIORAMA>/bridge/dist/cli.js"],
      "env": {
        "DIORAMA_OUTPUT_DIR": "/Users/<USER>/Diorama/exports"
      }
    }
  }
}
```

#### Claude Code (`~/.claude.json` or run CLI)
```bash
claude mcp add diorama -- node <ABSOLUTE_PATH_TO_DIORAMA>/bridge/dist/cli.js
```

#### Cursor (`.cursor/mcp.json` or `~/.cursor/mcp.json`)
```json
{
  "mcpServers": {
    "diorama": {
      "command": "node",
      "args": ["<ABSOLUTE_PATH_TO_DIORAMA>/bridge/dist/cli.js"],
      "env": {
        "DIORAMA_OUTPUT_DIR": "<ABSOLUTE_PATH_TO_DIORAMA>/exports"
      }
    }
  }
}
```

#### Environment Variables for MCP Bridge
- `DIORAMA_BRIDGE_PORT`: WebSocket server port (default: `47831`).
- `DIORAMA_OUTPUT_DIR`: Absolute path where exported `.mp4` video files will be saved (default: `~/Diorama/exports`).

---

### Phase 5: Install the Agent Skill

To give your agent the exact execution rules, install the `diorama` skill.

#### Universal Agent Skills (`~/.agents/skills/diorama`)
```bash
mkdir -p ~/.agents/skills/diorama
cp bridge/skill/SKILL.md ~/.agents/skills/diorama/SKILL.md
```

#### Claude Code Project Skills (`.claude/skills/diorama`)
```bash
mkdir -p .claude/skills/diorama
cp bridge/skill/SKILL.md .claude/skills/diorama/SKILL.md
```

#### Skill Definition (`SKILL.md`)
```markdown
---
name: diorama
description: Turn a live web page into an animated 3D video via the Diorama MCP tools.
---

# Diorama Workflow

Use the `diorama_*` MCP tools to capture web pages from Chrome into 3D scenes, compose camera and layer animations, and render MP4 videos.

## 1. Inspect
- Check readiness with `diorama_status`.
- Call `diorama_inspect_page` to retrieve the viewport screenshot and selectable elements (`candidates` with `[data-dio-id="N"]` selectors, `clusterSelectors`, `rect`, and `kind`).
- If `userSelection` is present, reuse the user's selected elements and brief.

## 2. Capture
- Choose 3 to 8 elements (hero, card cluster, CTA buttons).
- Call `diorama_capture({ selectors: [...], brief: "...", frameFormat: "landscape-16-9" })`.
- This detaches elements, extracts textures, and opens the Studio renderer tab.

## 3. Setup Scene & Framing
- Call `diorama_get_scene` to inspect layers, camera parameters, channels, presets, and easings.
- Adjust duration with `diorama_set_duration({ duration: 6 })`.
- Frame the composition with `diorama_fit({ target: "all", padding: 0.1, orbitX: -8, orbitY: 12 })`.

## 4. Animate
- Apply motion presets: `diorama_apply_preset({ preset: "orbit-reveal" })` or `diorama_apply_preset({ preset: "stagger-cascade" })`.
- Or set custom keyframes via `diorama_set_keyframes`.
- Rules of taste: 1 camera intention per shot; easing `expo.out` for entries, `back.out` for cascades; max 3 simultaneous motions.

## 5. Review & Export
- Check progress with `diorama_contact_sheet({ count: 6 })` or spot-check with `diorama_screenshot({ time: ... })`.
- Render final video with `diorama_export({ quality: "standard" })`.
- Report the saved MP4 path back to the user.
```

---

## Agent Step-by-Step Execution Guide

When an agent is requested to create a 3D video from a webpage, execute the following sequence:

### Step 1: Health Check (`diorama_status`)
Call `diorama_status` to verify connections:
- `connected`: Must be `true`.
- `background`: Must be `true` (Chrome service worker ready).
- `studio`: Becomes `true` once Studio is opened.

### Step 2: Inspect Target Page (`diorama_inspect_page`)
```json
{
  "screenshot": true,
  "limit": 50
}
```
Examine `candidates`:
- Look for hero elements, headings, card grids, CTA buttons.
- Note candidate selectors formatted like `[data-dio-id="12"]`.

### Step 3: Capture Elements into 3D (`diorama_capture`)
```json
{
  "selectors": ["[data-dio-id=\"12\"]", "[data-dio-id=\"14\"]", "[data-dio-id=\"16\"]"],
  "brief": "Smooth dolly-in reveal with floating cards",
  "frameFormat": "landscape-16-9"
}
```
This detaches elements, extracts high-res textures, and opens the Diorama Studio tab.

### Step 4: Scene Setup (`diorama_set_duration`, `diorama_fit`)
Set the timeline length and frame the 3D camera:
```json
// Set 6-second duration
{ "duration": 6 }
```
```json
// Fit camera around all detached elements
{
  "target": "all",
  "padding": 0.12,
  "orbitX": -10,
  "orbitY": 15
}
```

### Step 5: Compose Animation (`diorama_apply_preset`)
Apply camera motion:
```json
{ "preset": "dolly-in", "at": 0 }
```
Apply staggered layer cascades:
```json
{ "preset": "stagger-cascade", "at": 0.3 }
```

### Step 6: Review (`diorama_contact_sheet`)
Generate a contact sheet of 6 snapshots across the duration:
```json
{ "count": 6 }
```
Spot-check any key moment:
```json
{ "time": 2.5 }
```

### Step 7: Export MP4 (`diorama_export`)
```json
{
  "quality": "standard",
  "motionBlurSamples": 2
}
```
The MP4 is rendered via WebCodecs and saved to `DIORAMA_OUTPUT_DIR`. The tool returns:
- `filePath`: Absolute file path on disk.
- `width`, `height`, `fps`, `duration`, `sizeBytes`.

---

## Troubleshooting & Diagnostics

| Symptom | Cause | Solution |
| :--- | :--- | :--- |
| `L'extension Diorama n'est pas connectée (rôle background)` | Chrome is closed or extension not loaded | 1. Open Chrome.<br>2. Ensure extension is loaded at `chrome://extensions`.<br>3. Open any active `http://` or `https://` tab. |
| `L'onglet Studio n'est pas ouvert` | Studio tab was closed or not yet triggered | Call `diorama_open_studio` or `diorama_capture` to create the Studio tab. |
| `No cached paint record for element` | Tab is hidden or in the background | Chromium suspends paint on background tabs. Focus the Chrome window and keep the Studio tab in the foreground. |
| `WebCodecs indisponible` | Hardware acceleration disabled | Check `chrome://gpu`. Ensure WebGL 2.0 and Hardware Video Encode are supported and active. |
| Port conflict on `47831` | Another instance of bridge is running | Kill the lingering process (`lsof -ti :47831 \| xargs kill -9`) or override with `DIORAMA_BRIDGE_PORT=47832`. |

---

## License

MIT