# SceneCraft MCP — Text → Storyboard → Video (Test Version)

Author: snippetWizard

SceneCraft MCP turns plain text into a visual plan (storyboard) and—optionally—an assembled vertical video. It exposes the pipeline both as Python modules and as an MCP tool so agentic clients can orchestrate it.

- Text-to-Scene breakdown using an LLM (OpenAI, Ollama, or an offline Mock).
- Pydantic-backed models for Shots, Scenes, and Storyboards.
- Optional Stable Diffusion frame generation + MoviePy assembly into a vertical MP4.
- File-based storage of generated storyboards.
- MCP server (optional) so tools-savvy LLMs can call it directly.

This repository is a test version. More features are on the way, including cloud uploads and cross-platform social scheduling.

What’s MCP? The Model Context Protocol is a lightweight protocol that lets LLMs talk to local/remote tools securely and consistently. Here, we provide a simple MCP server exposing a “create_storyboard” tool.

Why this project? Planning shots is time-consuming. The idea came from wanting a zero-friction loop for solo creators: describe a scene → get a structured shot plan → (optionally) generate visuals → assemble a draft video. It’s a rapid prototyping lane for script-to-screen experiments.

## Pipeline Overview (Files & Flow)

1) Input (your text/script)
   - Entry points:
     - `examples/test_scene_local.py:1` — generates a storyboard only.
     - `examples/script_to_video.py:1` — full storyboard → frames → video demo.

2) Parse Script → Scenes
   - `scenecraft_mcp/engine/script_parser.py:1`
     - `parse_script(script)` — minimal parser that treats the whole text as one scene (replaceable later with a real slugline parser).

3) Plan Shots (LLM)
   - `scenecraft_mcp/engine/shot_planner.py:1`
     - `plan_shots(scene_text, scene_number, style_preset)` → `list[Shot]`
     - `_build_system_prompt(...)`, `_build_user_prompt(...)` craft prompt instructions.
     - Uses `scenecraft_mcp/llm/factory.py:1` → `get_llm_client()` to select provider.
   - Providers (implement `LLMClient`):
     - `scenecraft_mcp/llm/base.py:1` — abstract interface (`complete_json`, `complete_text`).
     - `scenecraft_mcp/llm/openai_client.py:1` — calls OpenAI Chat Completions and parses strict JSON.
     - `scenecraft_mcp/llm/ollama_client.py:1` — calls local Ollama `/api/chat` and parses JSON.
     - `scenecraft_mcp/llm/mock_client.py:1` — offline, deterministic mock for local demos (default in `.env`).
   - Config:
     - `scenecraft_mcp/config.py:1` — `LLMProvider` enum (`openai`, `ollama`, `mock`) and env-driven settings.

4) Data Models & Storage
   - `scenecraft_mcp/models.py:1` — Pydantic models: `Shot`, `Scene`, `Storyboard`, enums (`ShotType`, `CameraAngle`, etc.).
   - `scenecraft_mcp/storage/repository.py:1` — repo interface.
   - `scenecraft_mcp/storage/file_repository.py:1` — JSON-based persistence under `~/.scenecraft_mcp/projects/`.
   - `scenecraft_mcp/utils/ids.py:1` — `generate_project_id()` like `proj_ab12cd34`.

5) Optional: Frame Generation (Stable Diffusion)
   - `scenecraft_mcp/video/framegen_sd.py:1` — `SDFrameGenerator.generate_frame(...)`
     - Loads a Stable Diffusion pipeline (e.g., `runwayml/stable-diffusion-v1-5`) on CPU/GPU.
     - Produces stylized vertical 9:16 PNGs per shot.

6) Optional: Video Assembly (MoviePy)
   - `scenecraft_mcp/video/assembler.py:1` — `assemble_video(frames, output_path, fps=24)`
     - Resizes and concatenates the per-shot images into a vertical MP4 (requires ffmpeg).

7) MCP Server (Optional Integration)
   - `scenecraft_mcp/mcp_server.py:1` — `build_server()` with a `create_storyboard(script, title=None)` tool.
   - Expose storyboard creation to MCP-compatible hosts (e.g., Claude Desktop) once `mcp` is installed.

## Installation

Core (storyboard + MCP):

```bash
pip install -r requirements.txt
```

Optional (frames + video):

- ffmpeg (required by MoviePy)
- Python packages:

```bash
pip install moviepy diffusers
# Then install PyTorch appropriate for your system:
# CPU-only example:
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
```

Tip: For NVIDIA GPUs, use the official PyTorch site to get the correct CUDA wheels: https://pytorch.org/get-started/locally/

## Configuration

Copy `.env` (already included) and set a provider:

```
# LLM Provider (openai / ollama / mock)
LLM_PROVIDER=mock

# OpenAI
OPENAI_API_KEY=sk-your-key
OPENAI_MODEL=gpt-4.1-mini

# Ollama
OLLAMA_MODEL=llama3
OLLAMA_BASE_URL=http://127.0.0.1:11434

# Performance
LOW_LATENCY_MODE=true
```

- `mock` runs fully offline for predictable local demos.
- `ollama` requires `ollama serve` and a pulled model like `llama3`.
- `openai` requires a valid `OPENAI_API_KEY`.

## Quickstart

Storyboard-only (offline by default):

```bash
python examples/test_scene_local.py
```

Storyboard → Frames → Video (requires optional deps):

```bash
python examples/script_to_video.py
```

Outputs:
- Storyboard JSON: `~/.scenecraft_mcp/projects/<project_id>.json`
- Frames: `outputs/frames/*.png`
- Video: `outputs/videos/<project_id>.mp4`

## Using the MCP Server (optional)

```bash
pip install mcp
python -m scenecraft_mcp.mcp_server
```

Connect this server to an MCP-compatible host (e.g., Claude Desktop). The tool `create_storyboard` accepts `script` and returns `{ project_id, scene_count, shot_count }`.

## What This Solves

- Fast ideation: turn a draft description into a structured shot plan.
- Consistent visual language: shot types, angles, durations captured as data, not prose.
- Extensible pipeline: swap LLMs, models, or frame generators without changing the flow.
- Automation-ready: MCP tool interface and file-based outputs are easy to orchestrate.

## Roadmap (Test Version — Coming Soon)

- Cloud uploads after video generation (S3/GCS/Azure).
- Automation script to fetch by id/date and auto-post to YouTube and Instagram.
- Schedule entire months of content in one pass.
- Better text-to-scene parser for multi-scene scripts.
- Transitions, captions/overlays, TTS/VO integration, music bed.
- More MCP tools and richer schemas.

Follow along — more MCP projects are coming soon. Star and follow on GitHub: snippetWizard. Stay tuned!

## File-by-File Quick Reference

- `scenecraft_mcp/engine/script_parser.py:1` — `parse_script(script)` → minimal single-scene dict.
- `scenecraft_mcp/engine/shot_planner.py:1` — `plan_shots(...)` calls LLM to output `ShotPlan`.
- `scenecraft_mcp/llm/factory.py:1` — `get_llm_client()` from env-configured provider.
- `scenecraft_mcp/llm/openai_client.py:1` — OpenAI JSON completions.
- `scenecraft_mcp/llm/ollama_client.py:1` — Ollama JSON completions.
- `scenecraft_mcp/llm/mock_client.py:1` — offline JSON synthesis for demos.
- `scenecraft_mcp/models.py:1` — core Pydantic models and enums.
- `scenecraft_mcp/storage/file_repository.py:1` — save/load `Storyboard` to `~/.scenecraft_mcp/projects/`.
- `scenecraft_mcp/video/framegen_sd.py:1` — Stable Diffusion image generator per shot.
- `scenecraft_mcp/video/assembler.py:1` — MoviePy image sequence → MP4.
- `scenecraft_mcp/mcp_server.py:1` — MCP tool `create_storyboard`.
- `examples/test_scene_local.py:1` — storyboard demo.
- `examples/script_to_video.py:1` — full pipeline demo.

## License

This project is licensed under the MIT License. See `LICENSE` for details.

Copyright (c) 2025 snippetWizard