# SceneCraft MCP — Text → Storyboard → Video (Test Version)
Author: snippetWizard
SceneCraft MCP turns plain text into a visual plan (storyboard) and—optionally—an assembled vertical video. It exposes the pipeline both as Python modules and as an MCP tool so agentic clients can orchestrate it.
- Text-to-Scene breakdown using an LLM (OpenAI, Ollama, or an offline Mock).
- Pydantic-backed models for Shots, Scenes, and Storyboards.
- Optional Stable Diffusion frame generation + MoviePy assembly into a vertical MP4.
- File-based storage of generated storyboards.
- MCP server (optional) so tools-savvy LLMs can call it directly.
This repository is a test version. More features are on the way, including cloud uploads and cross-platform social scheduling.
What’s MCP? The Model Context Protocol is a lightweight protocol that lets LLMs talk to local/remote tools securely and consistently. Here, we provide a simple MCP server exposing a “create_storyboard” tool.
Why this project? Planning shots is time-consuming. The idea came from wanting a zero-friction loop for solo creators: describe a scene → get a structured shot plan → (optionally) generate visuals → assemble a draft video. It’s a rapid prototyping lane for script-to-screen experiments.
## Pipeline Overview (Files & Flow)
1) Input (your text/script)
- Entry points:
- `examples/test_scene_local.py:1` — generates a storyboard only.
- `examples/script_to_video.py:1` — full storyboard → frames → video demo.
2) Parse Script → Scenes
- `scenecraft_mcp/engine/script_parser.py:1`
- `parse_script(script)` — minimal parser that treats the whole text as one scene (replaceable later with a real slugline parser).
3) Plan Shots (LLM)
- `scenecraft_mcp/engine/shot_planner.py:1`
- `plan_shots(scene_text, scene_number, style_preset)` → `list[Shot]`
- `_build_system_prompt(...)`, `_build_user_prompt(...)` craft prompt instructions.
- Uses `scenecraft_mcp/llm/factory.py:1` → `get_llm_client()` to select provider.
- Providers (implement `LLMClient`):
- `scenecraft_mcp/llm/base.py:1` — abstract interface (`complete_json`, `complete_text`).
- `scenecraft_mcp/llm/openai_client.py:1` — calls OpenAI Chat Completions and parses strict JSON.
- `scenecraft_mcp/llm/ollama_client.py:1` — calls local Ollama `/api/chat` and parses JSON.
- `scenecraft_mcp/llm/mock_client.py:1` — offline, deterministic mock for local demos (default in `.env`).
- Config:
- `scenecraft_mcp/config.py:1` — `LLMProvider` enum (`openai`, `ollama`, `mock`) and env-driven settings.
4) Data Models & Storage
- `scenecraft_mcp/models.py:1` — Pydantic models: `Shot`, `Scene`, `Storyboard`, enums (`ShotType`, `CameraAngle`, etc.).
- `scenecraft_mcp/storage/repository.py:1` — repo interface.
- `scenecraft_mcp/storage/file_repository.py:1` — JSON-based persistence under `~/.scenecraft_mcp/projects/`.
- `scenecraft_mcp/utils/ids.py:1` — `generate_project_id()` like `proj_ab12cd34`.
5) Optional: Frame Generation (Stable Diffusion)
- `scenecraft_mcp/video/framegen_sd.py:1` — `SDFrameGenerator.generate_frame(...)`
- Loads a Stable Diffusion pipeline (e.g., `runwayml/stable-diffusion-v1-5`) on CPU/GPU.
- Produces stylized vertical 9:16 PNGs per shot.
6) Optional: Video Assembly (MoviePy)
- `scenecraft_mcp/video/assembler.py:1` — `assemble_video(frames, output_path, fps=24)`
- Resizes and concatenates the per-shot images into a vertical MP4 (requires ffmpeg).
7) MCP Server (Optional Integration)
- `scenecraft_mcp/mcp_server.py:1` — `build_server()` with a `create_storyboard(script, title=None)` tool.
- Expose storyboard creation to MCP-compatible hosts (e.g., Claude Desktop) once `mcp` is installed.
## Installation
Core (storyboard + MCP):
```bash
pip install -r requirements.txt
```
Optional (frames + video):
- ffmpeg (required by MoviePy)
- Python packages:
```bash
pip install moviepy diffusers
# Then install PyTorch appropriate for your system:
# CPU-only example:
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
```
Tip: For NVIDIA GPUs, use the official PyTorch site to get the correct CUDA wheels: https://pytorch.org/get-started/locally/
## Configuration
Copy `.env` (already included) and set a provider:
```
# LLM Provider (openai / ollama / mock)
LLM_PROVIDER=mock
# OpenAI
OPENAI_API_KEY=sk-your-key
OPENAI_MODEL=gpt-4.1-mini
# Ollama
OLLAMA_MODEL=llama3
OLLAMA_BASE_URL=http://127.0.0.1:11434
# Performance
LOW_LATENCY_MODE=true
```
- `mock` runs fully offline for predictable local demos.
- `ollama` requires `ollama serve` and a pulled model like `llama3`.
- `openai` requires a valid `OPENAI_API_KEY`.
## Quickstart
Storyboard-only (offline by default):
```bash
python examples/test_scene_local.py
```
Storyboard → Frames → Video (requires optional deps):
```bash
python examples/script_to_video.py
```
Outputs:
- Storyboard JSON: `~/.scenecraft_mcp/projects/<project_id>.json`
- Frames: `outputs/frames/*.png`
- Video: `outputs/videos/<project_id>.mp4`
## Using the MCP Server (optional)
```bash
pip install mcp
python -m scenecraft_mcp.mcp_server
```
Connect this server to an MCP-compatible host (e.g., Claude Desktop). The tool `create_storyboard` accepts `script` and returns `{ project_id, scene_count, shot_count }`.
## What This Solves
- Fast ideation: turn a draft description into a structured shot plan.
- Consistent visual language: shot types, angles, durations captured as data, not prose.
- Extensible pipeline: swap LLMs, models, or frame generators without changing the flow.
- Automation-ready: MCP tool interface and file-based outputs are easy to orchestrate.
## Roadmap (Test Version — Coming Soon)
- Cloud uploads after video generation (S3/GCS/Azure).
- Automation script to fetch by id/date and auto-post to YouTube and Instagram.
- Schedule entire months of content in one pass.
- Better text-to-scene parser for multi-scene scripts.
- Transitions, captions/overlays, TTS/VO integration, music bed.
- More MCP tools and richer schemas.
Follow along — more MCP projects are coming soon. Star and follow on GitHub: snippetWizard. Stay tuned!
## File-by-File Quick Reference
- `scenecraft_mcp/engine/script_parser.py:1` — `parse_script(script)` → minimal single-scene dict.
- `scenecraft_mcp/engine/shot_planner.py:1` — `plan_shots(...)` calls LLM to output `ShotPlan`.
- `scenecraft_mcp/llm/factory.py:1` — `get_llm_client()` from env-configured provider.
- `scenecraft_mcp/llm/openai_client.py:1` — OpenAI JSON completions.
- `scenecraft_mcp/llm/ollama_client.py:1` — Ollama JSON completions.
- `scenecraft_mcp/llm/mock_client.py:1` — offline JSON synthesis for demos.
- `scenecraft_mcp/models.py:1` — core Pydantic models and enums.
- `scenecraft_mcp/storage/file_repository.py:1` — save/load `Storyboard` to `~/.scenecraft_mcp/projects/`.
- `scenecraft_mcp/video/framegen_sd.py:1` — Stable Diffusion image generator per shot.
- `scenecraft_mcp/video/assembler.py:1` — MoviePy image sequence → MP4.
- `scenecraft_mcp/mcp_server.py:1` — MCP tool `create_storyboard`.
- `examples/test_scene_local.py:1` — storyboard demo.
- `examples/script_to_video.py:1` — full pipeline demo.
## License
This project is licensed under the MIT License. See `LICENSE` for details.
Copyright (c) 2025 snippetWizard
