SceneCraft MCP — Text → Storyboard → Video (Test Version)

Author: snippetWizard

SceneCraft MCP turns plain text into a visual plan (storyboard) and—optionally—an assembled vertical video. It exposes the pipeline both as Python modules and as an MCP tool so agentic clients can orchestrate it.

Text-to-Scene breakdown using an LLM (OpenAI, Ollama, or an offline Mock).
Pydantic-backed models for Shots, Scenes, and Storyboards.
Optional Stable Diffusion frame generation + MoviePy assembly into a vertical MP4.
File-based storage of generated storyboards.
MCP server (optional) so tools-savvy LLMs can call it directly.

This repository is a test version. More features are on the way, including cloud uploads and cross-platform social scheduling.

What’s MCP? The Model Context Protocol is a lightweight protocol that lets LLMs talk to local/remote tools securely and consistently. Here, we provide a simple MCP server exposing a “create_storyboard” tool.

Why this project? Planning shots is time-consuming. The idea came from wanting a zero-friction loop for solo creators: describe a scene → get a structured shot plan → (optionally) generate visuals → assemble a draft video. It’s a rapid prototyping lane for script-to-screen experiments.

Pipeline Overview (Files & Flow)

Input (your text/script)
- Entry points:
  - examples/test_scene_local.py:1 — generates a storyboard only.
  - examples/script_to_video.py:1 — full storyboard → frames → video demo.
Parse Script → Scenes
- scenecraft_mcp/engine/script_parser.py:1
  - parse_script(script) — minimal parser that treats the whole text as one scene (replaceable later with a real slugline parser).
Plan Shots (LLM)
- scenecraft_mcp/engine/shot_planner.py:1
  - plan_shots(scene_text, scene_number, style_preset) → list[Shot]
  - _build_system_prompt(...), _build_user_prompt(...) craft prompt instructions.
  - Uses scenecraft_mcp/llm/factory.py:1 → get_llm_client() to select provider.
- Providers (implement LLMClient):
  - scenecraft_mcp/llm/base.py:1 — abstract interface (complete_json, complete_text).
  - scenecraft_mcp/llm/openai_client.py:1 — calls OpenAI Chat Completions and parses strict JSON.
  - scenecraft_mcp/llm/ollama_client.py:1 — calls local Ollama /api/chat and parses JSON.
  - scenecraft_mcp/llm/mock_client.py:1 — offline, deterministic mock for local demos (default in .env).
- Config:
  - scenecraft_mcp/config.py:1 — LLMProvider enum (openai, ollama, mock) and env-driven settings.
Data Models & Storage
- scenecraft_mcp/models.py:1 — Pydantic models: Shot, Scene, Storyboard, enums (ShotType, CameraAngle, etc.).
- scenecraft_mcp/storage/repository.py:1 — repo interface.
- scenecraft_mcp/storage/file_repository.py:1 — JSON-based persistence under ~/.scenecraft_mcp/projects/.
- scenecraft_mcp/utils/ids.py:1 — generate_project_id() like proj_ab12cd34.
Optional: Frame Generation (Stable Diffusion)
- scenecraft_mcp/video/framegen_sd.py:1 — SDFrameGenerator.generate_frame(...)
  - Loads a Stable Diffusion pipeline (e.g., runwayml/stable-diffusion-v1-5) on CPU/GPU.
  - Produces stylized vertical 9:16 PNGs per shot.
Optional: Video Assembly (MoviePy)
- scenecraft_mcp/video/assembler.py:1 — assemble_video(frames, output_path, fps=24)
  - Resizes and concatenates the per-shot images into a vertical MP4 (requires ffmpeg).
MCP Server (Optional Integration)
- scenecraft_mcp/mcp_server.py:1 — build_server() with a create_storyboard(script, title=None) tool.
- Expose storyboard creation to MCP-compatible hosts (e.g., Claude Desktop) once mcp is installed.

Installation

Core (storyboard + MCP):

pip install -r requirements.txt

Optional (frames + video):

ffmpeg (required by MoviePy)
Python packages:

pip install moviepy diffusers # Then install PyTorch appropriate for your system: # CPU-only example: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu

Tip: For NVIDIA GPUs, use the official PyTorch site to get the correct CUDA wheels: https://pytorch.org/get-started/locally/

Configuration

Copy .env (already included) and set a provider:

# LLM Provider (openai / ollama / mock) LLM_PROVIDER=mock # OpenAI OPENAI_API_KEY=sk-your-key OPENAI_MODEL=gpt-4.1-mini # Ollama OLLAMA_MODEL=llama3 OLLAMA_BASE_URL=http://127.0.0.1:11434 # Performance LOW_LATENCY_MODE=true

mock runs fully offline for predictable local demos.
ollama requires ollama serve and a pulled model like llama3.
openai requires a valid OPENAI_API_KEY.

Quickstart

Storyboard-only (offline by default):

python examples/test_scene_local.py

Storyboard → Frames → Video (requires optional deps):

python examples/script_to_video.py

Outputs:

Storyboard JSON: ~/.scenecraft_mcp/projects/<project_id>.json
Frames: outputs/frames/*.png
Video: outputs/videos/<project_id>.mp4

Using the MCP Server (optional)

pip install mcp python -m scenecraft_mcp.mcp_server

Connect this server to an MCP-compatible host (e.g., Claude Desktop). The tool create_storyboard accepts script and returns { project_id, scene_count, shot_count }.

What This Solves

Fast ideation: turn a draft description into a structured shot plan.
Consistent visual language: shot types, angles, durations captured as data, not prose.
Extensible pipeline: swap LLMs, models, or frame generators without changing the flow.
Automation-ready: MCP tool interface and file-based outputs are easy to orchestrate.

Roadmap (Test Version — Coming Soon)

Cloud uploads after video generation (S3/GCS/Azure).
Automation script to fetch by id/date and auto-post to YouTube and Instagram.
Schedule entire months of content in one pass.
Better text-to-scene parser for multi-scene scripts.
Transitions, captions/overlays, TTS/VO integration, music bed.
More MCP tools and richer schemas.

Follow along — more MCP projects are coming soon. Star and follow on GitHub: snippetWizard. Stay tuned!

File-by-File Quick Reference

scenecraft_mcp/engine/script_parser.py:1 — parse_script(script) → minimal single-scene dict.
scenecraft_mcp/engine/shot_planner.py:1 — plan_shots(...) calls LLM to output ShotPlan.
scenecraft_mcp/llm/factory.py:1 — get_llm_client() from env-configured provider.
scenecraft_mcp/llm/openai_client.py:1 — OpenAI JSON completions.
scenecraft_mcp/llm/ollama_client.py:1 — Ollama JSON completions.
scenecraft_mcp/llm/mock_client.py:1 — offline JSON synthesis for demos.
scenecraft_mcp/models.py:1 — core Pydantic models and enums.
scenecraft_mcp/storage/file_repository.py:1 — save/load Storyboard to ~/.scenecraft_mcp/projects/.
scenecraft_mcp/video/framegen_sd.py:1 — Stable Diffusion image generator per shot.
scenecraft_mcp/video/assembler.py:1 — MoviePy image sequence → MP4.
scenecraft_mcp/mcp_server.py:1 — MCP tool create_storyboard.
examples/test_scene_local.py:1 — storyboard demo.
examples/script_to_video.py:1 — full pipeline demo.

License

This project is licensed under the MIT License. See LICENSE for details.

This server cannot be installed

-

security - not tested

A

license - permissive license

-

quality - not tested

How are these scores calculated?

hybrid server

The server is able to function both locally and remotely, depending on the configuration or use case.

Converts plain text scripts into structured storyboards with shot breakdowns using LLMs, and optionally generates visual frames via Stable Diffusion and assembles them into vertical videos for rapid content prototyping.

SceneCraft MCP