Skip to main content
Glama

⚡ FrameForge

AI Video Production Engine for Any Agentic IDE

Any IDE. Any topic. Any style. 1080p broadcast-quality videos from a single tool call.

License: MIT Node.js Version MCP Spec Tests FFmpeg

Quickstart · Connect Your IDE · 13 MCP Tools · Architecture · Deploy


What is FrameForge?

FrameForge is a rendering execution engine. Your IDE is the intelligence.

Connect IBM Bob, Codex, opencode, Cursor, Windsurf, Claude Code, or any MCP-capable tool via the Cloudflare Remote MCP endpoint. The IDE agent writes the video script — FrameForge handles TTS synthesis, AI image generation, FFmpeg compositing, audio mastering, quality gating, and delivery.

No hardcoded topics, styles, or characters. Any format, any language, any visual aesthetic.

Your IDE agent writes this:              FrameForge renders this:
──────────────────────────               ─────────────────────────
scenes: [                                final_video_1080p.mp4 ✓
  { narration: "...",                    short_1_vertical.mp4  ✓
    caption: "OVERLAY",                  video_meta.json       ✓
    visual_description: "..." }          → Telegram / YouTube
]

Related MCP server: short-video-maker

Key Features

Feature

Detail

☁️

Cloudflare Remote MCP

One HTTPS URL, every IDE. ~80% fewer context tokens vs local stdio.

🤖

Agent-first path

create_video_from_script — your IDE writes scenes, FrameForge renders. Zero internal LLM.

🎙️

Multi-platform voice

Switch between ElevenLabs, Azure, OpenAI TTS, and edge-tts per call — no .env change needed.

🎬

Bring your own media

Mix AI visuals with your own clips, images, and pre-recorded voiceovers per scene.

🎚️

Broadcast-grade audio

Voice compression, sidechain music ducking, EBU R128 −14 LUFS normalization.

📱

Vertical Shorts

Silence-detecting boundary clustering + focal-point-aware 9:16 crop.

🛡️

Hard quality gates

Resolution, codec, LUFS, black frames, frozen frames — broken output never ships.

🔁

Resume-safe pipeline

Content-addressed artifact manifest; re-runs skip already-rendered scenes.

💰

Cost tracking

NDJSON usage log with USD estimates per LLM / TTS / image call.


60-Second Quickstart

1. Install prerequisites

node --version    # need ≥ 18
ffmpeg -version   # need ≥ 5.0   (brew install ffmpeg / apt install ffmpeg)
npm install -g edge-tts   # free offline TTS — no account needed

2. Clone and configure

git clone https://github.com/AMG555/FrameForge.git
cd FrameForge
npm run doctor        # pre-flight check — tells you what's missing
npm run setup         # interactive key wizard (~2 min)

Minimum .env for a free, zero-cost run:

GROQ_API_KEY=gsk_...         # free tier — console.groq.com
PROVIDER_LLM=groq
PROVIDER_TTS=edge-tts        # free, offline — no key needed
PROVIDER_IMAGE=pollinations  # free — no key needed

3. Generate your first video

npm run make -- --topic "How does GPS actually work?"

Output in .frameforge-files/:

final_video_<id>.mp4    1080p landscape master
short_1_<id>.mp4        auto-extracted 9:16 vertical Short
video_meta.json         title, description, tags, chapter markers
usage.ndjson            per-call cost log

Connect Your IDE

One URL works everywhere. No local server process, no port forwarding, ~80% fewer context tokens than stdio.

Deploy once (< 2 minutes):

cd cloudflare-mcp
npm install
wrangler secret put WORKER_API_URL      # https://your-render-app.onrender.com
wrangler secret put MCP_AUTH_TOKEN      # any 32-byte hex string
wrangler secret put INTERNAL_API_TOKEN  # different 32-byte hex string
wrangler deploy

Add to any IDE:

{
  "mcpServers": {
    "frameforge": {
      "url": "https://video-mcp.YOUR_SUBDOMAIN.workers.dev/mcp",
      "headers": { "Authorization": "Bearer YOUR_MCP_AUTH_TOKEN" }
    }
  }
}

Same block works in IBM Bob, Codex, opencode, Cursor, Windsurf, Claude Code, VS Code + Copilot, JetBrains AI, and any MCP-capable tool. See cloudflare-mcp/ for the full deployment guide.


💻 Local stdio (alternative — no cloud account needed)

{
  "mcpServers": {
    "frameforge": {
      "command": "node",
      "args": ["/absolute/path/to/FrameForge/src/mcp-server/stdio.js"],
      "env": {
        "GROQ_API_KEY": "gsk_...",
        "PROVIDER_LLM": "groq",
        "PROVIDER_TTS": "edge-tts",
        "PROVIDER_IMAGE": "pollinations"
      }
    }
  }
}

Using FrameForge from Your IDE

Your IDE agent writes the scenes[] array. FrameForge renders it. No internal LLM needed.

create_video_from_script({
  scenes: [
    {
      narration: "WiFi sends data as invisible radio waves at 2.4 or 5 GHz.",
      caption: "RADIO WAVES",
      visual_description: "router emitting concentric waves, flat design, bold colors"
    },
    {
      narration: "Your laptop picks up the signal and decodes the packets instantly.",
      caption: "YOUR DEVICE",
      visual_description: "laptop with signal bars, white background, minimal illustration"
    }
  ],
  imageStyle: "flat design, white background, bold primary colors"
})
→ { masterPath: "/files/final_video_abc123.mp4", passed: true, sceneCount: 2 }

Auto-research path

Let FrameForge handle research and scripting internally:

1. research_topic({ topic: "How GPS works", style: "educational", format: "YouTube short" })
2. write_script({ ...claims, persona: "tech explainer", language: "en", targetDuration: 60 })
3. render_from_scenes({ productionId, skipImages: false })

Bring your own media

Pass per-scene paths to skip AI generation for any scene:

{
  "narration": "Here is the sorting algorithm running step by step.",
  "caption": "ALGORITHM",
  "video_path": "/recordings/screen_capture.mp4",
  "audio_path": "/recordings/my_voice.mp3"
}

Any combination of image_path, video_path, audio_path works. FrameForge generates only what you don't supply.


MCP Tools Reference

#

Tool

Description

1

research_topic

Research any topic. Extracts + fact-checks claims. Accepts persona, style, format, audience.

2

write_script

Generate a video script from claims. Any style, language, persona, duration, or visual aesthetic.

3

generate_scene_image

Generate one scene image using the configured image provider. imageStyle controls the aesthetic.

4

render_tts

Synthesize narration audio for one scene using the configured TTS provider.

5

assemble_video

FFmpeg render + 7-point quality gate → master MP4.

6

publish_video

Deliver finished video to Telegram and/or YouTube.

7

ingest_media

Import your own clips, images, and audio into a scenes manifest.

8

edit_scenes

Read and patch an existing scenes manifest — update captions, swap files, append scenes.

9

render_from_scenes

Render directly from a scenes manifest, skipping research and scripting.

10

external_platform_call

HTTP bridge to VoxForge, Runway, ElevenLabs, Remotion, or any external platform.

11

create_video_from_script

Agent-first. Your IDE writes the scenes, FrameForge renders. No internal LLM.

12

multi_voice_tts

Call any TTS platform per-request (ElevenLabs / Azure / OpenAI / edge-tts / webhook) without touching .env.

13

n8n_webhook_trigger

Trigger an n8n webhook to start an async pipeline run. Returns { jobId, status } immediately.


Providers

Switch any provider by setting the corresponding env var — no code changes.

LLM (10 providers)

PROVIDER_LLM=

Default model

Key env var

Free tier

groq

llama-3.3-70b-versatile

GROQ_API_KEY

✅

cerebras

llama3.3-70b (2000+ tok/s)

CEREBRAS_API_KEY

✅

openrouter

anthropic/claude-sonnet-4-5

OPENROUTER_API_KEY

✅ credits

anthropic

claude-sonnet-4-5

ANTHROPIC_API_KEY

—

gemini

gemini-2.0-flash

GEMINI_API_KEY

✅

mistral

mistral-large-latest

MISTRAL_API_KEY

—

together

Llama-3.3-70B-Instruct-Turbo

TOGETHER_API_KEY

✅ credits

openai

gpt-4o

OPENAI_API_KEY

—

ollama

llama3.3 (local)

—

✅ local

webhook

any OpenAI-compat endpoint

WEBHOOK_LLM_URL

depends

Override model: LLM_MODEL=your-model-id · Override endpoint: LLM_BASE_URL=https://...

TTS (5 providers)

PROVIDER_TTS=

Notes

Key

edge-tts

Free, offline, 300+ voices

—

elevenlabs

Voice cloning, highest quality

ELEVENLABS_API_KEY

azure

Natural voices, multilingual

AZURE_SPEECH_KEY

openai

Fast, clean narration

OPENAI_API_KEY

webhook

Any HTTP TTS endpoint

WEBHOOK_TTS_URL

Use multi_voice_tts to override the provider per call without changing .env.

Image generation (5 providers)

PROVIDER_IMAGE=

Notes

Key

pollinations

Free, no account

—

falai

Fast, high quality

FALAI_API_KEY

replicate

1000+ models

REPLICATE_API_KEY

dalle

DALL-E 3

OPENAI_API_KEY

webhook

Any HTTP image endpoint

CUSTOM_IMAGE_URL


Architecture

YOUR IDE (IBM Bob / Codex / opencode / Cursor / Windsurf / Claude Code)
         │
         │  HTTPS + SSE  (Cloudflare Remote MCP)
         ▼
[Cloudflare Workers]   ← caches schemas, proxies tool calls, keeps IDE context lean
         │
         │  HTTP + INTERNAL_API_TOKEN
         ▼
[FrameForge Node]  ← runs on Render free / VPS / Pi / your machine
         │
         ├─ src/pipeline/
         │   ├─ providers.js        10 LLM + 5 TTS + 5 Image providers
         │   ├─ state-machine.js    file + PostgreSQL production state
         │   ├─ retry-policy.js     exponential backoff, FATAL/NON_RETRYABLE/RETRYABLE
         │   ├─ telemetry.js        NDJSON cost tracking per provider
         │   ├─ artifact-manifest.js  content-addressed idempotency manifest
         │   └─ artifact-retention.js permanent / 30-day / temp sweep
         │
         ├─ src/workflows/
         │   ├─ research.js   research + fact extraction
         │   ├─ script.js     scene schema v2.0.0 generation + normalisation
         │   ├─ assets.js     parallel image gen + TTS (configurable concurrency)
         │   ├─ render.js     FFmpeg zoompan + captions + quality gate
         │   ├─ shorts.js     silence segmentation + focal-point 9:16 crop
         │   ├─ packaging.js  manifest + metadata JSON
         │   └─ publishing.js Telegram + YouTube OAuth2
         │
         ├─ src/mcp-server/
         │   ├─ server.js   HTTP MCP (port 3210) — 13 tools
         │   └─ stdio.js    stdio MCP — same 13 tools
         │
         └─ src/worker/
             ├─ server.js              full worker (local / VPS)
             └─ server-render-free.js  Render.com free tier (api_only mode)
                                       delegates FFmpeg to LOCAL_RENDER_URL

Quality Gates

Every video passes 7 automated checks before delivery. Any failure blocks the output.

Gate

Requirement

Resolution

1920×1080 (landscape) or 1080×1920 (shorts)

Video codec

H.264, high profile, level 4.1, yuv420p

Integrated loudness

−14 LUFS ± 1.5 LU

True peak

≤ −0.5 dBTP

Black frames

None longer than 0.5 s

Frozen frames

None longer than 0.5 s

File integrity

> 1 MB, valid video + audio streams


Deployment

FrameForge has a split-worker design. Network-bound work (LLM, TTS, image APIs) runs on the Render free tier (512 MB RAM). FFmpeg render jobs are delegated to your local machine, Pi, or VPS.

Render free tier  ← research, script, assets, packaging, publishing
Your local machine ← render, shorts (FFmpeg, ~800 MB RAM peak)

See render.yaml for the Render deploy config and docs/deployment.md for the full step-by-step guide.

Environment variables

cp docs/env.template .env
# then edit .env — minimum:
GROQ_API_KEY=gsk_...
PROVIDER_LLM=groq
PROVIDER_TTS=edge-tts
PROVIDER_IMAGE=pollinations

Full reference: docs/env.template

systemd (Linux VPS / Pi)

[Unit]
Description=FrameForge MCP Server
After=network.target

[Service]
User=node
WorkingDirectory=/home/node/FrameForge
EnvironmentFile=/home/node/FrameForge/.env
ExecStart=/usr/bin/node src/mcp-server/server.js
Restart=on-failure

[Install]
WantedBy=multi-user.target
sudo systemctl enable frameforge
sudo systemctl start  frameforge
sudo journalctl -fu   frameforge

CLI Reference

Command

What it does

npm run doctor

Pre-flight check — Node, FFmpeg, edge-tts, env vars

npm run setup

Interactive provider key wizard (writes .env)

npm run setup:check

Validate an existing .env

npm run make -- --topic "..."

Full pipeline: research → render → deliver

npm run mcp

HTTP MCP server on port 3210

npm run mcp:stdio

stdio MCP server

npm run start:worker

Background queue worker

npm run start:worker:render

Render-optimised worker (api_only mode)

npm run start:local-render

Local full-render worker (RENDER_MODE=full)

npm run validate

Deep structural validation (113 checks)

npm test

Unit test suite (272 tests, 47 suites)

npm run make flags:

npm run make -- \
  --topic "How GPS works" \
  --style educational \       # educational | comedic | documentary | vlog | tutorial
  --format "YouTube short" \  # YouTube video | YouTube short | Instagram reel | explainer
  --language en \             # BCP-47 code
  --no-publish \              # render only, skip delivery
  --no-shorts \               # skip vertical crop
  --channels telegram,youtube

Testing

272 tests across 11 modules — run with:

npm test

Module

Tests

Covers

retry-policy

30

Error classification, retry/dead-letter/FATAL, backoff math

telemetry

15

estimateCost all providers, NDJSON round-trip, cost aggregation

artifact-manifest

22

create/load/resume/overwrite, scene updates, idempotency checks

artifact-retention

21

All 4 retention categories, tagging, sweep, dryRun

external-adapter

23

isConfigured, 5 platform presets, auth headers, MIME, poll logic

script

34

normaliseScene all fields, clamping, legacy compat, free-form character_id

animation-system

40

All 11 camera hints, MAX_ZOOM constraint, transitions, auto quadrant

review-gate

22

All risk × status combos, formatReviewRequest, recordHumanDecision

log-redaction

30

14 credential patterns, nested object redaction, safe logger

server-render-free

18

timingSafeEqual, 4 MB body limit, path traversal guard

mcp-server

17

readBodyLimited, LLMProvider.parseJson, TTSProvider overrides, outputPath guard


Security

  • Timing-safe auth — All token comparisons use crypto.timingSafeEqual (no timing oracle)

  • Body size limits — All three HTTP servers reject requests > 4 MB with HTTP 413

  • Path traversal guards — productionId and outputPath validated before any file operation

  • Credential redaction — 14 regex patterns strip keys from all logs at the source

  • No secrets in code — All keys via .env; .gitignore covers it

Report vulnerabilities: SECURITY.md


Contributing

See CONTRIBUTING.md for the full guide.

Quick checklist:

  1. npm run doctor passes on your OS

  2. npm test passes (272/272)

  3. npm run validate passes (113/113 checks)

  4. FFmpeg commands keep -nostdin and cross-platform font paths

  5. Commit messages: plain imperative sentences — no feat:/fix: prefixes


License

MIT — see LICENSE.

Related MCP Connectors

Related MCP Servers