Scratch MCP Server

An MCP server that builds real Scratch 3 projects from a prompt. Ask for a scene in plain language — "put two characters on a classroom background and make them take turns talking" — and the assistant drives this server to produce a .sb3 file that opens in the Scratch editor with proper sprites, costumes, backdrops, sounds and block scripts.

Nothing is faked: the output is the same project format Scratch saves, so every script is editable in the Scratch editor afterwards.

What it can do

• Backdrops — nine built-in stage backgrounds (sky, hills, night, space, room, underwater, grid, blank, any flat colour), or your own image file.

• Sprites — built-in vector shapes, or your own art.

• Character import — hand it a jpg/png of a character on a flat background; it keys the background out, trims the margins, scales it to the stage and can build talk frames so a still picture reads as talking when the costumes alternate.

• Scripts — a compact JSON action DSL covering motion, looks, sound, events, control, sensing, variables, lists and the pen extension. It compiles to genuine Scratch blocks, hat blocks included.

• Sounds — seven synthesised built-ins, or your own .wav.

• Speech — add_speech turns a line of text into a spoken sound using the machine's own offline text-to-speech engine, and reports the clip length plus the exact mouth-loop count needed to lip-sync a talking animation to it.

• Preview — serves the project to TurboWarp over loopback so you can watch it run before saving.

• GIF export — an animated-GIF encoder, so a finished scene can be sent to someone who does not have Scratch.

Related MCP server: scratch-mcp

Install

npm install
npm run build

Then register it with your MCP client. For Claude Code:

claude mcp add scratch -- node "C:/path/to/Scratch-MCP-Server/dist/index.js"

Or in a client config file (Claude Desktop, VS Code, …):

{
  "mcpServers": {
    "scratch": {
      "command": "node",
      "args": ["C:/path/to/Scratch-MCP-Server/dist/index.js"],
      "env": { "SCRATCH_MCP_OUT_DIR": "C:/Users/you/Desktop" }
    }
  }
}

SCRATCH_MCP_OUT_DIR sets where save_project writes when no path is given; it defaults to ./scratch-projects under the working directory.

Tools

The script DSL

A script is a target, a hat block, and a list of actions. Every action is an object with exactly one key; an action that takes no argument can be written as a bare string.

{
  "target": "Cat",
  "when": "flag",
  "do": [
    { "say": ["Hello!", 2] },
    { "forever": [ { "move": 4 }, "bounce_on_edge", "next_costume", { "wait": 0.1 } ] }
  ]
}

Values can be expressions instead of literals, using the same one-key rule:

{ "if": { "test": { ">": [ { "var": "score" }, 10 ] },
          "do":   [ { "say": ["You win!", 2] }, { "stop": "all" } ] } }

Call dsl_reference for the complete list — it is written to be read by the assistant, so in practice you just describe what you want and it writes the scripts.

Making a still character talk

Scratch animates by swapping costumes. import_character builds two extra costumes that are subtly squashed and stretched, so alternating them gives the bob of someone speaking — no speech bubbles needed:

{ "target": "Girl", "when": { "broadcast": "girl talks" },
  "do": [ { "repeat": { "times": 9, "do": [
            { "switch_costume": "Girl-talk-a" }, { "wait": 0.11 },
            { "switch_costume": "Girl-talk-b" }, { "wait": 0.13 } ] } },
          { "switch_costume": "Girl" },
          { "broadcast": "boy talks" } ] }

Two characters passing broadcasts back and forth reads as a conversation, and it loops forever. examples/students-talking.mjs builds exactly that scene:

node examples/students-talking.mjs "C:/path/to/art-folder" out/students-talking.sb3

Speech and lip-sync

add_speech uses whatever offline engine the host already has — Windows SAPI and the WinRT speech API, macOS say, or espeak-ng on Linux. No network, no API key, no audio file to manage.

Which voices exist differs per machine, so ask list_voices first. Request a voice by gender or by part of its name; a voice that is not installed is an error rather than a silent substitution:

{ "target": "Boy", "text": "Almost. I still need to draw the diagram.", "voice": "male" }

On Windows this matters more than it looks. System.Speech only sees the old SAPI5 voice store, which on many machines contains female voices only; the male voices often live in the OneCore store and are reachable only through the WinRT API. Both paths are implemented, and each voice is tagged with the engine that can actually speak it.

The timing trap is worth stating plainly: play_sound_until_done blocks, so a mouth loop written after it animates after the audio has finished. Use the non-blocking play_sound, then the loop — add_speech returns the repeat count to use:

{ "target": "Boy", "when": { "broadcast": "boy talks" }, "do": [
  { "play_sound": "boy-line-1" },
  { "repeat": { "times": 13, "do": [
    { "switch_costume": "Boy-talk-a" }, { "wait": 0.11 },
    { "switch_costume": "Boy-talk-b" }, { "wait": 0.13 } ] } },
  { "switch_costume": "Boy" },
  { "broadcast": "girl talks" } ] }

examples/students-talking-voice.mjs builds the full voiced conversation:

node examples/students-talking-voice.mjs "C:/path/to/art-folder" out/students-talking-voice.sb3

Opening the result

• Scratch 3: File ▸ Load from your computer ▸ pick the .sb3.

• TurboWarp: drag the file onto the page, or use preview_project for a link that loads it straight from localhost.

Tests

npm test

npm test runs two suites.

test/smoke.mjs drives the built server over stdio exactly like an MCP client: it builds a project that exercises motion, looks, sound, speech, variables, conditionals and key/click hats, checks that bad input comes back as a readable error rather than a crash, and — when the sample art is present — builds the two-character talking scene.

test/gif-roundtrip.mjs encodes synthetic frames and decodes them again with an independent decoder written for the test, comparing pixels. This matters more than it sounds: a browser's <img> tag will happily display a GIF whose LZW stream is subtly malformed, so a strict decode is the only check that proves the encoder is correct.

Notes and limits

• Sounds import from .wav only. The built-in sounds and all generated speech are produced locally, so the server never needs the network.

• Speech quality is whatever the OS ships. The voices are serviceable rather than lifelike; this trades realism for working offline with no key and no upload.

• Images import from svg, png and jpg. Background removal flood-fills inward from the border, so white clothing in the middle of a character survives; it expects a reasonably flat background.

• Custom blocks ("My Blocks") and the music/video/translate extensions are not implemented yet. The pen extension is.

• The server holds one project in memory at a time; create_project replaces it.

Licence

MIT — see LICENSE.