stackchan-mcp-mod

An MCP server MOD for Stack-chan that exposes the robot's capabilities as MCP tools, so an MCP client (Claude Code, Codex, …) can drive the robot over the network: take a photo, listen to the room, move the head, light the LEDs, speak, and react to being touched.

It is a replacement for the stock "MCP Server" sample MOD, which offers only set_emotion and say_message and — because a MOD's onContextCreated replaces the host default — switches off the built-in behaviors such as head petting. This MOD runs the default behaviors first and then adds its own tools.

• Target: M5StackChan CoreS3 running stack-chan firmware v1.1.0 (host 9.0.0+stackchan.1)

• Transport: Streamable HTTP on port 8080, POST /mcp with Authorization: Bearer <mcp.token>, GET /health

• No host firmware rebuild: the MOD is a JavaScript archive written to the xs flash partition

This turns your robot into a networked camera and microphone. Read SECURITY.md before installing it, especially if the robot lives in a room where people talk.

For coding agents

Start with AGENTS.md (also reachable as CLAUDE.md), then docs/architecture.md. llms.txt is a short index of the same material. The two constraints that catch people out: an uncaught exception reboots the device, and an oversized response takes the HTTP server down (~28 KB bodies are known good, ~56 KB is fatal). scripts/check.sh runs the linter without needing hardware — but only the robot can tell you whether a change actually works.

Related MCP server: stackchan-relay

Status

Working and used daily on one robot; every tool listed below has been exercised on real hardware. Only the M5StackChan CoreS3 target is tested — other Stack-chan targets may work but nobody has tried. Expect the rough edges documented under Device limits; most of them are firmware behaviour this MOD works around rather than bugs in the MOD.

Tools

It also replaces the face with one that draws a real smile for HAPPY (the stock mouth ignores emotion) and reports touches on the face area as touch events with screen coordinates.

Device limits worth knowing

Measured on an M5StackChan CoreS3 running v1.1.0:

• A large response body does not just fail — it takes the HTTP server down with it. Bodies up to ~28 KB are served repeatably and ~56 KB reliably kills it; the threshold between those is unmeasured. Photos are therefore budgeted against 28 KB on the wire (base64 included), colour uses a 3-3-2 palette rather than truecolour, base64 is written straight into the response buffer, and the server restarts its listener if the accept loop dies.

• A software restart leaves the display dead until the robot is physically power-cycled (hold power until off, then press again). A hardware reset — the bottom reset button, or esptool — is safe. The host's own MOD manager calls System.restart(), so installing a MOD through it has the same effect.

• Screen touch reaches a MOD only through the face: the firmware does not hand it over, and opening the touch controller directly is refused (duplicate address) because the UI already holds it. The custom face installed by this MOD reports touches on the face area instead.

• No A/B/C buttons exist on this hardware; only power. The CoreS3 target's virtual buttons are compiled out.

• say_message returns only after playback finishes, which can take tens of seconds for a long sentence.

Getting it running

Full path from a robot still in its box — firmware, preferences over BLE, the capture policy, build and install — is in docs/getting-started.md. It is all hands-on-the-device work.

If the robot already runs stack-chan v1.1.0 and has its mcp.token set, the short version is:

# Moddable SDK 9.0.0 exactly - the device rejects a MOD whose XS version does not match the host
MODDABLE=~/moddable STACKCHAN=~/stack-chan scripts/build.sh
STACKCHAN_PORT=/dev/cu.usbmodem1101 scripts/install.sh

The archive goes to the xs partition at 0xfa0000 (256 KB) and replaces whatever MOD was there. POST /mcp rejects every request until mcp.token is set, which is a BLE operation — see the guide.

Try it

STACKCHAN_HOST=192.168.1.20 STACKCHAN_TOKEN=... scripts/mcp.sh '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

Register it with Claude Code:

claude mcp add --scope user --transport http stackchan http://<robot-ip>:8080/mcp --header "Authorization: Bearer <token>"

Moving it to another network

Wi-Fi credentials are ordinary preferences, so Settings mode rewrites them like any other. Put the password in a gitignored .env at the repository root rather than on the command line, and reference it:

uv run --with bleak scripts/set-prefs.py wifi.ssid=<new-network> wifi.password=@env:WIFI_PASSWORD

Start the script first - it scans for up to 150 s - then press the bottom reset button, tap the gear on the 3 s splash, and leave the Settings screen open. Press reset again afterwards without touching the screen, so the MOD reloads.

The robot joins the new network on that boot. Its address changes, and nothing announces the new one: mDNS does not work on this firmware. Find it by its MAC in the ARP table and re-run the claude mcp add above, because a client registered against the old address simply stops working.

Driving it from a Cowork cloud session

A cloud session cannot reach a LAN address. The route that works without exposing the robot is the Claude Desktop bridge, whose proxy runs on your own machine — docs/remote-access.md has the config, the three things that cost time, and what it costs, including that the token crosses your LAN in cleartext on every call.

Watching for events

The robot cannot push: there is no SSE on this firmware, and MCP's notification mechanism would need a stream the HTTP layer cannot produce. So watching means asking repeatedly — and doing that from inside a conversation costs a tool call and a turn for every empty answer.

scripts/watch-events.py moves the waiting into a process instead. It blocks in wait_for_event (so latency is a round trip, not a poll interval), pulls the backlog by sequence number after each event so bursts and gaps are covered, and prints one line per event:

STACKCHAN_HOST=192.168.1.20 scripts/watch-events.py
2026-09-17T07:30:01 watching from seq=41
2026-09-17T07:30:12 touch-panel gesture=forwardSwipe position=-50 intensity=3 seq=42
2026-09-17T07:31:44 unreachable: robot not answering

It also prints state changes, because anything watching only for event lines cannot otherwise tell a quiet room from a dead script. The bearer token is never handled by it — scripts/mcp.sh fetches it from the OS keychain per call.

Checking it still works

A build proves the JavaScript compiles, and a tool call can return success while the head never moves. scripts/selftest.py runs the checks that catch that: it asserts consequences rather than status — a pose call that returns before its own duration has elapsed fails, a gaze that leaves the head pointing straight ahead fails, a photo whose encoded size is over the body budget fails — and it fails if the robot serves a tool no check exercises.

STACKCHAN_HOST=192.168.1.20 scripts/selftest.py          # read-only checks; changes nothing
STACKCHAN_HOST=192.168.1.20 scripts/selftest.py --all    # also visual, motion, audio and capture
scripts/selftest.py --list                               # what it would run, no robot needed

Only the read tier runs by default, because the others light LEDs, turn the head, make noise and use the camera. --json prints one machine-readable report. restart_robot is never run by any flag: on this hardware a software restart leaves the screen dead until someone power-cycles it by hand.

Collecting diagnostics

scripts/diagnose.py writes one read-only bundle — health, robot info, input capabilities, power registers, head pose, recent events, the tool list, HTTP conformance probes, and the local repo state with file hashes — into build/diagnostics-<timestamp>/:

STACKCHAN_HOST=192.168.1.20 scripts/diagnose.py

Every file is passed through a redaction step first, so the bundle is safe to attach to an issue. A robot that does not answer is a finding, not a crash: the bundle is still written, and says so.

Picking expressions by hand

Comparing two expressions by watching a timed sequence is miserable, so scripts/panel.py serves a small local page instead: a button per emotion at each intensity, fields for a message and for speech, and a live readout of what the robot has on screen.

STACKCHAN_HOST=192.168.1.20 scripts/panel.py      # opens a browser; --no-open to just print the URL

It binds to 127.0.0.1 only, and the browser never sees the bearer token — the page posts to the local server, which calls the robot through scripts/mcp.sh. A fixed action allowlist keeps it a review tool rather than a general robot-control surface: no camera, no microphone, no restart.

Reacting to events

scripts/react.py pairs input events with small routines, so the robot does something when nobody is talking to it:

scripts/react.py examples/rules.json --events-from examples/captured-events.txt   # offline, no robot
STACKCHAN_HOST=192.168.1.20 scripts/react.py examples/rules.json                  # dry run
STACKCHAN_HOST=192.168.1.20 scripts/react.py examples/rules.json --act            # for real

Rules are data: a rule matches event fields by equality and names actions from a fixed vocabulary, so there is no expression to evaluate, and a typo is refused at load rather than silently never matching. The vocabulary covers the face, the LEDs, the head, speech and tones — and deliberately excludes the camera and microphone, so a background process cannot become a capture trigger nobody sees. Nothing reaches the robot without --act, every rule has a minimum interval, and the run as a whole has an actions-per-minute ceiling.

Troubleshooting

The screen is blank but the robot answers MCP calls. Two different failures, and the backlight tells them apart. If the panel is dark, display initialization failed after a warm reset: pulse EN with STACKCHAN_PORT=<port> scripts/reset.sh 4 (scripts/install.sh sends two of those after flashing), and if it stays dark, power-cycle by hand — hold the power button until the robot powers off, then press it again. A pulse that works twice can fail on the next attempt and a later one can succeed where an earlier one did not, so more pulses is a reasonable response to a dark screen. If the panel is backlit but empty, the display has stopped rendering while the app runs on; the bottom reset button clears it. In both cases read the uptime from get_robot_info before looking for a crash: continuous uptime means nothing restarted, so there is no reboot to explain. Everything except the display keeps working meanwhile. See docs/device-notes.md.

esptool cannot connect (No serial data received). The app can wedge the USB peripheral. Start the flash and press the bottom reset button while esptool retries.

A tool call hangs and the robot stops answering. Something produced a response larger than the device can send, which takes the HTTP server down. The server restarts its listener automatically, backing off from 2 s to a minute and never giving up; if it cannot bind three times running it lights one LED purple, so a robot that looks fine but answers nothing is worth a glance before you assume the network. If it does not come back, reset the robot. Report it, since a tool should refuse an oversized result rather than attempt it.

Recordings sound almost silent. They are: the capture path is about 30 dB quiet on this hardware, for reasons not yet established (the ES7210's analog preamp has only 4.5 dB left to give, so it is not the cause). listen mic_listen reports levels corrected for the measured offset, and the recording tools apply software gain by default.

Development

scripts/check.sh   # lint, format, syntax, and the example rules; no robot needed
STACKCHAN_HOST=<robot-ip> scripts/selftest.py --all   # the checks that need hardware

Please read CONTRIBUTING.md — the short version is that a change should be tried on a real robot, nothing may throw out of a handler, and responses must stay small.

Operating it from an assistant

The MCP tools tell an assistant what it can do; they do not teach it how to behave with a physical robot. This repository also ships a Claude skill for that — which tool to reach for, the hazards, recovery when the robot stops responding, and how to report physical outcomes honestly rather than assuming a successful call meant something visibly happened.

It is packaged as a Claude Code plugin, served from this repository as its own marketplace:

/plugin marketplace add yaniv-golan/stackchan-mcp-mod
/plugin install stackchan-robot@stackchan-robot-marketplace

Or, from a local clone:

/plugin marketplace add ./stackchan-mcp-mod
/plugin install stackchan-robot@stackchan-robot-marketplace

The plugin installs the skill, not the robot. The tools themselves come from the MOD on the device, which is registered separately — the skill describes tools you will not have until you do. The plugin ships a command that does it for you:

/stackchan-robot:setup

It finds the robot by MAC in the ARP table (or takes an address), checks that GET /health answers before registering anything, and passes the token straight from the macOS Keychain, so it never appears in the conversation. By hand, it is:

claude mcp add --scope user --transport http stackchan http://<robot-ip>:8080/mcp --header "Authorization: Bearer <token>"

That step is not folded into the plugin because there is nothing correct to put in it. A plugin manifest is a static file shared by everyone who installs it, while the address is whatever DHCP handed your robot and the token is yours alone. Manifests do support ${VAR} expansion, but the first run of a fresh install would have both unset, and what happens then is not specified.

To use it without the plugin machinery, copy the skill folder straight in:

cp -r stackchan-robot/skills/stackchan-robot ~/.claude/skills/

Both the skill and its packaging were produced with dedicated tooling:

• skill-creator-plus drafted and iterated stackchan-robot/skills/stackchan-robot/SKILL.md.

• skill-packager generated the plugin and marketplace manifests from skill-packager.json.

Working notes

docs/device-notes.md is the bring-up log for this MOD: the build toolchain, the measured device limits, the firmware bugs found along the way (an oversized response kills the HTTP server; a software restart kills the display), the unexplained microphone deficit, and the approaches that did not work.

Related

• stack-chan — the firmware this runs on, and its MOD gallery

• Moddable SDK — the JavaScript engine and build tools

• Model Context Protocol — the protocol the robot speaks

License

Apache-2.0 — see LICENSE and NOTICE. mod/mcp-server-rich.js is derived from the stack-chan firmware (Apache-2.0); the file header lists what changed. Changes are recorded in CHANGELOG.md.