Skip to main content
Glama

BIP

A desk pet on the Waveshare ESP32-S3-LCD-1.47B: a 172×320 face, one RGB LED, and a USB link a computer can drive. Cursor, Claude, and Codex connect through the built-in MCP server.

The board is ESP32-S3R8, 16 MB flash, 8 MB PSRAM, ST7789, QMI8658 IMU, and an 18-pin header for add-ons. Tilting the board turns the face. The only onboard key is BOOT (GPIO0). Hold BOOT while resetting to enter the ROM download mode.

Flash

Firmware builds in Docker with ESP-IDF 5.5.1. The image is pulled on first use. Output stays in .deps/, which is not part of the source tree.

bash scripts/build.sh
pip install esptool          # or: python3 -m venv .venv && .venv/bin/pip install esptool
bash scripts/flash.sh /dev/cu.usbmodem1101

On Linux the port is usually /dev/ttyACM0.

USB is enough. Wi-Fi is optional and only for the UDP state packet:

python3 -m venv .venv
.venv/bin/pip install -e .
.venv/bin/python scripts/provision.py --port /dev/cu.usbmodem1101 --ssid "Your2.4GHzSsid"

The password is written to the board and is not kept in the repo. The pairing file is .deps/pet-device.json (mode 600).

Related MCP server: obsbot-mcp

MCP

Install the bridge once:

python3 -m venv .venv
.venv/bin/pip install -e .

This checkout already has a project config at .cursor/mcp.json. Restart Cursor and the esp32-pet server is available. Tools:

Tool

What it does

bip_set_state

Face, gaze, mouth, blink

bip_play_timeline

Timed face keyframes

bip_speak

Speak on the Mac with say and move the mouth with the audio

bip_get_status

Port, last face, recent button presses

Set PET_USB_PORT when more than one USB serial device is plugged in.

Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "esp32-pet": {
      "command": "/absolute/path/to/esp32-pet/.venv/bin/python",
      "args": ["-m", "bip_bridge.mcp_server"],
      "cwd": "/absolute/path/to/esp32-pet"
    }
  }
}

Codex (~/.codex/config.toml):

[mcp_servers.esp32-pet]
command = "/absolute/path/to/esp32-pet/.venv/bin/python"
args = ["-m", "bip_bridge.mcp_server"]
cwd = "/absolute/path/to/esp32-pet"

The server speaks MCP over stdio. It prints protocol messages on stdout and nothing else.

Voice and the local page

Voice needs ElevenLabs. Copy .env.example to .env and fill in the key and agent id, then:

.venv/bin/pip install -e ".[voice]"
pnpm live
pnpm live -- --ptt    # BOOT (GPIO0) is the talk key
pnpm demo             # macOS say, no API key
pnpm dev              # dashboard at http://127.0.0.1:5173

pnpm dev needs Node 22 and pnpm install. The page is a host preview of the face plus buttons for the onboard key and three add-on slots.

Pins

Use

GPIO

LCD MOSI, SCLK, CS, DC, RST, backlight

45, 40, 42, 41, 39, 46

RGB LED

38

IMU SDA, SCL

48, 47

TF card CMD, SCK, D0, D1, D2, D3

15, 14, 16, 18, 17, 21

USB D−, D+

19, 20

BOOT

0

Free for add-ons: GPIO1, GPIO2, GPIO4–GPIO13. GPIO14–GPIO18 and GPIO21 are the TF slot. Leave them if the card should keep working.

Add-ons

An add-on is one C file and one line in firmware/pet/main/addon.c. firmware/pet/addons/example_button.c is a button on GPIO2 that reports button 4 (cycles the LED). It is not compiled until you add it to SRCS in firmware/pet/main/CMakeLists.txt and register &example_button_addon.

pet_emit_button(n) reuses the face actions: 1 pet, 2 cycle mood, 3 agent, 4 LED mode. The onboard BOOT key is separate. It always cycles the mood and is reported to the host as button 1.

The host hears one line per press:

PET_BTN button=1 mood=HAPPY led=0 energy=95

A JSON line on the same serial port sets the face (mood, gazeX, gazeY, flags, energy, title, subtitle). STATUS answers with PET_STATUS and PET_IP.

Layout

firmware/pet/     ESP-IDF project for this board
bip_bridge/       USB controller and the MCP server
scripts/          build, flash, provision, voice
src/              optional local dashboard

.env, .venv, .deps, and node_modules stay on your machine.

Available Tools

4 tools
bip_get_statusA

Return BIP's current expression state, USB port, Wi-Fi IP, and recent physical button presses.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are supplied, so the description carries the full burden. 'Return' strongly implies a non-mutating read, and the field list tells the agent what data comes back, but there is no explicit statement that it has no side effects, no note on freshness or whether repeated polling is acceptable, and no cost/latency context. Useful but not a full behavioral picture for an unannotated tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence, front-loaded with the verb and immediately enumerating the returned data. Every clause carries information and there is no filler or repetition of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must describe return values, and it does so by naming the four categories of status data. It could go further on the shape of those values (e.g., what a button-press record looks like, IP format), but for a simple status read this is largely sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to disambiguate; the baseline for a no-param tool applies. Nothing in the description misrepresents or omits parameter behavior because none exists.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb ('Return') plus an enumerated set of returned fields: expression state, USB port, Wi-Fi IP, and recent button presses. That is far more concrete than a bare 'get status', and the read-only nature is implicit against siblings like bip_set_state and bip_speak. It stops short of explicitly naming the sibling it complements, which keeps it from a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: an agent can infer this is the read counterpart to bip_set_state, but the description never says when to call it (e.g., before mutating state to check current conditions) or what alternatives exist. For a zero-parameter getter the inference burden is low, so a 3 rather than a 2.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bip_play_timelineC

Play a sequence of time-coded facial animation keyframes on BIP (non-blocking firmware loop).

NameRequiredDescriptionDefault
keyframesYes

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden, and it only discloses that playback is non-blocking. It does not say whether the timeline replaces the current facial state, what happens if playback is already running, whether keyframe times are relative to invocation, or how errors and completion are surfaced.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One tightly written sentence with no filler, and the essential purpose is front-loaded. Its brevity is efficient but contributes to the missing detail elsewhere.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a structurally complex tool — a required array of keyframe objects — with no annotations and no output schema, so the description should explain keyframe field meaning and playback lifecycle. Instead it covers only the high-level intent and one execution trait, leaving an agent unable to construct a valid, meaningful keyframe list confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single keyframes parameter has 0% schema description coverage and a nested object with six undocumented subfields (at_ms, state, blink, gaze_x, gaze_y, speaking). The phrase "time-coded" implies at_ms semantics, but state values, gaze coordinate ranges, and units are left entirely unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ("Play") and a specific resource ("a sequence of time-coded facial animation keyframes on BIP"), so the agent understands this schedules an animation timeline rather than a one-shot state change. It stops short of explicitly contrasting itself with bip_set_state or bip_speak, so the differentiation is inferred rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to choose a timeline playback over bip_set_state or bip_speak, and no mention of prerequisites such as connecting to the device first. The parenthetical about a non-blocking firmware loop is the only hint about invocation context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bip_set_stateB

Set BIP's physical facial expression, eye gaze direction, speaking mouth animation, or trigger an expressive blink over USB Serial.

NameRequiredDescriptionDefault
blinkNoForce an immediate blink.
stateYesTarget expression state for BIP.
gaze_xNoHorizontal gaze (-100 left .. 100 right).
gaze_yNoVertical gaze (-100 up .. 100 down).
speakingNoWhether the mouth talking animation is active.

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does supply useful context that this drives physical hardware over USB Serial. It omits what happens when the device is disconnected, whether the set state persists until changed, and how gaze/speaking interact with the state enum, which matters for a stateful hardware mutation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that enumerates the controllable surfaces with zero filler. It is slightly list-like and could be tightened, but nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter hardware-control tool with no output schema, the description covers the what but not the interaction model: it never clarifies whether the 'blink' boolean and the 'blink' state value are equivalent or whether gaze/speaking flags override the state. That ambiguity is the main gap an agent would hit in practice.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so ranges, the state enum, and each field's meaning are already documented. The description merely summarizes those parameters and adds no precedence or combination rules beyond the schema, making the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Set') and the resource (BIP's facial expression, gaze, speaking animation, blink) and even names the transport (USB Serial). It never references the siblings, so an agent must infer that bip_speak is the text-to-speech counterpart while this only drives the mouth animation, but the operation itself is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the enumerated capabilities rather than stated: nothing says when to prefer this over bip_speak or bip_play_timeline, nor whether animations from bip_play_timeline take precedence. An agent can guess the intent, but the choice among siblings is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bip_speakC

Speak English text aloud on Mac and synchronize BIP's mouth, gaze, and blinks to the audio duration.

NameRequiredDescriptionDefault
textYesText for BIP to speak.
blinkNo
gaze_xNo
gaze_yNo
emotionNohappy

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It discloses that audio is played and animation is synchronized to the audio duration, but never states whether the call blocks until speech completes, whether it can be interrupted, what happens if audio fails, or what platform prerequisites exist (English only, Mac only is mentioned but not why).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that leads with the primary action and appends the animation behavior. No filler, though the two ideas (speech and rig synchronization) are compressed into one line without structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-parameter mutation-style tool with no annotations, no output schema, and 80% of parameters undocumented, the description leaves too much unknown. The most decision-relevant fact for an agent — whether the call waits for the audio to finish — is absent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 20%, so the description needs to compensate. It does imply that gaze and blink are driven by this tool, giving indirect meaning to blink/gaze_x/gaze_y, but it never explains the coordinate ranges, units, or defaults for the gaze integers, nor what the emotion choices do to the delivery.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a concrete verb and resource ("Speak English text aloud on Mac") plus the distinctive side effect of lip/gaze/blink synchronization, which clearly separates it from bip_get_status or bip_set_state. It stops short of naming a sibling or stating how it differs from bip_play_timeline, which could also drive animation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use or when-not-to-use guidance and no mention of alternatives among bip_set_state, bip_play_timeline, or bip_get_status. The agent must infer that this is the tool for voiced dialogue rather than for silent posing or timeline playback.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv0.1.0
    • First observedbip_get_status
    • First observedbip_play_timeline
    • First observedbip_set_state
    • First observedbip_speak

TDQS

B3.4/5.0

Scored across 4 tools

Disambiguation4/5

Each tool targets a distinguishable action: setting a single state, playing a keyframe timeline, reading status, and speaking text. There is mild overlap since bip_set_state also controls speaking mouth animation while bip_speak synchronizes mouth/gaze/blink, so an agent might hesitate over which to use for mouth motion, but the descriptions clarify the intended split.

Naming Consistency5/5

All four tools use a consistent snake_case pattern with the shared 'bip_' prefix and a verb-led name (set_state, play_timeline, get_status, speak). The convention is predictable and readable across the whole surface.

Tool Count4/5

Four tools is on the lean side but each covers a distinct facet (state, sequence playback, status, speech) of driving a physical pet device. It is slightly thin — no separate stop/reset tool — but nothing feels redundant.

Completeness3/5

The core actions (set expression, play animation, speak, read status) are covered, but there is no way to stop or interrupt a running timeline, reset to a known idle state, or enumerate valid expressions/gazes. These gaps will force agents to improvise when recovering from an animation or discovering valid state values.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    A
    maintenance
    Enables MCP-compatible agents to control a desktop mascot by switching its appearance and voice, showing speech-bubble dialogue, and managing state such as affinity.
    12 npm
    1
    MIT
  • A
    license
    B
    quality
    C
    maintenance
    Enables natural-language control of DK-META2 devices through local BLE for vibration, linear extension, and rotation. It also supports scanning, connecting, status checks, and safe stop/disconnect operations without cloud access.
    8
    1
    MIT