# BIP

A desk pet on the [Waveshare ESP32-S3-LCD-1.47B](https://www.waveshare.com/product/esp32-s3-lcd-1.47b.htm): a 172×320 face, one RGB LED, and a USB link a computer can drive. Cursor, Claude, and Codex connect through the built-in MCP server.

The board is ESP32-S3R8, 16 MB flash, 8 MB PSRAM, ST7789, QMI8658 IMU, and an 18-pin header for add-ons. Tilting the board turns the face. The only onboard key is BOOT (GPIO0). Hold BOOT while resetting to enter the ROM download mode.

## Flash

Firmware builds in Docker with ESP-IDF 5.5.1. The image is pulled on first use. Output stays in `.deps/`, which is not part of the source tree.

```bash
bash scripts/build.sh
pip install esptool          # or: python3 -m venv .venv && .venv/bin/pip install esptool
bash scripts/flash.sh /dev/cu.usbmodem1101
```

On Linux the port is usually `/dev/ttyACM0`.

USB is enough. Wi-Fi is optional and only for the UDP state packet:

```bash
python3 -m venv .venv
.venv/bin/pip install -e .
.venv/bin/python scripts/provision.py --port /dev/cu.usbmodem1101 --ssid "Your2.4GHzSsid"
```

The password is written to the board and is not kept in the repo. The pairing file is `.deps/pet-device.json` (mode 600).

## MCP

Install the bridge once:

```bash
python3 -m venv .venv
.venv/bin/pip install -e .
```

This checkout already has a project config at `.cursor/mcp.json`. Restart Cursor and the `esp32-pet` server is available. Tools:

| Tool | What it does |
| --- | --- |
| `bip_set_state` | Face, gaze, mouth, blink |
| `bip_play_timeline` | Timed face keyframes |
| `bip_speak` | Speak on the Mac with `say` and move the mouth with the audio |
| `bip_get_status` | Port, last face, recent button presses |

Set `PET_USB_PORT` when more than one USB serial device is plugged in.

Claude Desktop (`~/Library/Application Support/Claude/claude_desktop_config.json`):

```json
{
  "mcpServers": {
    "esp32-pet": {
      "command": "/absolute/path/to/esp32-pet/.venv/bin/python",
      "args": ["-m", "bip_bridge.mcp_server"],
      "cwd": "/absolute/path/to/esp32-pet"
    }
  }
}
```

Codex (`~/.codex/config.toml`):

```toml
[mcp_servers.esp32-pet]
command = "/absolute/path/to/esp32-pet/.venv/bin/python"
args = ["-m", "bip_bridge.mcp_server"]
cwd = "/absolute/path/to/esp32-pet"
```

The server speaks MCP over stdio. It prints protocol messages on stdout and nothing else.

## Voice and the local page

Voice needs ElevenLabs. Copy `.env.example` to `.env` and fill in the key and agent id, then:

```bash
.venv/bin/pip install -e ".[voice]"
pnpm live
pnpm live -- --ptt    # BOOT (GPIO0) is the talk key
pnpm demo             # macOS say, no API key
pnpm dev              # dashboard at http://127.0.0.1:5173
```

`pnpm dev` needs Node 22 and `pnpm install`. The page is a host preview of the face plus buttons for the onboard key and three add-on slots.

## Pins

| Use | GPIO |
| --- | --- |
| LCD MOSI, SCLK, CS, DC, RST, backlight | 45, 40, 42, 41, 39, 46 |
| RGB LED | 38 |
| IMU SDA, SCL | 48, 47 |
| TF card CMD, SCK, D0, D1, D2, D3 | 15, 14, 16, 18, 17, 21 |
| USB D−, D+ | 19, 20 |
| BOOT | 0 |

Free for add-ons: GPIO1, GPIO2, GPIO4–GPIO13. GPIO14–GPIO18 and GPIO21 are the TF slot. Leave them if the card should keep working.

## Add-ons

An add-on is one C file and one line in `firmware/pet/main/addon.c`. `firmware/pet/addons/example_button.c` is a button on GPIO2 that reports button 4 (cycles the LED). It is not compiled until you add it to `SRCS` in `firmware/pet/main/CMakeLists.txt` and register `&example_button_addon`.

`pet_emit_button(n)` reuses the face actions: 1 pet, 2 cycle mood, 3 agent, 4 LED mode. The onboard BOOT key is separate. It always cycles the mood and is reported to the host as button 1.

The host hears one line per press:

```text
PET_BTN button=1 mood=HAPPY led=0 energy=95
```

A JSON line on the same serial port sets the face (`mood`, `gazeX`, `gazeY`, `flags`, `energy`, `title`, `subtitle`). `STATUS` answers with `PET_STATUS` and `PET_IP`.

## Layout

```text
firmware/pet/     ESP-IDF project for this board
bip_bridge/       USB controller and the MCP server
scripts/          build, flash, provision, voice
src/              optional local dashboard
```

`.env`, `.venv`, `.deps`, and `node_modules` stay on your machine.