Skip to main content
Glama

mcp-vroid-mac

Drive the native macOS VRoid Studio from any MCP client: screenshot the window, locate widgets with OCR, and inject real pointer/keyboard events with Quartz. No Wine, no plugins, no Linux.

This is a downstream macOS port of mcp-vroid by Nuri Hodges (MIT). The VRoid-specific logic — OCR locating, the UI map, the parameter/colour/export flows — is reused; the platform layer is selected behind driver/backend.py. The Linux/Hyprland implementation is kept in-tree and documented in README.upstream.md.

Piece

Upstream (Linux)

This port (macOS)

window

hyprctl

CGWindowListCopyWindowInfo + NSRunningApplication

capture

grim

screencapture -l <CGWindowID>

input

wlroots virtual pointer + XTEST

Quartz CGEvent

keyboard

XTEST keysyms

CGEventCreateKeyboardEvent / Unicode events

OCR

tesseract

tesseract (unchanged)

workspaces

Hyprland workspace 9

none (no-ops)

Requirements

  • macOS with the native VRoid Studio app (/Applications/VRoidStudio.app, bundle id net.pixiv.vroid.macosx).

  • Python 3.11+ and uv.

  • tesseract (brew install tesseract).

  • Two macOS permissions for whichever app hosts this MCP server (Terminal, iTerm, Claude Desktop, …):

    1. Screen Recording — System Settings → Privacy & Security → Screen & System Audio Recording.

    2. Accessibility — System Settings → Privacy & Security → Accessibility. Restart the host app after granting them. vroid_status reports both.

Related MCP server: mcp-vroid

Install

git clone https://github.com/wilsontr/mcp-vroid-mac ~/mcp-vroid-mac
cd ~/mcp-vroid-mac
uv sync

Register it with Claude Code:

claude mcp add --scope user vroid-mac -- \
  uv run --no-editable --directory /path/to/mcp-vroid-mac mcp-vroid-mac

Register it with pi (writes ~/.pi/agent/mcp.json):

pi mcp add vroid -- uv run --no-editable --directory /path/to/mcp-vroid-mac mcp-vroid-mac
pi mcp list        # expect: vroid: connected, 18 tools

Register it with DSH by adding an insert row to ~/.dsh/profiles/<profile>/cordis.patch.yml (the profile in use — desktop for the desktop app, which also serves the web GUI):

- insert:
    - id: mcp-vroid
      name: "@deepseek-ai/dsh-mcp-client"
      config:
        serverName: vroid
        transport: stdio
        command: /Users/you/.local/bin/uv
        args:
          - run
          - --no-editable
          - --directory
          - /path/to/mcp-vroid-mac
          - mcp-vroid-mac
        cwd: /path/to/mcp-vroid-mac
        env:
          HOME: /Users/you
          PATH: /Users/you/.local/bin:/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin

Validate without restarting: dsh --profile desktop --dump-config | grep -A12 mcp-vroid (a raw - id: … row at top level would be an override and logs "patch: entry … not found"; MCP servers are new rows, so they must be inside insert:). DSH's HMR picks it up live and the tools appear as mcp__vroid__vroid_*.

…or with any client that takes an mcpServers block:

{
  "mcpServers": {
    "vroid": {
      "command": "uv",
      "args": ["run", "--no-editable", "--directory", "/path/to/mcp-vroid-mac", "mcp-vroid-mac"]
    }
  }
}

Why --no-editable? Python 3.13 ignores .pth files whose names start with _, which is what uv 0.5.x writes for editable installs (_editable_impl_*.pth). Without the flag the server dies at import with ModuleNotFoundError: No module named 'mcp_vroid'. --no-editable installs the project as a regular wheel instead; newer uv releases may not need it.

Then call vroid_status first: it tells you whether VRoid is running, whether it is focused, and whether both permissions are in place.

Tools

The same tool set as the original server, 18 tools in four groups.

Lifecycle — vroid_launch (opens/brings the app forward), vroid_status (window, focus, permissions, helpers), vroid_release (no-op on macOS).

Seeing — vroid_screenshot, vroid_find_text, vroid_find_button, vroid_current_screen. Everything works in image pixels of the window capture, so coordinates from find_* go straight into click.

Acting (raw) — vroid_click, vroid_drag, vroid_scroll, vroid_type, vroid_key. ctrl+… shortcuts are translated to Command (⌘), because VRoid's macOS shortcuts use ⌘.

Acting (flows) — vroid_new_character, vroid_open_tab, vroid_set_slider, vroid_set_color, vroid_export_vrm, vroid_save_project.

How it works

screencapture -l <window> ──► PNG ──► tesseract / cv2 ──► (x,y) ──► CGEvent
      ▲                                                              │
      └────────────────────  screenshot again  ◄────────────────────┘

The loop is see → locate → act → see again:

  1. vroid_launch() → vroid_screenshot() (the image goes to the model)

  2. vroid_find_text("Export") / vroid_find_button() for coordinates

  3. vroid_click(x, y) with coordinates from a fresh capture

  4. vroid_screenshot() again to confirm what happened

Rules of thumb (inherited from the original, learned the hard way): read the whole frame rather than a crop so modals are not missed; judge change by a UI strip rather than the dithering 3D viewport; prefer typing exact values into numeric boxes over dragging sliders; a grey “primary” button means a required field is still empty.

Calibration and caveats

The VRoid-specific anchors and OCR needles in driver/actions.py were calibrated on VRoid Studio 2.14.0, English UI, at 2560×1440. The native macOS app here is 2.16.0, so some flows may need re-measuring:

  • vroid_new_character, vroid_open_tab, vroid_set_slider, vroid_set_color use fractional anchors of the window, which survive resolution changes but not a UI reflow.

  • vroid_export_vrm was written against Wine's file dialog. On macOS it now drives the native save panel (⌘⇧G → folder → filename → Return), but this path is the least tested part of the port.

  • The fractional anchors were measured on a borderless fullscreen window. The macOS window has a title bar and any size, so expect the anchor-based flows above to miss until they are re-measured on macOS.

  • The save panel is a separate floating window (titled "Save"), so a plain window capture does not show it. Use vroid_screenshot(whole_screen=true) to see it. The save flows find and capture it themselves.

  • ⌘S / ⌘⇧S are ignored while a parameter value box is still selected, even after Return. Click somewhere inert first (Save As does this itself).

  • Clicks, drags and scrolls refuse when another window (an always-on-top panel, another app) covers the target point; move it out of the way.

  • Keys: ctrl/cmd both mean ⌘, Control_L is the real Control key, Delete is forward-delete and BackSpace is ⌫.

  • If VRoid will not come to the front (macOS 14+ can ignore activation from a background process), allow the hosting app under Automation for VRoid Studio, or click VRoid's window once.

The raw tools (screenshot, find_text, find_button, click, type, key, drag, scroll) are the reliable foundation — the model can always fall back to them, look at the screen, and drive the app step by step.

Security

This server injects real mouse/keyboard events into your live desktop session and takes screenshots of it. Run it attended, on a session you are watching. vroid_launch(restart=true) quits VRoid Studio and loses unsaved work. Captures are written unencrypted to ~/Library/Application Support/mcp-vroid/captures (override with MCP_VROID_CAPTURES).

How this relates to upstream

The platform layer sits behind driver/backend.py:

driver/backend.py      picks the platform modules
driver/window_mac.py   CGWindowList + NSRunningApplication
driver/capture_mac.py  screencapture
driver/input_mac.py    Quartz CGEvent
driver/window.py       upstream Linux: Hyprland
driver/capture.py      upstream Linux: grim
driver/input.py        upstream Linux: virtual pointer + XTEST
driver/actions.py      shared VRoid flows
driver/locate.py       shared OCR / colour matching
server.py              shared MCP tools

So the macOS diff is additive: native/, the Linux modules and README.upstream.md stay as upstream shipped them, and the tool flows are shared.

To pull upstream changes:

git remote add upstream https://github.com/nhodges/mcp-vroid
git fetch upstream
git merge upstream/main      # conflicts should be confined to actions/server

If upstream merges its Windows backend seam (PRs #1/#2, which also introduce driver/backend.py), this port can be rebased onto that seam nearly as-is.

Licence

MIT, same as the original. Upstream is Copyright (c) 2026 Nuri Hodges; see LICENSE. VRoid Studio is a product of pixiv Inc.; this project is unaffiliated and simply drives the app's UI.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables MCP clients to control macOS via accessibility and screen recording, providing tools to list apps, observe UI, click, type, press keys, and scroll.
      MIT
    • A
      license
      A
      quality
      D
      maintenance
      Enables MCP clients to drive VRoid Studio's GUI by launching the app, capturing screenshots, locating UI elements via OCR/color, and simulating clicks/typing to adjust parameters and export .vrm files.
      18
      9
      MIT
    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables MCP clients to inspect and operate native macOS apps through the Accessibility tree with safety-gated semantic actions, stale-state rejection, and outcome verification. Supports tools for app state reading, element finding, clicking, typing, form filling, window capture, and guarded workflows like menu selection and Save As.
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables controlling VRoid Studio through desktop GUI automation, letting an MCP client launch the app, capture its window, locate controls via OCR, edit character parameters, and run save/export flows on macOS and Linux/Hyprland.
      MIT