# comfyui-local-mcp

Drive your **local ComfyUI** from MCP clients (Hermes, Claude Code, Cursor, any MCP agent) —
with ready-made **FLUX.2 Klein** text-to-image workflows tuned for **Apple Silicon (MLX + MPS)**.

Built and tested on: Mac mini M4 · 24GB unified memory · macOS 26 · ComfyUI 0.33 · mflux 0.19

## Features

- 🖼️ **generate_image** — text-to-image with auto model discovery:
  - `mflux:<name>` — FLUX.2 Klein 4B / 9B, 4-bit MLX (fast, low memory)
  - `bf16` — ComfyUI-native bf16 (MPS, best quality)
  - `auto` — picks the best model automatically (9B preferred)
- 📦 **list_models** — list installed models (auto-scans `models/mflux/`, add models without code changes)
- 🗂️ **list_outputs** — recent generated images
- 🩺 **system_status** — one-call diagnosis (ComfyUI service, models, disk)

## Quickstart

```bash
# 1. Install the server files into your ComfyUI + register the hermes MCP
./install.sh                # optional; also fine to just copy files

# 2. Register the MCP server manually (or via install.sh) — e.g. in ~/.hermes/config.yaml:
#   mcp_servers:
#     comfyui-local:
#       command: /path/to/python-with-mcp
#       args: [/path/to/comfyui-local-mcp/mcp_comfyui_server.py]
#       enabled: true

# 3. Test the server (stdio MCP handshake)
python mcp_comfyui_server.py

# 4. Ask your agent: "generate an image of ..." or "check my ComfyUI status"
```

## Requirements

- macOS with Apple Silicon (MLX / Metal)
- [ComfyUI](https://github.com/Comfy-Org/ComfyUI) running locally (default `127.0.0.1:8188`)
- Python env with `mcp` package (the server itself only needs stdlib + `mcp`)
- Optional: [mflux](https://github.com/filipstrand/mflux) + `ComfyUI-mflux-AnyModel` custom node for the MLX path
- Optional: FLUX.2 Klein bf16 weights + Qwen3-4B text encoder for the bf16 path

## Model setup (Apple Silicon, 24GB friendly)

| Model | Path (under `~/ComfyUI/models/`) | Notes |
|-------|----------------------------------|-------|
| FLUX.2 Klein 4B · 4bit MLX | `mflux/FLUX.2-Klein-4B-4bit/` | ~4.3GB, ~12.7s/step @1024² |
| FLUX.2 Klein 9B · 4bit MLX | `mflux/FLUX.2-Klein-9B-4bit/` | ~9.5GB, higher quality |
| FLUX.2 Klein 4B bf16 | `diffusion_models/flux-2-klein-4b-bf16.safetensors` | MPS path, best quality |
| Qwen3-4B text encoder | `text_encoders/qwen_3_4b.safetensors` | required by bf16 path |
| FLUX.2 VAE | `vae/flux2-vae.safetensors` | |

Download mflux MLX models with HF cache and symlink them into `models/mflux/`
(see [HF caching / symlinks](#hf-caching-and-symlinks)) — the server auto-discovers them.

### Apple Silicon notes

- **fp8 weights don't work on MPS** — convert to bf16 first (a CPU-side dequantize).
  `flux-2-klein-4b-bf16.safetensors` is a bf16 conversion of the official fp8 checkpoint.
- **HF downloads can stall in some networks** — `export HF_HUB_DISABLE_XET=1` fixes it.
- Performance (M4 10-core GPU, 1024×1024): 4B-4bit ~12.7s/step (peak ~11GB), bf16 ~12.1s/step (peak ~15GB).

## HF caching and symlinks

HuggingFace `snapshot_download` stores real data in `~/.cache/huggingface/hub/models--<org>--<name>/blobs/`
and exposes symlinks under `snapshots/<rev>/`. This project keeps **symlinks** in
`models/mflux/<Model-Name>/` pointing at those snapshots — zero duplicate disk usage,
and `list_models` picks them up automatically.

## Troubleshooting

```bash
./check_system.sh        # one-shot diagnosis
tail -50 ~/comfyui_run.log
```

- ComfyUI not responding? Make sure it runs on `127.0.0.1:8188` (kill stale instances first).
- Model missing from `list_models`? Symlink (or copy) it into `models/mflux/` with a `transformer/` subdir.
- MCP not visible in the agent? Restart the agent session after editing `config.yaml`.

## License

MIT