MCP-ImageGen
by halochamp
README.md
# MCP-ImageGen
MCP-ImageGen is a small standalone MCP stdio server that generates and edits
pictures **locally on Apple Silicon**. It has no inner LLM, no browser and no
remote image API: the only model is Qwen Image 2.1 (4-bit, run with
[mflux](https://github.com/filipstrand/mflux)) in a low-RAM recipe whose whole
process tree stays around **5.8 GiB** for generation and **6.7 GiB** for editing,
so it can run next to a large local LLM.
- Two MCP tools: `generate_image` (text to picture) and `edit_image` (change an
existing picture by instruction).
- One image per call, one job at a time per machine (host lock).
- Live progress through MCP progress notifications.
- A small web UI (`ui_server.py`) for using it without an agent.
For Thai operating guidance and the measurements, see [GUIDE_TH.md](GUIDE_TH.md).
If you are an agent working inside this repository, start with [AGENTS.md](AGENTS.md).
## Requirements
- Apple Silicon Mac (MLX), Python 3.11.
- About 32 GB of free disk for the weights: the 10.5 GB download plus the converted
generation copy (9.6 GB) and the converted edit copy (10.6 GB). Skip the edit copy
if you only need `generate_image`.
- Free RAM: 6 to 7 GiB while a job runs.
## Setup
1. Create the environment. mflux is installed last with `--no-deps` because it
asks for newer `torch`/`pillow` than the versions this project was measured with;
the set below is the one that was run:
```bash
python3.11 -m venv .venv
.venv/bin/python -m pip install mlx==0.32.0 torch==2.12.0 transformers==5.14.1 \
pillow==12.2.0 tokenizers==0.22.2 numpy==2.4.6 safetensors==0.8.0 \
fastapi==0.136.3 uvicorn==0.48.0 huggingface_hub==1.16.4 mcp==1.26.0 \
toml==0.10.2 sentencepiece==0.2.1 requests==2.33.0 urllib3==2.7.0 \
opencv-python==4.13.0.92 matplotlib==3.10.9 protobuf==6.33.6
.venv/bin/python -m pip install --no-deps -r requirements.txt # mflux, platformdirs, piexif
```
2. Download the source checkpoint (about 10.5 GB). `HF_HUB_DISABLE_XET=1` keeps
the transfer from buffering in RAM on a slow link:
```bash
HF_HUB_DISABLE_XET=1 .venv/bin/hf download mlx-community/Qwen-Image-2.1-MLX-4bit \
--local-dir ~/Models/ImageStudio/Qwen-Image-2.1-MLX-4bit
```
3. Convert it to the layout mflux loads (once for generation, once for editing).
The generation conversion only rewrites safetensors headers and takes seconds;
the edit one loads about 9 GB through MLX, so run it with free memory:
```bash
.venv/bin/python tools/convert_weights.py # ...-mflux
.venv/bin/python tools/convert_weights.py --edit # ...-mflux-edit
```
`~/Models/ImageStudio/` is the default location (a historical folder name).
Override it with `MCP_IMAGEGEN_MODEL_DIR` and `MCP_IMAGEGEN_EDIT_MODEL_DIR`, and
pass `--src` / `--dst-root` to the converter. Keep the downloaded folder: it is
the conversion source.
The runtime never downloads anything. It pins the byte size of every weight
file in `image_runtime.py` and answers `model_not_ready` when a file is missing
or changed. The Qwen Image weights have their own license: read the model card
before using the pictures.
4. Register the server in your MCP client, for example:
```json
{
"mcpServers": {
"mcp-imagegen": {
"command": "/absolute/path/to/MCP-ImageGen/.venv/bin/python",
"args": ["/absolute/path/to/MCP-ImageGen/server.py"]
}
}
}
```
A call takes about 3 to 4 minutes, so raise the client's per-tool timeout well
above its default (a client that gives up after 60 seconds fails every call).
Start the stdio server by hand with `.venv/bin/python server.py`. Start the web UI
with `./run_ui.sh` (port 7860) or by double-clicking `ImageGen_UI.command`.
## MCP surface
### `generate_image`
Renders ONE image per call:
```json
{
"prompt": "A quiet mountain lake at sunrise",
"aspect_ratio": "16:9",
"steps": 25,
"seed": 42
}
```
| field | values | default |
|---|---|---|
| `aspect_ratio` | `1:1` 1024×1024, `4:3` 1152×864, `16:9` 1280×720, `9:16` 720×1280 | `1:1` |
| `steps` | 25–40 (fewer steps under-converge at CFG) | 25 |
| `seed` | 0–2147483647, random when omitted | random |
The server owns the model path, guidance (3.0), the negative prompt, the decode
policy and the output path. Generated PNGs are written to `work/generated/`; the
result returns the absolute `output_path` plus a `next_action`. That hint asks the
caller to deliver the picture with a `send_file` tool: a client without such a tool
can ignore it and use `output_path`.
A call takes about 3 minutes (25 steps, CFG on). While it runs the server sends MCP
progress notifications (model load, prompt encode, `n/25` steps, decode, save) and a
heartbeat with the elapsed seconds when the worker is silent. The status texts are
Thai (`กำลังสร้างภาพ n/25`, …). The runtime rejects prompts over 512 tokens, takes an
advisory host-wide lock before a worker starts (a second caller waits up to 15
minutes, then gets a structured `busy` error), removes outputs after seven days and
caps the output directory at 1 GiB.
### `edit_image`
Changes an existing picture by an English instruction (colour, clothing, remove an
object or a person, fix an anatomy flaw) and returns a new PNG; the source is never
modified.
```json
{
"prompt": "Change the shirt to red. Keep everything else unchanged.",
"image_path": "/absolute/path/to/picture.png",
"steps": 25,
"seed": 42
}
```
`image_path` is a PNG/JPEG/WebP (<= 25 MB, <= 50 MP, aspect ratio within 4:1): a
file you have, or the `output_path` of an earlier `generate_image`/`edit_image`
result (edits can be chained). The server applies the EXIF rotation, flattens
transparency on white, drops metadata and resizes to the edit resolution (768 px
area, aspect ratio kept) before the worker sees it. The output is about 768 px and
takes 3 to 4 minutes. Describe only the change and end with what must stay.
Edit memory (process-tree footprint, CFG on): 6.7 GiB at 768 px; 1024 px would be
7.9 GiB and stock mflux 14.7 GiB. Three changes versus stock: the reference image
is VAE-encoded in 256 px tiles (a full pass adds 3.5 GiB), the K/V cache of the
reference tokens is off (adds 4.4 GiB at 768 px, 8 GiB at 1024 px) and the CFG
negative embeddings are evaluated before the text encoder is freed.
## How the low-RAM recipe works
The worker calls mflux's native `QwenImage21` API (never the CLI) with offline
flags set. Measured at 1024×1024, the footprint is 5.8 GiB versus 23.5 GiB for
stock mflux, with the same images:
1. evaluate the prompt embeddings before mflux frees the text encoder (lazy
embeddings otherwise keep the 4 GiB encoder alive next to the 4 GiB transformer),
2. decode the VAE in 256 px tiles,
3. run the decode in its OWN process: `worker.py` is a light orchestrator that runs
`--denoise` (encode + denoise, saves the latents) and then `--decode` (VAE only).
Freeing the transformer inside the process that used it is not reliable under
memory pressure (MLX kept about 2.4 GiB alive after every Python reference was
gone, a decode peak of 7.3 GiB); a process exit returns it for certain.
Per-process footprint: denoise 5.75 GiB (the peak), decode about 4.4 GiB, stable
over 6 cold/warm-cache runs.
Measured on a 48 GB Apple Silicon Mac (2026-11-05, 4:3, 25 steps): worker wall time
159 s, max RSS 4.96 GB; stock mflux at 1024×1024 and the same seed took 171–172 s
versus 181–187 s for the low-RAM recipe (+4 to 9 %).
## Tests
The normal suite uses mocked workers and skips the live model test:
```bash
PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -m unittest discover -s tests -v
.venv/bin/python -m py_compile server.py image_runtime.py worker.py
```
Opt into the real MCP stdio smoke (one real image, about 3 minutes, also checks that
the progress notifications arrive) only when the GPU is free:
```bash
MCP_IMAGEGEN_LIVE=1 PYTHONDONTWRITEBYTECODE=1 \
.venv/bin/python -m unittest tests.test_live_smoke -v
```
`tests.test_live_matrix` (`MCP_IMAGEGEN_LIVE_MATRIX=1`) renders one image per aspect
ratio sequentially (about 12 minutes).
## Privacy
Everything runs on your machine. Generated pictures stay under `work/` (git-ignored)
and are never committed; `.gitignore` also blocks common picture file types as a
safety net.
## License
MIT — see [LICENSE](LICENSE). The Qwen Image weights are not part of this repository and have their own license (see the model card).
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo recent community issues