Skip to main content
Glama
README.md
# comfyui-mcp-server

An MCP server that gives an agent direct control of a running
[ComfyUI](https://github.com/comfyanonymous/ComfyUI) instance: submit workflows,
run randomised batch discovery, shape LoRA stacks, and read back the provenance
of every image produced.

Built for **iterative aesthetic discovery** -- generate wide, curate narrow, and
keep a verifiable record of the parameters that produced the survivors.

> **Not affiliated with** `joenorton/comfyui-mcp-server` or any other project of
> a similar name. This is an independent implementation (`@retiform-labs`), and
> its tool surface is specific to the workflow templates it ships.

## What it does

Thirteen tools, split into four groups:

**Lifecycle** -- `system_check` · `start_comfyui` · `stop_comfyui` · `get_comfyui_status`

**Generation** -- `generate_image` · `generate_ipadapter` · `get_job_status`

**Batch** -- `batch_generate` · `batch_generate_ipadapter`

**Hardware** -- `list_models` · `download_model` · `get_vram_usage` · `emergency_cleanup`

### Randomised batch discovery

The point of the batch tools is *divergence*, not repetition. Pass a `randomize`
spec and each job in the batch draws its own parameters:

```jsonc
{
  "randomize": {
    "seed":      "random",            // fresh seed per job, deduplicated
    "sampler":   "random",            // from a curated safe pool
    "cfg_scale": [5.0, 9.0],          // uniform in range
    "loras":     {"mode": "auto", "count": [1, 3], "weight": [0.25, 0.7]}
  }
}
```

Ranges are **2-element lists** `[min, max]`. A dict like `{"min": 1, "max": 5}` is
rejected with an error rather than silently sent to the workflow as a literal --
that mistake is easy to make and produces a batch where nothing varies.

### LoRA stack shape matters more than the sum

Two LoRAs at equal weight average two aesthetics into a muddle. A stack
generally wants **one dominant adapter and one supporting it**:

- dominant: `0.55` -- `0.77`
- supporting: `0.25` -- `0.40`
- optional style pin: `~0.45`

A total weight around `1.3` is normal and is **not** a ceiling -- well-shaped
stacks run above it and produce the best results.


## Tool reference

Every tool returns JSON. `generate_*` and `batch_*` are **asynchronous**: they
queue work and return a job id immediately, because a diffusion run takes far
longer than any sane request timeout. Poll `get_job_status` for the result.

### Lifecycle

#### `system_check()`

Reports GPU name, driver version, total and free VRAM, and whether ComfyUI is
responding. Reads `nvidia-smi`, so it sees the whole GPU -- including memory
held by ComfyUI, which a torch call inside this process never could.

```jsonc
// no arguments
{"gpu": "NVIDIA GeForce RTX 4090", "vram_total_mb": 24564,
 "vram_free_mb": 3703, "comfyui_running": true, "comfyui_version": "0.x"}
```

#### `start_comfyui(lowvram=True, fp16=True, port=8188)`

Launches ComfyUI if it is not already running. `lowvram` is the safe default for
cards under ~8 GB.

```jsonc
{"success": true, "pid": 41233, "port": 8188, "already_running": false}
```

#### `stop_comfyui(force=False)`

Stops the ComfyUI process. `force=True` sends SIGKILL when a normal shutdown
hangs.

#### `get_comfyui_status()`

Lightweight liveness check -- cheaper than `system_check` because it never shells
out to the driver.

```jsonc
{"running": true, "port": 8188, "queue_remaining": 0}
```

### Generation

#### `generate_image(...)`

Text-to-image through the packaged SDXL workflow.

| Parameter | Default | Notes |
|---|---|---|
| `prompt` | *required* | What to generate |
| `negative_prompt` | `"text, watermark, blurry, low quality"` | What to avoid |
| `negative_embedding` | `None` | A textual-inversion embedding name |
| `width` / `height` | `1024` / `1024` | |
| `steps` | `20` | |
| `cfg_scale` | `7.0` | |
| `sampler` | `"euler_ancestral"` | |
| `scheduler` | `"normal"` | |
| `lora` | `None` | One LoRA filename from `models/loras/` |
| `lora_weight` | `0.8` | |
| `loras` | `None` | Several at once: `[{"name": ..., "weight": ...}]` |
| `seed` | `None` | Omit for a random seed |
| `output_format` | `"png"` | PNG carries provenance metadata |

```jsonc
{"prompt": "a geometric fox mark, single flat ink colour, high contrast",
 "width": 1024, "height": 1024, "steps": 30, "seed": 1777732588}
// -> {"job_id": "a3f9...", "status": "queued", "params": {...}}
```

#### `generate_ipadapter(...)`

Reference-guided generation. Feeds 1--4 images into the model to steer style,
composition or content.

| Parameter | Default | Notes |
|---|---|---|
| `prompt` | *required* | |
| `reference_images` | *required* | 1--4 filenames from `comfyui/input/` |
| `reference_weight` | `0.6` | How hard the references pull |
| `weight_type` | `"style transfer"` | Also `"linear"`, `"ease in"`, `"strong style transfer"` |
| `start_at` / `end_at` | `0.0` / `1.0` | Fraction of sampling the adapter applies over |
| `embeds_scaling` | `"V only"` | |
| `width` / `height` | `512` / `512` | |
| `steps` | `10` | |
| `cfg_scale` | `7.5` | |
| `sampler` / `scheduler` | `"dpmpp_2m"` / `"karras"` | |
| `clip_vision_model` | env default | Override the CLIP vision tower |
| `ipadapter_model` | env default | Override the IP-Adapter weight |

```jsonc
{"prompt": "the reference mark, redrawn as a woodcut",
 "reference_images": ["ref-a.png", "ref-b.png"],
 "reference_weight": 0.65, "weight_type": "style transfer"}
```

> **Reference images must already exist in ComfyUI's `input/` directory.** This
> tool does not upload files; it names them. A missing file is a raised error,
> not a silent no-op.

#### `get_job_status(job_id)`

Polls a queued job. Returns `queued`, `running`, `completed` or `failed`, and on
completion the output image filenames with their provenance payload.

```jsonc
{"job_id": "a3f9...", "status": "completed",
 "images": [{"filename": "image_output_00042_.png", "subfolder": "", "type": "output"}],
 "params": {"seed": 1777732588, "sampler": "euler_ancestral", "steps": 30, ...}}
```

### Batch

#### `batch_generate(prompts, shared_params=None, randomize=None, deduplicate_seeds=True, priority="normal", keep_loaded=False)`

Queues one job per prompt, optionally randomising parameters across the batch.

- `prompts` -- a list of positive prompt strings, one per job.
- `shared_params` -- static parameters applied to every job (accepts an object
  or a JSON string).
- `randomize` -- per-job draws; overrides `shared_params` where it names the same key.
- `deduplicate_seeds` -- guarantees no two jobs share a seed.
- `keep_loaded` -- leave the model resident after the batch. See *VRAM behaviour*.

```jsonc
{"prompts": ["a fox mark", "an owl mark", "a wave mark"],
 "shared_params": {"width": 1024, "height": 1024, "steps": 24},
 "randomize": {"seed": "random", "cfg_scale": [5.0, 9.0],
               "loras": {"mode": "auto", "count": [1, 2], "weight": [0.25, 0.7]}}}
```

Randomisation spec forms:

| Form | Meaning |
|---|---|
| `"random"` | Draw from a curated safe pool |
| `"random_unsafe"` | Draw from the full pool, including values that often break |
| `[min, max]` | Uniform draw in a numeric range |
| `{"mode": "auto", "count": [1,3], "weight": [0.25,0.7]}` | LoRA stack selection |
| scalar (e.g. `24`) | Fixed at that value |

#### `batch_generate_ipadapter(prompts, reference_images, ...)`

The reference-guided equivalent, taking the same batch arguments plus the
IP-Adapter parameters from `generate_ipadapter`.

### Hardware

#### `list_models(model_type="all")`

Lists what is actually on disk, with sizes. `model_type` filters to one of
`unet`, `clip`, `vae`, `loras`, `ipadapter`, `clip_vision`.

```jsonc
{"model_type": "loras"}
// -> [{"name": "copperplate_v2.safetensors", "type": "loras",
//      "size_mb": 172.4, "format": "safetensors"}]
```

#### `download_model(url, model_type, filename=None, expected_sha256=None, token=None)`

Downloads from a Civitai-style URL. Writes atomically -- a partial download never
lands under the final name -- and sanitises `filename` to a single path
component, so a crafted URL cannot write outside the model directory.

```jsonc
{"url": "https://civitai.com/api/download/models/12345",
 "model_type": "loras", "filename": "copperplate_v2.safetensors"}
```

#### `get_vram_usage()`

Current GPU memory as the driver reports it. Reads `nvidia-smi` rather than this
process's CUDA context, which never allocates and would always read zero.

```jsonc
{"used_mb": 9216, "total_mb": 24564, "free_mb": 15348}
```

#### `emergency_cleanup(restart_comfyui=False)`

Frees GPU memory by asking ComfyUI to unload its models, then re-reads the driver
to report what was **actually** released. Useful before handing the GPU to a
non-ComfyUI job.

```jsonc
{"models_unloaded": ["sdxl-base"], "vram_freed_mb": 9001,
 "vram_before_mb": 9216, "vram_after_mb": 215}
```

> `restart_comfyui=True` is the heavier option: it stops and restarts the server.
> Use it when a normal unload does not recover the memory.

## Requirements

- **Python 3.11+**
- **A running ComfyUI instance** with the nodes your workflows use
  (`UnetLoaderGGUF` and the IP-Adapter nodes, for the shipped templates)
- **`nvidia-smi`** on `PATH` for the VRAM tools (NVIDIA drivers)
- **Models, LoRAs, embeddings and CLIP/VAE weights are NOT included.** Source
  them yourself and place them where ComfyUI expects them. This repository ships
  code, workflow templates and instructions -- never another party's licensed
  artifacts.

The IP-Adapter templates reference these filenames; substitute your own:

| Node | Default filename | Override with |
|---|---|---|
| `CLIPVisionLoader` | `CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors` | `clip_vision_model=` or `COMFYUI_CLIP_VISION_MODEL` |
| `IPAdapterModelLoader` | `ip-adapter-plus_sdxl_vit-h.bin` | `ipadapter_model=` or `COMFYUI_IPADAPTER_MODEL` |

These are **defaults, not requirements.** If your install uses a different CLIP
vision tower or IP-Adapter weight, pass the filename to `generate_ipadapter` or
set the environment variable -- no code change, and no failure to diagnose
silently.

A negative embedding is optional; the templates accept one via the
`negative_embedding` argument (e.g. `DeepNegative_xl_v1`).

## Install

```bash
pip install retiform-comfyui-mcp
```

> The **distribution** is `retiform-comfyui-mcp`; the installed **module** is
> `comfyui_mcp_server` and the **console script** is `comfyui-mcp-server`. The
> three names differ deliberately: PyPI rejects the bare `comfyui-mcp-server`
> as too similar to two existing projects, and keeping the module path and the
> script name stable means an install never breaks a working MCP client config.

Or from source:

```bash
git clone https://github.com/retiform-labs/comfyui-mcp-server
cd comfyui-mcp-server
pip install -e ".[dev]"
```

> **`mcp` is pinned below 2.0** -- in the MCP SDK 2.x line,
> `mcp.server.fastmcp` is a compatibility stub that raises on import. Do not
> widen that pin without migrating the server to `MCPServer`.

## Configure

All configuration is environment-driven. Nothing is assumed about where ComfyUI
lives.

| Variable | Default | Meaning |
|---|---|---|
| `COMFYUI_HOST` | `127.0.0.1` | ComfyUI host |
| `COMFYUI_PORT` | `8188` | ComfyUI port |
| `COMFYUI_PATH` | `~/ComfyUI` | ComfyUI install root (the dir with `main.py`) |
| `COMFYUI_MODELS_PATH` | `$COMFYUI_PATH/models` | Models root |
| `COMFYUI_OUTPUT_PATH` | `$COMFYUI_PATH/output` | Where renders land |
| `COMFYUI_CLIP_VISION_MODEL` | `CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors` | CLIP vision tower for reference-guided generation |
| `COMFYUI_IPADAPTER_MODEL` | `ip-adapter-plus_sdxl_vit-h.bin` | IP-Adapter weight (plus variant) |
| `COMFYUI_IPADAPTER_BASIC_MODEL` | `ip-adapter_sdxl_vit-h.bin` | IP-Adapter weight used by the VRAM-saving fallback |
| `COMFYUI_DOWNLOAD_TOKEN` | *(none)* | Bearer token for gated model hosts, so it need not travel through a tool argument |

### MCP client

```jsonc
{
  "mcpServers": {
    "comfyui": {
      "command": "comfyui-mcp-server",
      "env": { "COMFYUI_PATH": "/opt/ComfyUI" }
    }
  }
}
```

## VRAM behaviour

The MCP server is a **separate process** from ComfyUI. It never allocates CUDA
tensors, so `torch.cuda.memory_allocated()` inside it is structurally zero no
matter what ComfyUI is holding. This server therefore reads VRAM from the
driver (`nvidia-smi`) and releases it by asking ComfyUI to unload --

- `get_vram_usage` -- live used/free/total from the driver
- `emergency_cleanup` -- `POST /free {"unload_models": true, "free_memory": true}`
  on ComfyUI, reporting real before/after MB and `freed_mb`

`torch` is deliberately **not** a dependency. It cost ~2GB of CUDA to produce a
number that measured the wrong process.

Unloading after every job is usually the wrong trade: the next job then pays a
full model-load penalty. If you release between jobs, release at **batch**
boundaries.

## Provenance

Generated PNGs carry an A1111-compatible `parameters` block plus a structured
`GenerationInfo` record. Values that were not supplied are written as `?` and are
**never** defaulted -- an invented parameter in an embedded block is a permanent,
machine-readable lie.

Images rendered before the rename carry the same payload under the key
`LogoGenInfo`. Both keys are read; only the current one is written.

## Development

```bash
pip install -e ".[dev]"
python -m unittest discover -s tests -v
ruff check src tests
```

Nothing under `src/` may contain an absolute filesystem path; a test enforces
this, because a package that only works on its author's machine is broken for
everyone else.

## License

MIT -- see [LICENSE](LICENSE).

Copyright (c) 2026 David Newman and Guan.

---

## Sponsors

If comfyui-mcp-server is useful to you, consider supporting its continued development:

- **[GitHub Sponsors](https://github.com/sponsors/guan-tends)**
- **Bitcoin:** `bc1q0gd3mwjg3zy9sghv22kmpg823vss4c0zzdzg24`
- **Solana:** `Eu8wQcW68TKMs1a6eqzZu8znzU52QLqQugAMG8uCD6y6`
- **Ethereum / EVM:** `0x2733ff7c865C56d565a99BE1DC11B81cc76850A5`
- **XRP Ledger:** `r4X6e7McAQj7e8vBCeued1RYu4mCJrREDG`

---

Crafted with ❤️ by [Retiform Labs](https://retiform.dev)

TDQS

B3.3/5.0

Scored across 13 tools

Disambiguation3/5

The core generation matrix is reasonably separable (generate_image vs generate_ipadapter vs batch_generate vs batch_generate_ipadapter), but system_check and get_vram_usage both read driver-level GPU/VRAM data and are easy to swap, and get_comfyui_status vs get_job_status are only distinguished by careful reading. Four near-parallel generation tools plus two overlapping health tools make misselection plausible without studying descriptions.

Naming Consistency4/5

All names are snake_case and mostly verb-first (get_vram_usage, get_job_status, start_comfyui, list_models). Minor deviations: system_check is noun-first, the comfyui_ prefix is applied inconsistently (start_comfyui/stop_comfyui but not generate_image), and batch variants use a prefix while single variants use a suffix.

Tool Count5/5

Thirteen tools is well within a healthy range and each maps to a distinct operational need: lifecycle control, monitoring, model management, and four generation paths. Nothing feels padded or missing at the count level.

Completeness4/5

Lifecycle (start/stop/status), GPU monitoring/cleanup, model discovery and download, async generation with polling, plus batch and IP-Adapter variants give broad coverage. Gaps remain: no cancel/abort job, no queue management or job listing, and no way to run custom or list stored workflows.

Maintenance

ActivityMaintained
ResponsivenessNo recent community issues