mcp-local-asset-gen

A local MCP server (FastMCP) that lets AI coding agents generate custom UI assets — icons, illustrations, avatars — via a locally hosted image pipeline (ComfyUI + FLUX.1-schnell) and embed them straight into the codebase. No cloud, no uploads: diffusion, background removal, and vector-ready exports all run on your machine.

What it does

An agent call like this:

{
  "prompt": "cute bear avatar, 3D claymorphism style, soft lighting",
  "category": "avatar",
  "save_path": "assets/images/bear_avatar.png",
  "remove_bg": true
}

triggers local ComfyUI generation → background removal (rembg) → trimmed, padded, transparent PNG (+ @2x/@3x, optional pixel sizes and .ico) → saved into your project, with the relative path returned for direct embedding:

<img src={bearAvatar} alt="User Avatar" />

Related MCP server: ComfyUI MCP Server

Tools (5)

Path confinement is server-side: tools can only write inside an approved root — the project root plus any named workspace (../../etc/pass.png → POLICY_DENIED), never overwrite without an explicit flag. The agent chooses the output path: save_path may be relative (resolved under the selected workspace root) or absolute (used exactly as given when it lands inside an approved root; --allow-any-path lifts that check). The leaf must be a lowercase .png file — or .svg when format:"svg" (the vector variant traces the result with vtracer; only that file is written) — so the requested path is the written path. Results carry asset_path (relative to the owning root), asset_root (that owning root) and asset_path_absolute; opt-in include_image:true also attaches the @1x PNG as an MCP image block so the agent can see what it made. refine_asset uploads the source PNG to ComfyUI automatically (POST /upload/image); no manual input-dir copy is needed.

Prerequisites

• Python ≥ 3.11 (tested on 3.13), local ComfyUI (tested 0.41.0, any URL — default http://127.0.0.1:8188)

• A FLUX.1-schnell workflow: GGUF Q4_K_S (~6.8 GB, fits 10 GB VRAM) or the FP8 single file (~17.2 GB). Encoders: t5xxl_fp8 + clip_l; VAE: ae.safetensors. GGUF needs the ComfyUI-GGUF custom node. See src/gen_backend/setup_notes.md + docs/guides/.

• GPU recommended (RTX 3080 10 GB verified: 512 px icon ≈ 30 s); CPU works, slower.

Install

python -m venv .venv
.venv/Scripts/activate            # Windows; use bin/activate on Linux/macOS
pip install -r requirements.txt   # runtime (add -dev for pytest)

Download the weights into your ComfyUI models/ folders (diffusion_models/, text_encoders/, vae/) — they are not in this repo.

The background-removal model is downloaded once on first use (network required); to pre-download it, run python -c "from rembg import new_session; new_session('u2net')". Do not install rembg[cpu,cli]: its CLI extra pins python-multipart<0.0.22, which conflicts with fastmcp 4.1.0 (python-multipart>=0.0.26) — the rembg console script is therefore not usable; use the Python one-liner. Details: docs/guides/setup-and-first-asset.md.

Run the server / register with a client

Direct:

python -m src.mcp_server.server --comfyui-url http://127.0.0.1:8000

OpenCode (opencode.json, V2 shape — adjust paths/URL to your machine):

{
  "mcp": {
    "servers": {
      "asset-gen": {
        "type": "local",
        "command": ["<your-venv-python>", "<this-repo>/asset-gen.py",
                    "--comfyui-url", "http://127.0.0.1:8000"]
      }
    }
  }
}

Use the asset-gen.py launcher: it works from any working directory (no PYTHONPATH, no pinned cwd). With no --project-root, the server jails relative save_path values under the client's workspace — the project you are working in — so the agent's chosen path lands where you expect. Call server_info once to confirm project_root is your project; add "--project-root", "<dir>" to pin a different tree.

Verify: call server_info → {"backend":"comfyui", ..., "reachable":true}.

Then generate, e.g. generate_app_asset(prompt="...", category="icon", save_path="assets/icons/app.png", icon_sizes=[32,64], ico=true) → PNG + @2x/@3x + ‑32px/‑64px + .ico, all embedded by path. Patterns/hoa văn: category="pattern" (or texture) + style_preset="pattern" + seamless=true → an edge-cross-faded tileable image plus a <stem>-tile.png 2x2 preview (width/height max 1024). Background removal defaults to u2net; pass remove_bg_model="birefnet-general" (best edges; ~900 MB one-time download) or "birefnet-general-lite" / "isnet-general-use" on either generation tool.

Optional style LoRA: drop any .safetensors into ComfyUI models/loras/, verify it with check_model(name=..., kind="lora"), then pass lora="your_file.safetensors" (+ lora_strength, 0–2) to generate_app_asset (generate-only). The server bundles no weights — check the LoRA's own license before commercial use (most public FLUX style LoRAs are trained on FLUX.1-dev, which is non-commercial).

Every generated base PNG carries tEXt metadata (software, prompt, seed, job_id, …) so any later regeneration/refine can be traced back to its exact parameters.

CI: .github/workflows/ci.yml runs the full CPU-only suite on ubuntu + windows × Python 3.11/3.13 for every push/PR.

Verify

pytest tests/ -q   # 391 passed + environment-conditional skips on a full env

server_info unreachable never errors (reachable:false); generation without ComfyUI maps to BACKEND_UNAVAILABLE — both covered by tests.

Layout

• src/mcp_server/ — FastMCP tools, validation, orchestration, OpenCode wiring doc

• src/gen_backend/ — ComfyUI adapter (submit/poll/fetch/cancel) + pinned API-format workflows

• src/postprocess/ — rembg removal, trim/pad, multi-res, icon sizes, .ico export

• examples/react/ — sample UserProfile.tsx + layout README

• docs/ — architecture, interface contracts, setup guides

• fixtures/, tests/ — canned payloads + suites

Licenses / third-party notes (not legal advice)

• This repo: Apache-2.0 (LICENSE).

• Pinned stack: FastMCP (Apache-2.0), rembg lib (MIT), websocket-client (Apache-2.0), Pillow (HPND), FLUX.1-schnell weights (Apache-2.0, commercial OK), FLUX.1-dev weights are not used (non-commercial).

• bria-rmbg is an opt-in only background-removal model under a non-commercial BRIA license — default stays u2net. Check docs/contracts/ for the pinned contract versions.