hy3d-mcp

An MCP server that turns a single concept image into a game-ready textured 3D model (GLB), fully locally on Apple Silicon, by wrapping the Hunyuan3D-MLX pipeline (Swift + MLX). One tool call: background cutout → shape → PBR paint → optional game-look finishing pass. Proven in production on a real game fleet.

Models land as file paths, never blobs — importing them into your engine is the caller's job (for Godot: copy into the project and run godot --headless --import).

Requirements

• Apple Silicon Mac with ~48GB unified memory (texture paint peaks ~25–33GB)

• Xcode or the Command Line Tools (for swift), and uv

• ~15GB free disk: 12GB of weights, 1.3GB of build output

• A built Hunyuan3D-MLX checkout and a worker Python environment — ./install.sh builds both for you; see that section before doing any of it by hand.

The server itself carries no ML dependencies; it shells out to the Swift binary and the worker venv.

Related MCP server: trident-mcp

Install as a Claude Code plugin (recommended)

The repo is also a Claude Code plugin that bundles the MCP server plus a create-3d-model skill (prompt → concept image → GLB, with all the input doctrine baked in):

/plugin marketplace add JimCline/hy3d-mcp
/plugin install hy3d-gen@hy3d-mcp

Once installed, ask for a 3D model in plain language or invoke /hy3d-gen:create-3d-model. The server starts via uv run --project <plugin-root> hy3d-mcp — uv resolves the venv on first run.

Install as a bare MCP server

git clone https://github.com/JimCline/hy3d-mcp ~/git/repos/hy3d-mcp

Register with your MCP client (e.g. in .mcp.json or Claude Code's claude mcp add):

"hy3d-gen": {
  "command": "uv",
  "args": ["run", "--project", "~/git/repos/hy3d-mcp", "hy3d-mcp"],
  "env": {
    "HY3D_REPO": "~/git/repos/hunyuan3d-mlx",
    "HY3D_PY": "~/.hy3d/worker-venv/bin/python",
    "HY3D_OUT": "~/hy3d-output"
  }
}

All three env vars are optional; the values above are the defaults. HY3D_PY is any python interpreter with the worker packages installed.

Set up the engine

The server is a thin wrapper — the actual pipeline is a separate Swift checkout that has to be cloned, built, and fed 12GB of weights. Either let the installer do it or follow the manual sequence below; both end at the same place.

The installer

./install.sh --plan     # print exactly what it would do, change nothing
./install.sh            # do it, confirming the build and the download

Seven phases — preflight, clone, swift build, metallib, weights, paint-large relayout, worker venv. Every phase inspects before it acts, so it is safe to re-run: finished work is skipped and a failed run resumes where it stopped. The cheap and idempotent phases run unattended; the two expensive ones (a ~4 minute build, a ~12GB download) stop and ask first. --yes runs unattended, --only N runs one phase, and --repo / --worker-venv relocate the targets.

From inside an MCP client, the setup_engine tool is the same script. It defaults to a dry run and returns the plan; it only executes when called again with confirm=true, so the agent has to show you the cost before spending it.

When it finishes it prints the HY3D_REPO and HY3D_PY values to put in your MCP config, and server_status should then come back all green.

Or by hand

The engine's own README covers steps 2–4; steps 5–7 are the parts it does not mention.

# 1. clone
git clone https://github.com/ZimengXiong/Hunyuan3D-MLX.git ~/git/repos/hunyuan3d-mlx
cd ~/git/repos/hunyuan3d-mlx

# 2. build (~4 min)
swift build -c release

# 3-4. weights (~12GB)
uvx --from huggingface_hub hf download \
  zimengxiong/hunyuan3d-mlx-shape-small --local-dir weights/shape-small
uvx --from huggingface_hub hf download \
  zimengxiong/hunyuan3d-mlx-paint-large --local-dir weights/paint-large

# 5. metallib — swift build never emits it; harvest it from the pip mlx wheel.
#    NOTE: mlx-swift and pip mlx are separate version series. Package.resolved
#    pins mlx-swift 0.31.4, but no such pip release exists — take the newest
#    pip mlx in the matching 0.31.x series (0.31.2 at time of writing).
uv venv /tmp/mlxharvest
uv pip install --python /tmp/mlxharvest/bin/python mlx==0.31.2
SRC=$(find /tmp/mlxharvest -name mlx.metallib | head -1)
for d in metallib .build/arm64-apple-macosx/release; do
  mkdir -p "$d" && cp "$SRC" "$d/mlx.metallib" && cp "$SRC" "$d/default.metallib"
done

# 6. paint-large ships flat, the binary wants it nested
cd weights/paint-large
mkdir -p hunyuan3d-paint-v2-0 hunyuan3d-paintpbr-v2-1
ln -s ../vae ../unet hunyuan3d-paint-v2-0/
ln -s ../vae ../unet hunyuan3d-paintpbr-v2-1/
ln -s dinov2 dinov2-giant
cd ../..

# 7. worker venv — uv, not pip: uv-created venvs have no pip in them
uv venv ~/.hy3d/worker-venv
uv pip install --python ~/.hy3d/worker-venv/bin/python \
  opencv-python numpy trimesh pillow scipy pyrender pygltflib

Then set HY3D_REPO=~/git/repos/hunyuan3d-mlx and HY3D_PY=~/.hy3d/worker-venv/bin/python.

Either way, verify with the server_status tool. It re-checks every requirement and each failing check carries its own fix.

The three setup gotchas

install.sh handles all three; they are documented here because they are what a by-the-book install of the upstream repo gets wrong, and what server_status is looking for when it fails.

1. Metallib — swift build never emits the MLX metallib (mlx-swift SwiftPM limitation). Harvest mlx.metallib from the pip mlx wheel — not the version string in Package.resolved, which is mlx-swift's own series and has no pip counterpart (there is no pip mlx 0.31.4). Take the newest pip mlx sharing its major.minor, and copy it as both mlx.metallib and default.metallib into metallib/ and into .build/arm64-apple-macosx/release/ (the real dir — .build/release is a symlink).

2. Weight layout — the paint-large HF repo ships flat, the binary expects nested: symlink hunyuan3d-paint-v2-0/{vae,unet} and hunyuan3d-paintpbr-v2-1/{vae,unet} → ../vae, ../unet, and dinov2-giant → dinov2, inside weights/paint-large.

3. Paint model flag — the server always passes --paint-model pbr; the rgb default targets a weight set that isn't installed.

Tools

Generation is serialized — one job at a time; concurrent calls queue rather than OOM the machine. generate_model streams MCP progress notifications the whole way through, so a slow job stays distinguishable from a hung one, and cancelling the call kills the engine process rather than leaving it holding the queue.

Notes from production use

• Outputs carry vertex normals. The engine writes only POSITION and TEXCOORD_0; Godot does not synthesise the rest, and lights the whole mesh off one constant vector when NORMAL is missing — which presents as a bad material, not a missing attribute, and is expensive to diagnose. generate_model and finish_model inject it by default (normals=false opts out). Injected, not re-exported: a round trip through a mesh library rebuilds the material block and destroys the emissive map the finish pass writes.

• Previews degrade rather than fail. pyrender wants a window-server connection despite the "offscreen" name, and a daemonised MCP server usually has none. When rasterising fails, render_preview returns the <name>.glb.views.png and .rendercheck.png contact sheets the paint pass writes beside every GLB, and marks source: "generator_sheets" so you know they're fixed views, not the ones you asked for. Shape-only output has no sheets to fall back on.

• octree costs scale with concept detail, not just the number. A smooth-hulled subject at octree=384 finished in ~6 minutes; a lattice/greeble-heavy one ran 16.5 minutes and pushed the machine deep into swap — and at defaults that same subject resolved its truss braces fine in 790s. Reach for octree when thin struts fuse together, not as a general quality dial.

• Vertex counts vary ~4.5× across subjects at identical settings (82k for a gun housing, 367k for a trussed deck). There is no knob that trades detail back down; budget for the heavy case, or simplify the concept.

• accent_coverage_pct near 0 is usually the extractor's range, not your concept. It keys on saturated red-dominant regions and is tuned for broad accent panels; thin indicator strips score near zero.

• Long jobs and client timeouts. A detailed concept can legitimately paint for 13+ minutes, which exceeds some clients' idle-abort defaults. The progress stream is what keeps those timers alive; if your client still gives up, raise its tool timeout (Claude Code: CLAUDE_CODE_MCP_TOOL_IDLE_TIMEOUT, or a per-server timeout in MCP settings). If a job is ever abandoned mid-flight, cancel_job frees the queue without hunting for a pid.

Input doctrine

• Feed naturally lit concept art — the model de-lights internally. Pre-flattened "albedo-style" input bakes pale and featureless.

• No drop shadows in the source image — they reconstruct as literal geometry under the model.

• Single object, plain background, roughly centered; ¾ view works best. prepare_concept / auto_cutout handle the background keying.

Non-goals

• No cloud fallback.

• No mesh post-processing (decimation/repair proved destructive on generated meshes; LODs belong to your engine's importer).

• No batch tool — loop generate_model; the queue serializes.

• No multiview input yet — the pipeline is single-image at every entry point. Investigated and specced, not built; see below.

Investigations

• docs/multiview-routes-2026-08-02.md — multi-image → 3D. Three routes costed (native MLX port, ComfyUI hybrid, upstream PR), six open questions, and a Phase 0 A/B that settles whether multiview earns its keep before anything is built. Tabled, decision open.

• docs/multiview-findings-2026-08-02.md — the investigation behind it. Read this for why contact sheets must never be fed back in, why generator sheets must never be used to judge geometry, and the measurement showing +31% geometry from input quality alone.

• docs/field-report-2026-08-01.md

License

MIT — but that covers this wrapper code only. This repo distributes no model weights and no Tencent code.

Model weights license (read this)

The pipeline runs on Tencent's Hunyuan3D weights, which you download yourself and which are governed by the Tencent Hunyuan 3D 2.0 / 2.1 Community License Agreements (2.0, 2.1) — the paint stage uses both generations, so both apply. Highlights, not legal advice; read the licenses:

• Territory: the license does not apply in the European Union, the United Kingdom, or South Korea. If you're there, you may not use the weights at all.

• Scale: products/services exceeding 1M monthly active users require written permission from Tencent.

• Attribution: distributing or productizing anything built on the weights requires the Tencent license notice; 2.1 asks for a "Powered by Tencent Hunyuan" mark.

• Acceptable use: no training competing models on it, no undisclosed synthetic-media deception, no military use, among others.

• Your outputs are yours: Tencent claims no rights to generated 3D models; you own them and are responsible for how you use them.

The Hunyuan3D-MLX Swift port this server shells out to is itself MIT-licensed.