trellis2-mcp

An MCP server for the image → 3D → rigged character pipeline.

Generate a reference image with FLUX, turn it into a textured mesh with TRELLIS, then generate a skeleton and skin weights with SkinTokens/TokenRig — all as MCP tools, driven from a single conversation.

  generate_image          generate_3d              rig_model
  FLUX.2 / mflux    →     TRELLIS           →      SkinTokens / TokenRig
  (bmb, Apple MLX)        (big, RTX 3090)          (big, RTX 3090)
       .png                 .glb + .ply              rigged .glb

The two GPU stages hot-swap on one RTX 3090 — see GPU hot-swapping.

Tools

generate_3d returns an output_name; pass it straight to rig_model:

generate_3d(image="ref.png", output_name="gargoyle")
rig_model(mesh="gargoyle.glb", num_beams=10)

Relative names in rig_model resolve against the TRELLIS output dir, so no path plumbing is needed between the two stages.

Related MCP server: BFL MCP Server

Installation

Three independent installs. The MCP server itself runs inside the TRELLIS venv; the rigger runs as a subprocess in its own.

The two GPU environments cannot be merged. TRELLIS needs Python 3.10 with torch 2.6.0+cu124; SkinTokens needs Python 3.11 with torch 2.7.0+cu128 (open3d ships no cp313 wheel, and bpy >= 5.1 is cp313-only). This is why rig_model shells out instead of importing.

Prerequisites

• NVIDIA GPU, ≥ 14 GB VRAM for rigging (a 24 GB 3090 has comfortable headroom)

• NVIDIA driver ≥ 525, CUDA toolkit ≥ 12.1

• uv

• An SSH alias to a Mac running mflux, if you want generate_image

1. TRELLIS (Python 3.10) — hosts the MCP server

git clone https://github.com/microsoft/TRELLIS.git /home/ladvien/trellis/repo
cd /home/ladvien/trellis/repo

uv venv --python 3.10 /home/ladvien/trellis/.venv
source /home/ladvien/trellis/.venv/bin/activate

# TRELLIS's own installer pulls the heavy CUDA extensions
# (spconv, xformers, flash-attn, nvdiffrast, diffoctreerast, ...).
. ./setup.sh --basic --xformers --flash-attn --diffoctreerast --spconv --mipgaussian --nvdiffrast

This deployment runs torch 2.6.0+cu124 and xformers 0.0.29.post3.

Download the TRELLIS model:

uv pip install huggingface_hub
hf download JeffreyXiang/TRELLIS-image-large \
    --local-dir /home/ladvien/trellis/models/TRELLIS-image-large

You should end up with pipeline.json and a ckpts/ directory holding the sparse-structure and SLAT encoder/decoder/flow safetensors.

Install this server into that venv:

cd /home/ladvien/trellis2_mcp
uv pip install --python /home/ladvien/trellis/.venv/bin/python -e .

2. SkinTokens / TokenRig (Python 3.11) — the rigger

One script does the whole thing:

./scripts/install_skintokens.sh

It is idempotent — re-running will not re-download torch. It performs:

1. Clone VAST-AI-Research/SkinTokens to /home/ladvien/skintokens/repo

2. Create a Python 3.11 venv at /home/ladvien/skintokens/.venv

3. Install torch 2.7.0 / torchvision 0.22.0 / torchaudio 2.7.0 (cu128)

4. Install requirements.txt (the bpy wheel alone is several hundred MB)

5. Install a prebuilt flash-attn 2.8.3 wheel, auto-selecting the C++ ABI variant matching your torch build

6. Patch bpy_server to bind loopback only (see Security)

7. Download checkpoints via download.py --model

8. Verify imports and checkpoint sizes, failing loudly on any gap

Rigging model checkpoints (~1.6 GB, fetched by step 7 into /home/ladvien/skintokens/repo/experiments/):

It also pulls the Qwen3-0.6B config only into models/Qwen3-0.6B/ (ignore_patterns=["*.bin", "*.safetensors"]). The transformer weights live inside the TokenRig checkpoint, so that download being tiny is correct.

Do not build flash-attn from source (--no-build-isolation) — it takes 1–3 hours and can OOM system RAM. Do not try flash-attn 4 either; Qwen3, the TokenRig backbone, does not support it.

Verify:

cd /home/ladvien/skintokens/repo
/home/ladvien/skintokens/.venv/bin/python demo.py \
    --input examples/giraffe.glb --output /tmp/giraffe.glb \
    --use_transfer --num_beams 3

A quadruped is the right first test — it is exactly what template-based riggers cannot do. A correct install yields a sane spine, four leg chains, and a neck.

3. mflux on the Mac (optional, for generate_image)

On bmb, install mflux into ~/mflux-venv and make sure ~/mflux-output exists. The server reaches it over SSH and copies results back with scp, so key-based SSH must work non-interactively.

Configuration

All paths are environment variables with the defaults below.

Outputs land in $TRELLIS_OUTPUT_DIR, with images/ and rigged/ beneath it.

Running

sudo cp trellis2-mcp.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now trellis2-mcp

Serves SSE on 0.0.0.0:9786. After changing server code, restart the unit — a running instance keeps serving the old code.

Where output goes

Local $TRELLIS_OUTPUT_DIR is the working area. The durable location is the codex_fs library (NFS export from three, autofs-mounted), written on every run:

An asset directory holds asset.glb, asset.ply, reference.png, and metadata.json. rig_model adds asset_rigged.glb to the same directory and flips rigged to true in the metadata, so a rigged asset stays one bundle.

asset_type selects the gen_ai subfolder — characters, props or environments. Publishing refuses to overwrite an existing directory: these are library assets, and silently clobbering one would destroy its provenance. Pick a different output_name or remove the directory deliberately.

Note on the raw/curated split. /mnt/codex_fs/models/ is the documented target for raw pipeline output, with gen_ai/ reserved for hand-promoted assets. This server writes generated assets straight into gen_ai/ by explicit choice, so that tree now contains everything generated, not a curated subset.

Record provenance at generation time. Pass source_prompt and source_image_model to generate_3d; they are the only record of how an asset was made and cannot be reconstructed afterwards. Omitting source_prompt writes a _provenance_warning into the metadata rather than leaving the gap silent. generate_image returns the prompt-derived reference_dir to carry forward.

Getting the best mesh and texture

Scale: every asset comes out ~1 m tall

TRELLIS normalizes into a unit cube, so a generated human, a teacup and a cathedral all export about 1 metre. Anything reading a measurement in metres — garment conforming, physics, engine import — is silently wrong until it is scaled.

generate_3d always reports geometry.dimensions_m, and target_height_m scales at export:

generate_3d(image="ref.png", target_height_m=1.75)   # adult human

The .ply Gaussian splat is always exported at normalized scale; only the .glb is scaled. rig_model reports dimensions_m too, since use_transfer inherits whatever scale the source mesh had.

Texture: render_resolution is the real ceiling

The texture is baked from renders of the Gaussian splat. Upstream to_glb hardcodes those renders at 1024px, which means a texture_size above 1024 only upsamples — four times the file size, no additional detail. This server calls the bake internals directly so the source resolution is a parameter, and rejects texture_size > render_resolution rather than letting you pay for empty pixels.

Bake VRAM is the binding constraint. bake_texture in opt mode holds every view's observation, mask, UV and UV-derivative buffer on the GPU simultaneously for all 2500 optimizer steps, so cost is nviews x resolution^2 x 37 bytes:

On a 24 GB card, 2048 x 100 fits only with TRELLIS evicted (evict_trellis_for_bake=true, costing a ~30s reload next generation). 2048 x 150 does not fit at all. Both are checked up front — a request that cannot work fails in seconds with the numbers, rather than OOMing after minutes of decimation and UV unwrap.

Geometry: simplify is a removal ratio

simplify is passed to pyvista's decimate(target_reduction), which removes that fraction of faces. simplify=0.95 keeps 5%; 0.8 keeps 20%; 0 disables simplification entirely.

(Upstream's postprocess_mesh docstring says "ratio of faces to keep" — it is wrong; to_glb's "ratio of faces to remove" is correct.)

Geometric detail is capped by the mesh decoder's resolution: 64 lattice, so below roughly 0.5 you are adding redundant triangles rather than detail.

The biggest lever is multi-view conditioning

From a single image the entire unseen side of the object is hallucinated — which, for a character, is exactly the surface garments and armor conform to. No sampler setting comes close to fixing this; only more conditioning views do.

generate_views produces the set, then feed it straight through:

views = generate_views(prompt="<subject only, no view language>", seed=7)
generate_3d_multi_image(images=views["images_json"], source_prompt=..., asset_type="characters")

What makes it work, confirmed against a single-image control with identical bake settings:

• One seed across every view. This does most of the consistency work.

• One clause changed, nothing else. The tool appends the view phrase and leaves the rest of the prompt byte-identical, so the subject does not drift.

• The same image model for all views.

Give it the subject only — clothing, pose, lighting, background. View language in your prompt (seen from, facing directly, in profile, …) fights the injected clause and desynchronises the set, so it is rejected rather than silently blended.

Avoid strongly asymmetric details. A patch on one arm tends to mirror between front and back, handing TRELLIS contradictory evidence about which arm carries it. It survived that in testing, but the cleaner run omits it.

The default set is front,back, and that is deliberate. FLUX.2 klein will not produce a 90° profile no matter how forcefully the prompt demands one — tested with plain and emphatic phrasing on both klein-4b and klein-9b. left and right come back as front-facing three-quarters, nearly identical to each other and close to the front view, so they cost ~110s each and add almost nothing. They remain available if you want to try them on a different subject.

If you need genuine side coverage, that is the point where a purpose-built multi-view diffuser (Zero123++ / SV3D / MV-Adapter) earns its setup cost.

Watch for pose drift. Limb positions can shift between views — arms hanging in front/back but spread in the sides. That is contradictory geometry evidence, which matters more than texture drift. Pin the pose explicitly in the prompt ("arms held straight out at shoulder height") rather than loosely ("arms away from the body").

Budget the time: each view is a full generation (~110s on flux2-klein-9b), so a four-view set is 7–8 minutes before TRELLIS even starts.

ss_steps / slat_steps are past the knee around 24; 12→24 is visible, 24→50 mostly is not. Input image quality matters more: clean cutout, subject filling the frame, flat even lighting.

Getting the best rig

rig_model reports mesh_diagnostics before rigging and a skeleton report after, plus a merged warnings list. Both are recorded into the published metadata.json. What they mean:

Tighten the bounding box — the biggest non-obvious lever

Asset.normalize_vertices() divides by (v_max - v_min).max() — uniformly, by the largest bounding-box dimension — and joint coordinates are then quantized into 256 bins. So joint placement resolution is largest_dimension / 256, and anything that widens the box without adding riggable geometry costs precision on all three axes: a held weapon, a pedestal, wide wings, stray fragments.

Measured on a real asset: appending one stray triangle 1.2 m from the body took bbox_inflation from 1.03 to 1.99 and joint resolution from 3.91 mm to 7.56 mm. Half the precision, from one triangle. Rig the body alone and reattach props afterwards.

joint_resolution_mm and bbox_inflation are in every rig response.

Rigs are not reproducible, and cannot be made so

demo.py generates with do_sample=True and ships no seed. Adding one was tried and measured — seeding torch/numpy/random at startup, again per-batch immediately before generate, then additionally with torch.use_deterministic_algorithms(True, warn_only=True), CUBLAS_WORKSPACE_CONFIG=:4096:8 and cuDNN determinism. The same mesh at the same seed still produced 47 vs 44 joints.

Hashing the input batch across two runs gave identical digests, so the data pipeline is deterministic — the divergence is inside the CUDA forward, where warn_only leaves flash-attn's nondeterministic kernels in place (there is no deterministic flash-attn), and autoregressive beam-sample amplifies float jitter into a different token, which changes the whole skeleton.

Practical consequence: rerolling is just re-running. Generate 3–5 candidates at num_beams=3, compare their skeleton reports, then re-run the winner's settings at num_beams=10. You cannot recreate a specific good rig, so keep the GLB — metadata.json marks this with reproducible: false rather than implying its recorded parameters would reproduce the asset.

Judge the skeleton first

The model generates the entire skeleton, then the skin tokens conditioned on it. A bad skeleton poisons the skinning and no postprocessing recovers it. The skeleton block gives joints, roots, chain_tips, branch_points, max_depth and symmetry — check those before looking at deformation.

A known-good reference: the rigged succubus scores 43 joints, 1 root, 12 tips, depth 7, symmetry 0.86, zero dead vertices.

use_postprocess can only take away

It applies geodesic voxel binding as a multiplicative mask (asset.skin *= voxel_skin(...)), so it can remove influence but never add it. Use it when weights bleed across gaps — arm driving the ribcage, a cape picking up leg weights — because weight transfer is Euclidean nearest-neighbour and jumps across anatomical gaps. But on thin geometry it can strip a vertex entirely, which is what zero_weight_vertices catches. Always compare a run with and without.

Other levers

• temperature: lower (0.7–0.8) for conventional bipeds and quadrupeds. Raise it for novel creatures — the supervised model regresses to average solutions on out-of-distribution assets and drops auxiliary limbs, so a wingless dragon wants higher temperature, not lower.

• use_skeleton is the highest-quality path: rig once, fix the skeleton by hand in Blender, then re-skin against it. Skeleton errors propagate into skinning but not the reverse, and skinning is the part the model is genuinely good at.

• target_height_m does not affect rig quality. SkinTokens renormalizes, so scale is irrelevant to it — don't tune it hoping.

• Vertex count: the training mixture spans ~1.3k–17k vertices. Note the shape encoder samples a fixed number of surface points, so raw count matters less for the skeleton than for skin decode and transfer. If you decimate to a proxy, remember transfer to a different vertex count drops off exact Umeyama alignment onto a PCA fallback that can come back mirrored or rotated.

• group_per_vertex stays at 4 and is not exposed: it's hardcoded in demo.py and the GRPO sparsity reward explicitly penalises more than 4 influences above 0.1, so raising it works against the training objective.

• Bone names are bone_0..bone_N, not Mixamo or UE5. demo.py hardcodes the class as "articulation", so the configs/skeleton/ templates aren't wired into this path.

This feeds back into your image prompt

A-pose with limbs clearly separated from the torso and fingers splayed is required by the rigging step, not just by TRELLIS. Weight transfer is Euclidean nearest-neighbour, so arms pinned to the ribcage bleed torso weights into the arms and clenched fists cross-contaminate finger weights.

GPU hot-swapping

TRELLIS and TokenRig each want most of a 24 GB card, so exactly one holds the GPU at a time. Every GPU tool serializes on a single lock:

• TRELLIS stays resident between generations — a reload costs ~30 s, so repeated generate_3d calls are fast.

• rig_model evicts TRELLIS first, then runs the rigger as a subprocess. That process's VRAM is reclaimed by the OS when it exits, so there is nothing to unload on the way back.

• The next generate_3d reloads TRELLIS automatically. trellis_evicted in the rig_model response tells you whether a swap actually happened.

• free_gpu hands the card back manually.

Measured on a 3090 with a desktop session running: 18.0 GB free → 12.2 GB with TRELLIS resident → back to 18.0 GB after eviction.

Because the lock lives in the server process, run one GPU tool at a time; a second process bypasses it entirely.

Security

Upstream SkinTokens issue #7: src/server/bpy_server.py binds 0.0.0.0:59876 and deserializes untrusted request bodies with torch.load(weights_only=False) — i.e. pickle, which executes arbitrary code during unpickling, before any validation. That is unauthenticated RCE reachable from the LAN whenever a rig job is running.

Its only client is demo.py on loopback, so install_skintokens.sh rebinds it to 127.0.0.1 and aborts the install if the patch does not apply. If you install SkinTokens by hand, do this yourself.

Known issues and gotchas

Rigging hangs forever at the progress bar. Upstream issue #3: http_proxy / https_proxy intercept the loopback call from the dataloader to bpy_server. rig_model strips all proxy variables from the subprocess environment and sets NO_PROXY=localhost,127.0.0.1, so this should not bite here — but it will if you run demo.py by hand.

demo.py exits 0 even when export fails. It prints [Error] ... and returns normally, so the exit code is not a success signal. rig_model verifies the output file exists and is non-empty instead.

Blender import errors on a rigged GLB. Delete the glTF_not_exported node after importing. No deeper fix upstream.

Bone names are generic. TokenRig emits bone_0 … bone_N, not Mixamo or UE5 conventions, and output is always GLB. If you need FBX or UE5 Mannequin naming, the third-party Rizzlord/ComfyUI-SkinToken wrapper adds both.

flash_attn undefined symbol on import. Wrong C++ ABI wheel — install the other variant. The install script picks this automatically.

Distilled klein models reject guidance. flux2-klein-4b / -9b / -9b-kv accept only guidance=1.0; use a flux2-klein-base-* model to vary it. generate_image validates this and tells you rather than failing deep in mflux.

num_beams is the rigging speed/VRAM lever. Default 10 for quality; drop to 1–3 for fast iteration.

Skin weights need review. Shoulders and hips are where auto-generated weights fail. Pose to extremes before trusting them — true of any auto-rigger.