blender-agent-sdk

Drive Blender from AI agents with a plain Python SDK — no addon to install. The same tools are also available as an MCP server and as a command line for building a scene over a conversation.

BlenderSession launches Blender (headless -b or with its GUI), injects a small bridge with --python, and talks to it over a private Unix socket (loopback TCP with a per-session token on Windows). Linux, Windows and macOS. On top of that sit computer-use style tools — observe (scene graph, object state, screenshots), discover (search 2,498 operators) and act (operators, RNA properties, checkpoints) — exported as ready-made OpenAI and Anthropic function-calling schemas with a one-line dispatch().

The design is grounded in a reverse-engineering pass over the Blender 5.2.2 binary itself with REA (Ghidra provider). See docs/re/ for the evidence and its limits.

한국어 설명은 아래에 있습니다.

from blender_agent import BlenderSession

with BlenderSession() as bl:                      # blender -b, factory startup
    bl.call_operator("mesh.primitive_monkey_add", {"location": [0, 0, 1]})
    bl.call_operator("object.modifier_add", {"type": "SUBSURF"})
    bl.set_property("objects", "Suzanne", "modifiers['Subdivision'].levels", 2)
    print(bl.scene_graph()["objects"])
    png, meta = bl.screenshot(640, 480)          # PNG bytes; renders from the scene camera

Install

Requires Python ≥ 3.10 and Blender. CI runs the full suite against Blender 5.2.2 on Linux x64 (including the GUI test under Xvfb), Windows x64 and macOS arm64; other Blender versions are untested. The wheel is pure Python, so one file covers every platform. Blender itself is not bundled; the SDK finds it through BlenderSession(blender=...), $BLENDER, PATH, or the default install location (/Applications/Blender.app on macOS, C:\Program Files\Blender Foundation\Blender * on Windows).

pip install https://github.com/yazzang-homelab/blender-agent-sdk/releases/download/v0.2.0/blender_agent_sdk-0.2.0-py3-none-any.whl
# or the development version
pip install git+https://github.com/yazzang-homelab/blender-agent-sdk

Related MCP server: Blender Open MCP

Work on a scene in a conversation

A script's BlenderSession ends with the script. To build something over many turns (from Claude Code, GJC, Cursor, or by hand) keep one Blender alive with either of these; both use the SDK's tools and exec policy unchanged.

MCP server (stdio, standard library only):

claude mcp add blender -- blender-agent mcp --http 8765                    # Claude Code
gjc mcp add blender --command blender-agent --arg mcp --arg --http --arg 8765   # GJC

Besides the SDK tools the server offers show (render, save to the gallery, return the image), gallery (where the user can look) and reset_scene. MCP hosts start servers with a minimal environment, so pass settings as options rather than environment variables: --blender PATH, --render-engine CYCLES (no GPU), --gui, --allow-exec.

Background session + CLI, for agents without MCP or for typing commands yourself:

blender-agent start --name tree --http 8765        # Blender keeps running between commands
blender-agent call --name tree call_operator '{"idname": "mesh.primitive_cone_add", "params": {"depth": 3}}'
blender-agent show --name tree --label "trunk"    # render, record, draw it in the terminal
blender-agent call --name tree checkpoint '{"label": "v1"}'
blender-agent stop --name tree

Seeing what the agent did

The CLI keeps per-session state under ~/.cache/blender-agent (%LOCALAPPDATA% on Windows): a local socket or named pipe plus a random key that every request must present.

Plug it into an agent

import anthropic
from blender_agent import BlenderSession, anthropic_tools, dispatch
from blender_agent.tools import result_text

client = anthropic.Anthropic()
with BlenderSession() as bl:
    messages = [{"role": "user", "content": "Build a snowman and show me."}]
    while True:
        reply = client.messages.create(model="claude-sonnet-4-5", max_tokens=2048,
                                       tools=anthropic_tools(), messages=messages)
        messages.append({"role": "assistant", "content": reply.content})
        calls = [b for b in reply.content if b.type == "tool_use"]
        if not calls:
            break
        messages.append({"role": "user", "content": [
            {"type": "tool_result", "tool_use_id": c.id, "content": result_text(dispatch(bl, c.name, c.input))}
            for c in calls]})

openai_tools() is the same list in OpenAI's {"type": "function", ...} shape, and dispatch() accepts OpenAI's JSON-string arguments directly. examples/agent_loop.py is a complete loop for both APIs (screenshots are sent back as images) plus a scripted policy that needs no API key:

python examples/agent_loop.py --policy scripted --out agent-output
python examples/agent_loop.py --policy anthropic "build a low-poly tree"

Tools

Errors raised inside Blender come back as RemoteError (type, message, remote traceback) and the session keeps running; dispatch() turns them into {"ok": false, "error": ...} so the model can correct itself. A request over the 64 MiB frame limit raises RequestTooLarge without being sent (the session is unaffected; dispatch() returns it as an error too). A hung or crashed Blender raises SessionLost after the request timeout and the process is killed.

Background vs GUI sessions

Set BLENDER_AGENT_RENDER_ENGINE=CYCLES on machines without a GPU (CI does); Cycles then renders on the CPU.

How it works

your agent ──dispatch()──► BlenderSession ──Unix socket or token-checked loopback TCP, 4-byte length + JSON──► bridge.py in Blender
                              │  owns the process, timeouts,                                 │ bpy on the main thread
                              │  Blender log → file (never blocks)                           │

The process is launched as blender [-b] [--disable-autoexec] [file.blend] [--factory-startup] [extra_args...] --python-exit-code 1 --python bridge.py -- --connect <address> --workdir <dir> [--allow-exec]. <address> is the socket path, or tcp:127.0.0.1:<port> on Windows, where CPython has no AF_UNIX. The bridge's first frame carries a random token the client put in Blender's environment, and the client drops any peer without it, so another local process cannot pose as Blender on the loopback port. The socket, Blender's log, renders and checkpoints all live in one private directory (0700; the user's own %TEMP% on Windows) that the client removes on every exit path (close, timeout, crash, failed start).

Trust model

Without allow_exec=True the bridge, not only the client, refuses every known way to run agent-controlled Python inside Blender:

• the exec_python method;

• operators that run scripts or code: script.*, console.*, text.run_script, preset add/execute operators, add-on and extension management (preferences.*, extensions.*), external openers (wm.url_open, wm.path_open, image.external_edit), and any operator taking a Python-evaluated parameter (data_path, eval_string, …, as in wm.context_*) or a settings collection of NodeSetting values (node.add_node, node.swap_node and the zone, group and bundle adders: 12 operators in 5.2.2);

• operators that write or re-evaluate agent-controlled Python: anim.keying_set_export (writes a .py) and anim.update_animated_transform_constraints (eval()s every F-Curve and driver data_path);

• wm.read_homefile/wm.read_factory_settings with app_template, since an app template is a Python package; invoked (INVOKE_*) render.* operators and node.render_changed, whose render jobs outlive the request (operators are also refused while a render job runs);

• operators that start another Blender which could run scripts: asset.open_containing_blend_file, and wm.previews_batch_generate/wm.previews_batch_clear with use_trusted=True (forced to False otherwise);

• embedded scripts in .blend files: Blender starts with --disable-autoexec, every file load (open_file, restore, wm.open_mainfile, …) runs with use_scripts=False, and an explicit use_scripts=True is refused;

• Freestyle, which runs Python whenever it renders regardless of auto-exec: style-module text blocks in script mode, and in editor mode an expression built from view-layer and line-set names that escapes only quotes. The bridge keeps render.use_freestyle off in every scene (linked ones included) at startup and after every request, before GUI redraws can render; a request that turned it on, e.g. by opening a file, gets a FreestyleDisabled error after it completes (a failed request keeps its own error, with a note). Turning it on with set_property is refused, and so are scene previews (wm.previews_batch_generate with use_scenes=True, forced to False otherwise), which a child Blender renders with the file's settings. Files opened by operators load without their own UI (load_ui=False; load_ui=True is refused).

This closes code execution inside the Blender process. It is not a filesystem sandbox: ordinary operators can still read and write files the user can reach (save, wm.save_as_mainfile, image and mesh import/export), including Python text blocks saved with text.save in GUI sessions. Run untrusted agents in a container or VM with only the files they need.

The guarantee covers what an agent can reach through tool calls. Constructor arguments are the caller's: extra_args (e.g. --enable-autoexec, --python-expr) and factory_startup=False (user add-ons and startup file) are passed to Blender unchecked, and in GUI sessions a blend_file given at launch opens with its own UI. Open untrusted files with open_file instead.

SDK first, MCP optional

Typical Blender MCP servers need an addon installed into the user's Blender and a host that speaks MCP. Here the core is a child process you own: it can be headless, sandboxed per task, run in CI, and its tools drop into any function-calling loop. blender-agent mcp is a thin adapter over the same tools for hosts that speak MCP. Prior art for comparison: ahujasid/blender-mcp (MCP addon, GUI only) and ptrthomas/blender-agent (HTTP exec endpoint, no SDK).

Reverse engineering

The SDK's shape comes from taking the stripped 174 MB (166 MiB) Blender 5.2.2 binary apart with REA + Ghidra (full auto-analysis: 301,393 procedures, 70 min). The results were cross-checked against the v5.2.2 source and a full runtime RNA dump. Highlights from docs/re/:

• CPython is statically linked (.PyRuntime section). gpu, bmesh and friends live in an internal _inittab, and _bpy is created at startup with PyModule_New. None of them is exported, so the only bridge that needs no install is a --python script inside Blender's own interpreter.

• The -b handler just sets G.background and mutes audio. The window manager survives, but there is no event loop and the GPU is not initialized, so the bridge owns the main thread under -b. Screenshots come from a render rather than GPUOffScreen, which would need gpu.init() and a real GPU (GitHub runners have none).

• bpy_op_fn_call_impl (decompiled) polls, converts kwargs to RNA, releases the GIL and calls WM_operator_call_py(..., is_undo). That is_undo flag is why GUI sessions pass it, and why -b sessions use .blend checkpoints.

• The binary holds 2,247 operator idnames from about 2,135 static WM_operatortype_append call sites. At runtime there are 2,498 operators. Every difference is accounted for: Python operators, add-on operators, dead and commented-out registrations, keymap strings, and one linker tail-merge. The SDK therefore searches the live registry instead of a static list.

Development

pip install -e ".[test]"
BLENDER=~/blender/blender pytest              # needs a real Blender; it fails, not skips, without one
xvfb-run -a pytest                            # also runs the GUI smoke test
UPDATE_GOLDEN=1 pytest tests/test_tools.py    # after an intentional tool-schema change

Regenerate the operator/RNA catalog: blender -b --factory-startup --python tools/dump_rna.py -- data/rna-5.2.2.json.gz.

License

GPL-3.0-or-later. The bridge imports bpy, so the project follows Blender's license. Blender binaries are never redistributed here.

한국어

AI 에이전트가 MCP 없이 순수 Python SDK로 Blender를 조작(computer-use)하게 해 주는 라이브러리입니다.

• BlenderSession이 Blender를 자식 프로세스로 띄우고(-b 헤드리스 또는 GUI), --python으로 브리지를 주입해 전용 Unix 소켓(0700 디렉터리, Windows는 세션 토큰으로 인증하는 127.0.0.1 TCP)과 4바이트 길이 접두 JSON으로 통신합니다. 애드온 설치가 필요 없습니다.

• Linux x64, Windows x64, macOS arm64에서 Blender 5.2.2로 전체 테스트를 CI에서 돌립니다(GUI 테스트는 Linux). 설치: pip install https://github.com/yazzang-homelab/blender-agent-sdk/releases/download/v0.2.0/blender_agent_sdk-0.2.0-py3-none-any.whl

• 관찰(scene_graph, object_info, screenshot) · 탐색(search_operators, describe_operator) · 조작(call_operator, set_property, checkpoint/restore, save) 도구를 OpenAI/Anthropic function-calling 스키마로 제공하고 dispatch() 한 줄로 실행합니다.

• exec_python은 기본 비활성이며, 브리지가 실행 인자로 직접 차단합니다. 스크립트를 실행하는 연산자 (script.*, text.run_script, data_path/eval_string 인자, .blend 내장 스크립트 등)도 막지만, 파일시스템 샌드박스는 아니므로 신뢰할 수 없는 에이전트는 컨테이너에서 돌리십시오.

• Blender 내부 예외는 RemoteError(타입·메시지·원격 traceback)로 돌아오고 세션은 유지됩니다. 멈추거나 죽으면 타임아웃 후 프로세스를 정리하고 SessionLost를 던집니다.

• 대화하면서 장면을 완성하려면 Blender를 계속 살려 두는 blender-agent mcp(Claude Code·GJC 등 MCP 호스트) 또는 blender-agent start/call/show/stop(CLI)을 씁니다. 결과는 터미널 인라인 이미지(kitty·iTerm2·하프블록), tmux 옆 창의 blender-agent watch, 갤러리 폴더(index.html 자동 새로고침), --http 웹(내부망은 --http-bind 0.0.0.0), --gui Blender 창 실시간 보기로 확인합니다.

• 헤드리스(-b)에는 undo 스택이 없어 checkpoint/restore(.blend 스냅샷)를 쓰고, GUI 세션에서는 ed.undo와 뷰포트 오프스크린 캡처를 씁니다.

• 설계 근거는 REA(Ghidra 프로바이더)로 Blender 5.2.2 바이너리 자체를 리버싱한 결과입니다. 증거와 한계는 docs/re/에 있습니다.

테스트는 실제 Blender로 돕니다: BLENDER=/path/to/blender pytest (GUI까지: xvfb-run -a pytest). 라이선스는 GPL-3.0-or-later이며 Blender 바이너리는 포함하지 않습니다.