Skip to main content
Glama

Bionics

An AI agent that controls Unreal Engine 5 with 199 tools.

One prompt → AnimGraph wired, Blueprints validated, assets spawned, PIE tested. The animation-pipeline layer no other UE5 AI tool ships.

See docs/demos/ for the BPDoctor hero GIF (T1.A) and the 4-minute "One Prompt Full Locomotion" demo script (T1.B).


What it does

  • 199 tools across 37 categories — UE5 actors, Blueprints, AnimGraphs, StateTrees, Control Rigs, Niagara, audio, materials, PIE, rigging, retargeting, EventGraph (K2), Linked Anim Layers, async tasks, session progress tracking, divine_powers NL→UE5 entry point

  • Native C++ bridge (BionicsBridge plugin) — architecturally expected ~5-20ms (in-process JSON-RPC over loopback HTTP) vs ~100-400ms for Python multicast remote-exec; benchmark pending. See plugins/BionicsBridge/README.md for the technical breakdown.

  • MCP server (FastMCP 2.11+ / 3.x) — drops straight into Claude Code, Cursor, Windsurf, or any MCP-aware client. Full MCP 2025-11-25 spec: annotations, outputSchema on query tools, async Tasks for long-running ops.

  • BPDoctor — 34-check static analysis with auto-fix for Blueprint/AnimBP errors (missing MM schema, dead cached poses, unconnected slots, empty state machines, blend-weight sum violations, etc.).

  • Full AAA animation pipeline — Motion Matching schema setup, IK Rig + IK Retargeter creation, batch retargeting of N animations, Linked Anim Layer AnimBPs, Control Rig asset creation and AnimBP binding.

  • Watch Mode — read-only screen-analysis loop with TTS overlay that explains UE5 systems while you work.

  • Persistent memory with optional sqlite-vec semantic search — cross-session SQLite memory and proven-sequence warm-starts. Opt-in vector search via pip install bionics-agent[vector,embeddings_local].

  • Sub-agent fan-out — named AgentDefinitions with tool subsets + dispatch_parallel for multi-perspective plans, critic ensembles, or parallel research.

  • OpenTelemetry — opt-in BIONICS_OTEL_ENABLE=1 emits OTLP spans per tool call.

  • 3-tier safety + lifecycle hooks + guardrails — safe / moderate / destructive, with confirmation gates and PreToolUse / PostToolUse / Stop hooks.

Highlights (copy-paste ready)

Bearer-token auth on the C++ bridge

# C++ plugin auto-generates a 256-bit token and writes it to:
#   <ProjectDir>/.bionics-bridge/instance.json
# The Python side auto-reads that file — you don't have to configure anything.
# To force a known token (CI):
export BIONICS_BRIDGE_TOKEN=my-known-token

CORS on the /bridge endpoint is locked to http://127.0.0.1 — browser-origin requests from another page cannot use a stolen token.

Session resume after a crash

from core.agent import AgentCore
agent = AgentCore(state_machine, safety, capture, executor)
sessions = agent._session.list_sessions()          # all saved sessions
resumable = agent._session.list_running_sessions() # just the ones that crashed mid-run
if resumable:
    agent.resume_from_session(resumable[0]["id"])  # continues exactly where it died

Sub-agent fan-out (parallel multi-perspective)

from core.agent_definitions import AgentDefinition, dispatch_parallel_sync

planner = AgentDefinition(name="planner",  system_prompt="Decompose the task.", tools=["list_tools"])
critic  = AgentDefinition(name="critic",   system_prompt="Find risks in the plan.")

plan, critique = dispatch_parallel_sync(
    [planner, critic],
    ["Build combat locomotion for Sworder:721"],  # broadcast one prompt to both
)

Force Claude to emit a tool call (never free-text)

from core.agent_definitions import AgentDefinition
pinned = AgentDefinition(
    name="structured",
    tools=["my_schema_tool"],
    tool_choice={"type": "tool", "name": "my_schema_tool"},  # or "any" / "required"
)

OTel observability

# Opt-in, exports OTLP spans per tool call
export BIONICS_OTEL_ENABLE=1
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318   # Tempo / Jaeger / any OTLP receiver
python mcp_server.py

Span attrs: bionics.tool.{name,category,safety_tier,ok,elapsed_ms,arg_count}.

Vector memory (opt-in)

pip install -e ".[vector]"           # sqlite-vec
pip install -e ".[embeddings_local]" # + sentence-transformers (~80 MB model)
from core.memory import BionicsMemory
from core.embeddings import HashEmbedder          # zero-dep, deterministic
# from core.embeddings import LocalEmbedder       # quality upgrade, 80 MB model

store = BionicsMemory(embedder=HashEmbedder())
store.remember("task_outcome", "ANIMATION", "mm_setup_v1", {"worked": True})
hits = store.search("motion matching locomotion", mode="semantic", limit=5)
# each hit has .distance (lower = closer)

Voyager-style self-verification (retry proven sequences)

from core.tool_cache import get_tool_cache
cache = get_tool_cache()
hit = cache.replay_with_verification(
    topic="ANIMATION",
    prompt="build motion matching for the trooper",
    execute_fn=lambda seq: my_executor.run(seq),   # returns bool
    max_attempts=3,
)
# Each failed sequence has its confidence decayed — stale sequences self-heal out.

Async task manager (long-running UE5 ops)

from core.bridge import ToolGate
gate = ToolGate(); gate.set_bypass_safety(True)
task_id = gate.execute("bionics_task_submit", {"tool_name": "ue5_live_coding", "args": {}}).data["task_id"]
# ...returns immediately, work runs on a thread pool...
status = gate.execute("bionics_task_status", {"task_id": task_id}).data
# When status.status == "completed":
result = gate.execute("bionics_task_result", {"task_id": task_id}).data

Quick start

# 1. Clone
git clone https://github.com/itsribbZ/Bionics.git
cd Bionics

# 2. Install in editable mode (exposes `bionics`, `bionics-gui`, `bionics-mcp` CLI commands).
#    Requires Python 3.12+ (3.14 tested). Optional extras unlock observability and vector memory.
pip install -e .
# ...or, for a dev contributor install with pytest/ruff/mypy:
# pip install -e ".[dev]"
# ...or with OTel + sqlite-vec enabled:
# pip install -e ".[otel,vector]"

# 3. Set your Anthropic API key
# Windows:  setx ANTHROPIC_API_KEY sk-ant-...
# Linux/Mac: export ANTHROPIC_API_KEY=sk-ant-...

# 4. Copy the config template and set your paths
cp config.yaml.example config.yaml
# (edit config.yaml — at minimum set paths.ue5_project to your .uproject directory)

# 5. Run one of three modes
python main.py                 # PyQt6 GUI (Auto Mode + Watch Mode)
python mcp_server.py           # MCP stdio/HTTP server (for Claude Code etc.)
python cli.py list             # Command-line tool runner

Optional (recommended for UE5 workflows): build the BionicsBridge C++ plugin into your UE5 project to drop tool-call latency — architecturally expected ~5-20ms (in-process loopback HTTP) vs the ~100-400ms of Python remote-exec; benchmark pending. See plugins/BionicsBridge/README.md.

Requirements

Requirement

Version

Python

3.12+ (3.14 tested)

OS

Windows 10/11 (full support). Mac/Linux partial — pywinauto and keyboard are Windows-only; cross-platform port tracked.

UE5

5.4+ (5.7 tested). "Python Editor Script Plugin" + "Web Remote Control" must be enabled for the Python fallback path.

Anthropic API key

Sonnet 4.5 recommended

Visual Studio (optional)

Required only to build the BionicsBridge C++ plugin

Connecting to Claude Code

Add Bionics as an MCP server. At your project root (or globally in ~/.claude/.mcp.json):

{
  "mcpServers": {
    "bionics": {
      "command": "python",
      "args": ["<ABSOLUTE_PATH_TO_BIONICS>/mcp_server.py"],
      "cwd": "<ABSOLUTE_PATH_TO_BIONICS>"
    }
  }
}

Restart Claude Code. All 199 tools show up as native tool-use.

Tool categories

All 37 categories, live-measured from the registry (register_all() → 199 tools). Run python cli.py list for the full per-tool listing.

Category

Tools

What it does

input

9

click, type, hotkey, drag, scroll, mouse control

capture

3

screenshot, region capture, monitor listing

vision

3

template match, OCR, wait-for-image

system

7

windows, clipboard, processes, system info

general

1

wait

meta

5

list / describe tools, categories, config, version

plans

4

save / list / load / execute multi-step automation plans

memory

7

persistent cross-session + Voyager tool-cache

tasks

6

async task manager — submit / status / result / cancel / list / clear

audit

5

audit-log tail + session listing / resume / progress

bionics

1

divine_powers NL→UE5 entry point

bp_doctor

2

AnimBP Doctor CLI wrapper — locate + scan

ue5_actor

17

spawn / query / delete / modify actors, properties, components, CVars, console

ue5_blueprint

15

Blueprint graph CRUD + interfaces

ue5_asset

11

asset create / save / delete / query + DataAsset bulk-set

ue5_animgraph

14

AnimGraph node create / wire / query, state machines, Linked Anim Layer (C++ plugin)

ue5_eventgraph

5

EventGraph (K2) events, function calls, variable nodes, pin wiring (C++ plugin)

ue5_bpdoctor

4

34-check Blueprint static analysis + auto-fix

ue5_rigging

4

IK Rig + IK Retargeter + batch retarget

ue5_retarget

1

native batch retarget (UE5.7 duplicate_and_retarget)

ue5_autorig

2

fail-closed humanoid autorig + skeleton bone validation

ue5_controlrig

3

Control Rig asset + AnimBP assignment

ue5_niagara

2

VFX emitter spawn + user-exposed param bind

ue5_audio

2

SoundWave import + SoundAttenuation configure

ue5_statetree

2

StateTree inspection + task add

ue5_material

4

material inspect / compile / scalar + vector set

ue5_pie

5

Play-In-Editor start / stop / pause / resume / state

ue5_native

11

C++ bridge status + native actor / asset / CVar / console ops

ue5_runtime

7

level save / load / new, viewport capture, logs, project info, class hierarchy

ue5_python

2

run Python code / file inside the editor

ue5_build

2

Live Coding compile (native bridge + fallback)

ue5_profiling

3

Insights trace start / stop + stat commands

ue5_debug

1

native log tail

ue5_widget

2

widget query + runtime widget listing

ue5_uasvc

2

UE5 Asset Service — preflight + native skeletal import

watch

11

Watch Mode start / stop / pause / resume / task / context / sessions

market

14

MarketBot: PDF parse + Claude-generated posts

Architecture

                        ┌──────────────────────────┐
  Claude Code / Cursor  │ MCP (stdio / HTTP)       │
  Windsurf / CLI       ─┤ mcp_server.py            │
                        │   199 tools registered    │
                        └──────────┬───────────────┘
                                   │
                                   ▼
                        ┌──────────────────────────┐
                        │ core/bridge.py           │
                        │   ToolRegistry, Gate,     │
                        │   Safety tiers            │
                        └──────────┬───────────────┘
                                   │
         ┌─────────────────────────┼─────────────────────────┐
         ▼                         ▼                         ▼
┌───────────────┐    ┌──────────────────────┐    ┌────────────────────┐
│ core/agent.py │    │ UE5 Remote Control   │    │ BionicsBridge      │
│  Auto Mode    │    │ HTTP :30010 (~400ms) │    │ C++ plugin         │
│  Watch Mode   │    │ Python RE socket     │    │ :8090 JSON-RPC     │
│  divine_powers│    │ :9998 (fallback)     │    │ ~5-20ms native     │
└───────────────┘    └──────────────────────┘    └────────────────────┘

Latency figures above (~400ms RE / ~5-20ms native) are architectural estimates, not measured benchmarks — a benchmark is pending.

Pre-built plans

The plans/ directory contains automation scripts runnable via python cli.py run <plan_name> or the GUI's Plan Library. Most shipped plans reference the Sworder:721 project — use them as templates rather than running them verbatim. New project-agnostic plans are queued on the roadmap.

Safety

Every destructive tool runs through a 3-tier gate:

Tier

Examples

Default

safe

mouse_move, screenshot, read_screen, query tools

no confirm

moderate

click, type_text, hotkey, open_file

1 confirm

destructive

ue5_delete_asset, delete_actor, overwrite_file

2 confirms

Under MCP (no GUI), destructive tools require explicitly setting BIONICS_MCP_ALLOW_DESTRUCTIVE=1 in the environment.

Testing

pytest tests/    # 580+ tests — core modules, tool registry (locked reads + summary), safety, UE5 rigging, memory, integration, OTel, task manager (DESTRUCTIVE gate + future-snapshot wait + auto-evict + clear tool), session (traversal guard), vector memory, sub-agent fan-out (DESTRUCTIVE gate + async-context-safe sync wrapper), Voyager verification

Status

  • Version: 0.8.4 (2026-05-29) — autorig now builds the 7-chain retarget canon (was 9) with len(CHAINS) as the single source of truth, live-verified against UE5 5.7 over the :8090 bridge; see CHANGELOG.md for the full history. 582-test suite (collect-verified).

  • Stability: v1.0 hardening in progress (see hellscape audit roadmap)

  • Active development

Roadmap

Short term: the internal hellscape audit (memory snapshot at project_hellscape_audit_2026-04-23.md) lays out T0 ship blockers, T1 WWZ-launch assets, T2 SOTA 2026 parity (native tool-use protocol, session resumability, lifecycle hooks, vector memory, subagent fan-out, MCP Tasks/Sampling/annotations), T3 moat deepening (Reflexion, zoom-before-click, Process Reward Models, internal eval harness), T4 ecosystem/community.

Contributing

Contributions welcome. The codebase is split cleanly between:

  • core/ — agent loop, executor, capture, safety, state, memory, verification, bridge

  • bionics_tools/ — tool modules (grouped by category)

  • plugins/BionicsBridge/ — UE5 C++ plugin

  • gui/ — PyQt6 interfaces

  • plans/ — pre-built automation sequences

  • tests/ — pytest suite

Tool authoring is a single @bionics_tool decorator — see bionics_tools/ue5_niagara.py for a minimal example (2 tools, 192 lines).

License

All rights reserved. Source available for portfolio review; not licensed for reuse. See LICENSE.