# llmgine

**The LLM-native game engine.** An ECS game engine where intelligence is a core
primitive: attach a **Mind** (LLM cognition), **Eyes** (perception/vision), and
**Voice** (neural TTS) to *any* entity the same way you attach physics or a
sprite. An NPC, a boss, a monster, a quest giver, a faction, the weather — if
it's in the world, it can think, see, and speak.

> Working title. TypeScript · 3D (three.js) + headless + 2D canvas · MIT.

```
deterministic 60 Hz ECS sim  ←— validated intents —— async LLM minds
        │                                                  ▲
        └—————— perception snapshots + pixel vision ———————┘
```

The hard problem: games are deterministic real-time loops; LLMs are slow,
async, and non-deterministic. llmgine resolves it structurally:

- The sim **never blocks on a thought**. Minds observe snapshots, think on
  their own cadence (plus event wakeups: *damaged*, *spoken to*), and return
  intents.
- Intents pass the same **validated action pipeline** as player input — a Mind
  can only do what its body allows. No hallucinated teleports, no 9999 damage.
- Every LLM-augmented module has a **deterministic fallback** (behavior
  policies, weighted loot tables, quest state machines). **If the API is down,
  the game still runs.**
- **Genesis** turns the LLM into a content generator: prefabs, loot, quests as
  validated JSON. The model proposes; the engine disposes.

## Quickstart

> **Requires Node >= 18.** llmgine is **not published to npm yet** — today you
> run it from a clone (which works end-to-end); `npm install llmgine` will
> work once it's published.

**From a clone (the working path today):**

```bash
git clone https://github.com/lordbasilaiassistant-sudo/llmgine
cd llmgine
npm install
npm run build       # engine → dist/
npm test            # unit tests, no network (pretest rebuilds dist/ automatically)
npm run demo        # the 3D arena demo → http://localhost:4173
```

Optional: copy [.env.example](./.env.example) to `.env` and add a `ZAI_API_KEY`
to give minds a live model (see below) — the demo auto-detects it locally.

**Start your own game** — scaffold from the clone; the generated project links
back to it with a `file:` dependency (no npm registry needed):

```bash
node dist/cli/index.js create my-game    # from the clone root
cd my-game && npm install && npm run dev # http://localhost:4173
```

**From npm (once published):**

```bash
npm install llmgine        # not yet — 404s today, tracked for a deliberate release
npx llmgine create my-game
```

What the code looks like:

```ts
import {
  World, GameLoop, SpatialGrid, ActionRegistry, actionSystem,
  Transform, Velocity, Named, Health, Speech, Behavior,
  STANDARD_VERBS, behaviorSystem, movementSystem,
  Mind, MindMemory, CognitionDriver, OpenAICompatibleProvider,
} from "llmgine";

const world = new World(42);            // seeded — deterministic
const grid = new SpatialGrid();
const actions = new ActionRegistry();
for (const v of STANDARD_VERBS) actions.register(v);

// any entity + Mind = intelligent entity
const guard = world.create();
world.add(guard, Transform, { x: 0, y: 0 });
world.add(guard, Velocity);
world.add(guard, Named, { name: "Gate Guard" });
world.add(guard, Health);
world.add(guard, Speech);
world.add(guard, Behavior, { mode: "idle" });
world.add(guard, Mind, {
  persona: "A vigilant town guard. Suspicious of strangers.",
  goals: ["guard the gate"],
  thinkEvery: 8,                        // seconds between thoughts
  fallbackMode: "wander",               // deterministic policy if the LLM is unavailable
});
world.add(guard, MindMemory);

const driver = new CognitionDriver({
  provider: new OpenAICompatibleProvider(),   // reads ZAI_API_KEY / LLM_API_KEY + LLM_BASE_URL
  actions, grid,
});
world.addSystem(actionSystem(actions));
world.addSystem(behaviorSystem());
world.addSystem(movementSystem(grid));
world.addSystem(driver.system());

new GameLoop(world).start();            // browser; or loop.advance(n) headless
```

## Get a free model (GLM)

The default provider targets [z.ai](https://z.ai)'s OpenAI-compatible API,
where **glm-4.5-flash is free** — free minds for every NPC in your game:

1. Create a key at z.ai and set `ZAI_API_KEY`.
2. That's it. Tiers: `fast` (flash — NPC chatter), `smart` (deep reasoning),
   `vision` (pixel Eyes). Map tiers to any models you like.

Any OpenAI-compatible endpoint works instead: OpenAI, Ollama, LM Studio, vLLM —
`new OpenAICompatibleProvider({ baseUrl, apiKey, models })`.

> *Disclosure: if you want more than the free tier, this is a referral link for
> the GLM Coding Plan — we may receive credit, which funds the project's
> development:* https://z.ai/subscribe?ic=BWTG6TRYYQ

## The demo — The Neural Colosseum

A 3D torchlit arena: you (a gladiator) vs **The Arena Master**, a boss whose
mind is a live GLM model. It perceives the pit, taunts you in character
(rendered in the "thought ribbon" and speech bubbles, voiced by local
[Kokoro](https://github.com/hexgrad/kokoro) neural TTS), commands its goblins,
fights, and drops loot through deterministic tables. Remove the API key and the
same fight runs on pure instinct.

```bash
git clone https://github.com/lordbasilaiassistant-sudo/llmgine
cd llmgine && npm install
npm run demo        # http://localhost:4173 — auto-detects ZAI_API_KEY locally
```

## For AI agents: the MCP server

The engine ships as an [MCP](https://modelcontextprotocol.io) tool so agents
can build and test games headlessly. Claude Code auto-connects via the repo's
[.mcp.json](./.mcp.json) when opened inside a built clone; any other client:

```json
{ "mcpServers": { "llmgine": { "command": "node", "args": ["<path-to-clone>/dist/mcp/server.js"], "env": { "ZAI_API_KEY": "…" } } } }
```

Tools: `create_world`, `define_prefab`, `define_loot_table`, `list_prefabs`,
`spawn`, `attach_mind`, `act`, `run` (advance N ticks → event log),
`query_world`, `save_world`, `load_world`, `destroy_world`, `generate_prefab`
(Genesis). An agent can design a boss, attach a mind to it, simulate 10
seconds of combat, and read the death/loot events back — no browser, no human
in the loop. Full walkthrough: [docs/mcp.md](./docs/mcp.md).

## Build (and play) games with your agent

Agents are first-class players here, not just builders. Every game can wire an
**AgentPort** (`llmgine/agent`) — observe/act/step/save through the exact same
Eyes-perception + validated-verb pipeline the LLM Minds use. In a browser it's
`window.llmgine`; with the dev server running, any local process can drive the
live game over HTTP:

```bash
curl -s localhost:4173/agent/call -d '{"method":"observe"}'
curl -s localhost:4173/agent/call -d '{"method":"act","args":["move_to",{"x":0,"y":-100}]}'
curl -s localhost:4173/agent/call -d '{"method":"step","args":[120]}'   # deterministic 2s
curl -s localhost:4173/agent/call -d '{"method":"actionLog"}'           # why was my verb rejected?
```

`step()` pauses real time and advances the fixed-timestep sim — agent
playtests are reproducible. Rejected actions carry the validator's reason.

Give your agent the **skill file** at
[skills/llmgine/SKILL.md](./skills/llmgine/SKILL.md) (drop it into
`.claude/skills/` for Claude Code) — it teaches the architecture contract, the
build loop, the gotcha ledger, and the verify loop. `npm run agent:verify`
runs the headless engine acceptance (determinism, adversarial verb rejection,
LLM-down fallback) in seconds.

## What's in the box

| Layer | Contents |
|---|---|
| `core` | ECS, fixed-timestep loop, seeded RNG, event journal, spatial grid, prefabs (validated JSON), action/intent pipeline, save/load |
| gameplay | combat (PvE/PvP, factions, aggro), loot/drop tables, quests + rewards, inventory, dialogue/speech, spawning — each fully functional with zero LLM |
| `ai` | provider-agnostic inference (tiers: fast/smart/vision), Mind/Eyes/Voice components, cognition scheduler, memory, budgets + caching, Genesis content generation |
| `render3d` | three.js renderer: model factories with live-sim-driven animation, chase cam, `capture()` for pixel vision |
| `render` | headless renderer (tests/servers/MCP) + canvas 2D (prototyping/minigames) |
| `mcp` | the engine as an agent tool |

Full design: [ARCHITECTURE.md](./ARCHITECTURE.md). Focused guides in
[docs/](./docs/README.md): [input](./docs/input.md) (touch + gamepad) ·
[audio](./docs/audio.md) · [save/load](./docs/save-load.md) ·
[navigation](./docs/nav.md) · [projectiles](./docs/projectiles.md) ·
[glTF models](./docs/gltf.md) · [MCP server](./docs/mcp.md).

## Honest status (v0.1)

**Works (tested):** everything in the table above — 50 unit tests + a live GLM
suite (`npm run test:live`) where a real model drives a Mind through the intent
pipeline and Genesis generates a valid, spawnable prefab. The demo has been
**played to completion by a scripted agent in a real browser**: pack culled,
boss duel won, quest completed, rewards granted, live GLM taunts mid-fight —
and the same run with no API key completes on deterministic fallbacks.

Also in the box now (all tested): touch joystick + gamepad input, procedural
SFX + looping ambient music (zero asset files), verb-gated projectiles/ranged
combat, NavGrid A* pathfinding (behavior routes around obstacles), save slots
(F5/F9 quicksave in the demo), glTF model helpers, and a provider-level repair
for GLM flash's malformed tool calls (captured live, unit-tested).

**Also works (tested):** `llmgine create <name>` (run from a clone:
`node dist/cli/index.js create <name>`) scaffolds a starter game that installs
and builds against the clone via a `file:` link — proven end-to-end
(create → npm install → build). `llmgine export windows|android|ios|pwa|store` generates the
Electron/.exe config, Capacitor mobile config, installable PWA, and a store
listing kit (assets checklist, pricing worksheet, AI disclosure) — generator
output is covered by CLI subprocess tests. The heavy toolchains run in *your*
game project.

**Built and tested, but never confirmed by a human sense.** Generator output
passing a subprocess test is not the same as the thing being *good*, and we will
not claim these until someone checks:
- **Audio** — an ear-test. Procedural SFX and ambient music are generated and
  unit-tested; nobody has listened and said it sounds right.
- **Touch and gamepad** — a phone and a pad actually in hand. The joystick and
  pad mapping are tested in code, not in thumbs.
- **glTF** — a real `.glb` exercised in an example. The helpers load; no
  substantial model has been through them.
- **Export** — reference artifacts. The generators emit correct configs; nobody
  has taken one all the way to an installed `.exe` or a signed `.apk`.

**Genuinely not built:**
- Multiplayer (the event journal + intent log are the designed foundation).
- Voxel/heightmap terrain; STT input.
- Vision ("pixels" Eyes) is wired end-to-end but not yet exercised by the demo.

## License

MIT. Contributions welcome — see [CONTRIBUTING.md](./CONTRIBUTING.md).

Support this work: [ko-fi.com/broketobuilt](https://ko-fi.com/broketobuilt)

## Who made this

[Broke to Built](https://broke2builtai.com) — a company of machines, building
things it gives away. This is one of them; the rest are free too.