arc-world-engine-mcp
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@arc-world-engine-mcp Simulate current model and diff against last observation."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
ARC-AGI-3 World Model Lab
中文说明 | English
A research workbench for reconstructing ARC-AGI-3 games as explicit, executable world models. The project combines three layers:
a deterministic C11 actor/rule engine exposed through Python and MCP;
a lightweight Grid-SLAM and replay workbench for stable maps, masks, and trajectories;
a visual discovery harness for human gold masks, traditional tracking, and multimodal-model experiments over complete action animations.
This is an independent research prototype, not an official ARC Prize project or submission. It deliberately focuses on discovering and verifying game dynamics, not planning or leaderboard optimization.

Why this project exists
ARC-AGI-3 interactions are short, discrete, and visually exact. A useful agent should not have to retain a long stream of screenshots in hidden neural state. Instead, it should be able to construct an external model:
action animation
-> stable coordinates and candidate masks
-> action-conditioned object events
-> actor-owned event-condition-effect rules
-> deterministic simulation and observation comparisonThe engine is both a simulator and a constrained visual-description language. An agent can propose actors and rules, replay observations, inspect residuals, and revise the model without writing arbitrary native code.
Related MCP server: Meaning Model MCP Server
Current status
Area | Status | What works now |
C world engine | Usable prototype | Actors, static map, ARC colors 0-15, collision, topology transforms, state/model hashes, snapshots, traces, rendering, diffs |
ECE rule system | Usable prototype | Action/click/tick/collision/all-event triggers, validated condition expressions, multi-frame effect programs |
MCP interface | Usable prototype | Typed actor, topology, rule, snapshot, simulation, render, and observation-comparison operations |
Grid-SLAM | Experimental | Integer camera registration, typed grid-corner features, static/dynamic separation, status-bar filtering, object tracks |
Web workbench | Integrated | Real/demo backend, full animation playback, replay, map/object views, actor/rule editor, native debug trace |
Visual harness | Operational experiment | Full-sequence annotator, unified ROI, traditional bidirectional tracker, UniPixel/K3/Seed adapters and benchmarks |
Automatic ontology discovery | Open research | Candidate masks exist, but deciding actor boundaries and grouping still needs model/human hypotheses |
Full game reconstruction verifier | Partial | Engine can compare predictions with observations; automatic synthesis of complete game rules is not finished |
Planner | Out of scope | No gameplay planner is included yet |
World engine
world_engine/ contains the deterministic C11 kernel and Python bindings.
The static map is separate from dynamic actors. Every actor can own
event-condition-effect (ECE) rules.
Condition code is parsed with a restricted AST and compiled to a native postfix
program; Python eval is not used:
actor.visible == true and
(actor.color == 8 or (action.id == 6 and action.x >= actor.x))The validator checks actor references, property fields, booleans, ARC color ranges, viewport coordinates, and action fields before changing the model.
Effect code uses one validated operation per line and can expose intermediate frames:
frame 0: actor.recolor(8)
frame 1: actor.move(1, 0, block)
frame 2: actor.rotate_to(actor_12, 0, -1)rotate_to lets an agent use another actor as an explicit post-transform pixel
topology. This supports ARC-style rotations and reflections without making the
LLM manually rewrite the full frame.
MCP
Start the stdio MCP server:
arc-world-engine-mcpSet ARC_WORLD_MCP_TRANSPORT=streamable-http for an HTTP transport. The MCP
surface intentionally exposes typed cognitive operations rather than arbitrary
pixel writes.
Grid-SLAM workbench
The workbench processes every intermediate frame returned by one ARC action. Its current pipeline is:
detect typed cell-corner features and estimate an integer camera offset;
align frames before computing added/removed masks;
update a confidence-weighted static map while excluding dynamic cells;
match masks under translation to maintain object trajectories;
detect one-cell-wide, two-color monotonic status bars and remove them from ordinary object candidates and map landmarks.
When a level finishes, map and object tracking reset. Exact previous-level object templates remain available as cross-level priors. A full game reset also clears replay, action history, and Grid-SLAM state.
Run the demo
Requirements: Python 3.12+, a C compiler, Node.js 22+, and uv.
make -C world_engine test build
uv venv
uv pip install -e ".[dev]"
cd frontend
npm ci
npm run build
cd ..
python -m grid_slam.server --demo --game demo-gridOpen http://127.0.0.1:8765. To use a real public ARC environment, copy
.env.example to .env, set ARC_API_KEY, and run:
python -m grid_slam.server --game ft09 --operation-mode normalThe frontend is desktop-only by design.
Docker
docker build -t arc-world-model-lab .
docker run --rm -p 8765:8765 arc-world-model-labVisual discovery and summarization harness
vision_harness/ is a separate Python package. It imports action animations
from the workbench, builds one shared ROI across all frames, and stores one
human mask per target per frame. The browser annotator supports target
splitting, multi-label change types, traditional forward/backward propagation,
and side-by-side model overlays.
python -m pip install -e vision_harness
python -m arc_unipixel.import_workbench \
--url http://127.0.0.1:8765 --output vision_harness/data/real --limit 12
python -m arc_unipixel.annotator \
--data-root vision_harness/data --port 8788Open http://127.0.0.1:8788. Real replay frames, human annotations, model
weights, and provider outputs are intentionally excluded from this repository.
Experiment conclusions so far
The complete action animation matters. Reducing an interaction to two endpoint frames loses transient motion, blinking, occlusion, and causal grouping evidence.
A shared ROI is useful, but it is only a search window. It should not be treated as an object mask or supplied as semantic ground truth.
Traditional tracking is strong for exact rigid motion. Integer translation and color-map matching outperform general vision models on easy pixel-exact cases, but fail on deformation, split/merge, and ambiguous appearance changes.
UniPixel-7B can often describe the animation but is unreliable at exact ARC masks without domain adaptation. Natural-video object priors do not map cleanly to small, disconnected, discrete-color game actors.
Tool-assisted matrix reasoning is promising. Kimi K3 and Seed 2.1 Pro can use a native grayscale matrix tool to recover precise target topology, but generated origins and masks still require deterministic matching and review.
Semantic quality and mask quality must be measured separately. A model may correctly describe a target but emit no mask, or split one gold actor into several masks whose union is exact.
Incomplete gold can reverse model rankings. In the 12-sample endpoint benchmark K3 scored
0.8206mean endpoint IoU and Seed scored0.7986, but the difference is driven by one Seed mask-generation failure and by unlabelled status-bar changes that Seed detected. Excluding that failure, Seed's endpoint score is higher. These numbers must not be interpreted as a general model ranking.
The next evaluation revision should report semantic correctness, endpoint union, full temporal coverage, target grouping, grounding reliability, and unlabelled-change coverage as separate dimensions.
See vision_harness/README.md for protocols and vision_harness/docs/annotation-guide.md for the annotation format.
Tests
make -C world_engine test build
python -m pytest -q tests/unit
cd frontend
npm ci
npm run typecheck
npm run build
cd ../vision_harness
python -m unittest discover -s tests -p "test_*.py"Repository layout
world_engine/ C11 kernel, Python bindings, ECE DSL, MCP server
grid_slam/ map registration, masks, tracking, status bars, backend
frontend/ Preact/Vite desktop workbench
vision_harness/ annotation and multimodal visual-discovery experiments
tests/ engine and Grid-SLAM tests
docs/ architecture imagesLicense and attribution
Original project code is licensed under Apache License 2.0. Portions derived
from arcprize/arc-agi-3-benchmarking remain available under its MIT license;
the original notice is preserved in
LICENSES/ARC-PRIZE-BENCHMARKING-MIT.txt.
See NOTICE and THIRD_PARTY_NOTICES.md.
Available Tools
18 toolsadd_click_effectC
Apply an effect when ACTION6 clicks the source actor.
Inputs: effect (required), source_actor_id (required), args, target_actor_id. Returns: object
add_collision_effectB
Apply an effect when one actor collides with another actor or static map.
Inputs: effect (required), moving_actor_id (required), collider_actor_id (required), args, +1 more. Returns: object
add_directional_movementC
Bind ACTION1-4 to up/down/left/right movement for an actor.
Inputs: actor_id (required), collision_policy. Returns: object
add_ruleB
Attach actor ECE code. Use actor.value predicates and frame N effects.
condition_code supports and/or, for example:
actor.visible == true and (actor.color == 8 or action.id == 6).
ACTION6 coordinates are action.x and action.y. Effects use one
operation per line, for example frame 0: actor.recolor(8) followed by
frame 1: actor.rotate_to(actor_2, 1, -1). Invalid code is rejected with
an agent-facing line/column error. Legacy structured fields remain accepted.
Inputs: name (required), effect (required), trigger (required), args, +17 more. Returns: object
add_timer_effectC
Apply an actor effect on discrete action ticks, not wall-clock time.
Inputs: every (required), effect (required), source_actor_id (required), args, +2 more. Returns: object
compare_observationB
Compare a real ARC frame against the model's current rendered viewport.
Inputs: observed (required). Returns: object
create_actorC
Create a generic actor; cells are [local_x, local_y, optional_color].
Inputs: x (required), y (required), name (required), color (required), +6 more. Returns: object
create_snapshotB
Clone the complete native world for later rollback.
Inputs: name. Returns: object
create_worldB
Create an empty deterministic world using raw ARC colors 0-15.
Inputs: width (required), height (required), debug, background. Returns: object
get_world_stateA
Return rendered layers, actors, rules, hashes, trace, and debug history.
Inputs: no named parameters. Returns: object
load_static_mapA
Replace the static color and collision layers without creating actors.
Inputs: colors (required), blocked. Returns: object
merge_actorsC
Merge actors into the first ID using their rendered world topology.
Inputs: actor_ids (required). Returns: object
remove_actorA
Delete an actor and automatically remove ECE rules that reference it.
Inputs: actor_id (required). Returns: object
rollbackB
Restore a previously named native snapshot.
Inputs: name (required). Returns: object
set_viewportC
Set the integer camera origin and viewport, up to 64 by 64 cells.
Inputs: x (required), y (required), width (required), height (required). Returns: object
simulate_actionB
Advance one deterministic turn; ACTION6 uses viewport click coordinates.
Inputs: action (required), x, y. Returns: object
split_actorC
Split one actor by partitioning all of its local topology cells.
Inputs: groups (required), actor_id (required). Returns: object
update_actorC
Update only the provided generic actor state and transform fields.
Inputs: actor_id (required), x, y, color, +7 more. Returns: object
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 18 tools
Most tools have clear, distinct purposes, such as creating/updating/removing actors, adding effects by trigger type, and managing snapshots. The only minor overlap is between the general add_rule and the specific add_*_effect tools, but descriptions clarify their scopes.
Tool names mostly follow a consistent verb_noun snake_case pattern (create_actor, update_actor, remove_actor, add_rule). Minor deviations include merge_actors using plural while other actor operations are singular, and rollback lacking a noun object.
18 tools is on the higher side but each addresses a distinct aspect of the world engine: world setup, actor management, simulation, effects, and state inspection. No tool seems redundant, and the count is appropriate for the complex domain.
The tool surface covers the full lifecycle of world management: creating worlds, loading static maps, actor CRUD, simulation, rule/effect binding, observation comparison, and snapshot rollback. There are no obvious gaps that would prevent an agent from completing common tasks.
Maintenance
Related MCP Connectors
Your personal world model for any agent.
Deterministic reasoning stack for AI agents: simulate, decide & compute, plus cross-domain tools.
Event-sourced world model for multi-LLM agents: propose, validate, and read a shared state.
A living memory and world-building layer for your AI: cards, worlds, personas and agent dispatch.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables LLMs to interact with the Blocksworld Simulation to perform block manipulation actions, query real-time state, and verify multi-step plans. It bridges the Model Context Protocol with the simulation's REST API for AI planning research and agent development.MIT
- AlicenseNot gradedqualityBmaintenanceEnables an AI assistant to build and revise an explicit shared world model, then reason, create, narrate, remember, and give writing feedback through inspectable processes, concepts, and cuts. Supports general-purpose modeling, storytelling, agent/user memory, and ideation over one engine and graph while keeping estimates, interpretations, and observations distinct.MIT
- AlicenseNot gradedqualityBmaintenanceEnables LLM and agent hosts to create and manipulate structured game-art documents through typed deterministic MCP operations, render them to SVG/PNG, validate output, inspect visuals, and make targeted edits toward reproducible, game-ready assets.Apache 2.0
- AlicenseNot gradedqualityBmaintenanceEnables LLM agents to inhabit a shared simulated world where they can move, communicate, trade, leave signs, and try to survive under deterministic, data-driven rules that can be changed at runtime while spectators observe events and state.MIT