interact
Plays keyboard, mouse, touch, wheel, scroll, resize and clock steps on an open Three.js page, then returns state changes, console output, motion, sound, and optional screenshots for testing interactions.
Instructions
Plays the open page: keyboard, mouse, touch, wheel, scroll, resize and clock steps sent frame by frame (steps lists the kinds), and returns what changed in the state (its new values; the whole state when nothing changed), the console log/warn/info lines and errors the steps raised, a Motion line when the subject (window.__game.subject, or where the state puts its player) turned, or stopped while a key was still held, a Sound line with how many sounds the steps started on a page that makes sound, and with look: true look's picture and list after the steps, in the same reply. Every mouse and touch step says which named object in the scene is under it, and its distance. Use it to play an interaction the request names and read the state it leaves, to set a moment up ({ state }) and step to it ({ until }), or to keep a play as a check (save).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| game | No | The name the game was opened under with the `open` tool, to act on ("main" by default; a subagent uses its own). | |
| look | No | Also return `look`'s picture after the steps. | |
| save | No | Save this call (its steps and `expect`, which it needs) as a check of this name in game-tests.json when every Expect entry holds. A call with `save` plays its steps from a fresh reload, as `test` and the after-edit hook will rerun them: leading state steps become its setup. | |
| hover | No | After the steps, move the mouse to each named object's centre on screen and return what changed there after `frames`, one diff per name. | |
| steps | No | Steps, in order. { key, frames? } presses the key, holds it `frames` frames (default 1) and releases it; { key, down: true } holds it across steps until { key, down: false }; { mouse: click|dblclick|down|up|move, x, y }; { mouse: move, relative: [dx, dy] } (mouse look under pointer lock); { wheel: deltaY, x?, y? }; { resize: [w, h] }; { scroll: px | { progress: 0..1 } | { to: "#section" } } (scrolls the window; the reply says where, and the progress of Lenis, ScrollTrigger or drei ScrollControls when found); { touch: start|move|end|tap, x, y }; { wait: frames, draw? }; { action, ...args } for window.__game.act; { state: {...} } (sets observe()'s keys through window.__game.setState, then draws a frame: set the scene up first, as { state: { player: { x: 2, z: 0 } } }); { until: {...}, max? } (steps one frame at a time until the state holds, at most `max` frames, 600 by default, and says the frame or what it saw; its values as `expect`'s); { clock: { timescale?, paused? } } (sets the clock's time scale, or resumes (paused: false) or pauses real-time play; the clock reads 60 s at frame 0 on purpose, past any boot, so a page's own time starts there). A step's frames run without drawing except its last (an until draws none); draw: true on a step draws every frame, for a picture that builds up over frames (trails, progressive rendering). Each other step advances `frames` (default 1); `frames: 0` sends it on the same frame as the next step (two keys down at once, for a diagonal). | |
| trace | No | Up to 6 dotted paths into observe() read every frame of the steps (after leading state steps): one line per path with its first and last value, its peak or low and their frames, and the first frame it changed, as `player.y: 0 → peak 2.04 at frame 22 → 0 at frame 44; first changed at frame 2`. Tune by numbers: jump height and airtime, stopping distance, frames from key to motion. Read for the first 600 frames of the steps; a longer run keeps those. | |
| expect | No | Checked against the state after the steps, in this reply: { score: 1, "player.x": [2.9, 3.1] } gives `Expect: score 1 ✓ · player.x 2.97 ✗ expected 2.9 to 3.1`. A value is a number, string or boolean (equal), [min, max] (a range; two numbers are always a range), a list of other values (one of them, as ["banana", "star"]) or null (exists), keyed by dotted paths into observe() ("coins.0.x" for lists). | |
| frames | No | Frames to wait at each hovered object (default 1). | |
| record | No | true: play the steps (after leading { state } steps) in real time, each frame 1/60 s, and record them: a GIF in the run folder (480 px, each frame shown for its real time, HTML over the canvas included) and four moments of it in the reply, with the game's real frame intervals at a 60 Hz pace (median, p95, max; a slower frame reads as its own time, not a 60 Hz screen's 33 ms) on the GPU named; with look: true, look's picture too. 15 s at most; the clock is paused again after. Evidence of real-time play for an agent or a judge. |