Allows interaction with a running Unreal Engine 5 game, providing virtual gamepad/keyboard/mouse input injection, screen capture, live logs, UI inspection and clicking, game state queries, and console command execution.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@ue5-gameplay-mcp Move the character forward with the gamepad, then observe what’s on screen."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An MCP server that plays a running Unreal Engine 5 game. It accepts virtual gamepad, keyboard, and mouse input and returns screen capture, log lines, and UMG status.
This is not an engine plugin; it is a client. Two plugins already handle the in-engine work, and each holds its own port. This server dials into both and exposes them as a single tool interface.
Plugin | Port | Features |
RemoteConsole2 | 10101 | Gamepad/keyboard/mouse injection via |
RemoteCapturePlugin | 10102 | JPEG/PNG screen capture (works in PIE and packaged builds) ships with |
A small amount of C++ was added to both plugins specifically for this server. They remain independent and keep their own protocols. The additions are backward compatible, so even older clients can still communicate with a rebuilt game:
FImageMeta.SourceSize — the game's back-buffer size. It is packed into a previously reserved area to keep the struct at 24 bytes, so the client can map a specific point on a downscaled capture back to actual window pixels.
CMD_GET_GAME_STATE (520) and IRemoteGameStateProvider — more on this below.
cd ue5_gameplay_mcp
uv syncMCP Python SDK v2 (mcp.server.MCPServer) is required.
Related MCP server: VERA MCP Server
Start the game first. In this project, a standalone game is run from the editor binary because the Game target exits immediately in a project that has not been cooked:
"C:/Program Files/Epic Games/UE_5.8/Engine/Binaries/Win64/UnrealEditor.exe" "<PATH>/MyProject.uproject" -game -windowed -resx=1280 -resy=720 -log -nosplashNext, register the server. The .mcp.json in the project root already does this, so Claude Code will pick it up automatically. The manual command equivalent is:
claude mcp add ue5-gameplay -- uv run --directory <PATH>/ue5_gameplay_mcp -m ue5_gameplay_mcpThe server uses a lazy connection, so the actual startup order does not matter. If it starts before the game does, it will connect on the first tool call.
Options: --host, --console-port, --capture-port, --format, --quality, --max-size, --grid-step, --transport streamable-http --mcp-port 14102.
Session — game_connect, game_status, game_reset_input
Observation — game_observe, game_state, game_log, game_wait_for_log
Actions — game_pad, game_pad_sequence, game_key, game_mouse, game_console, game_time_scale
UMG — game_ui_dump, game_ui_click, game_ui_focus
An agent round trip takes several seconds, but the game runs at 60 Hz. Since frame-by-frame actions are unrealistic, the design works as follows:
All action tools take a duration and perform a press / hold / release sequence locally, paced to the communication speed. One round trip carries one intent, not one frame.
Action tools observe by default. game_pad(ly=1.0, duration=0.5) moves forward and returns the resulting frame. This takes half as many round trips as doing the action and the observation separately.
game_pad_sequence packs an entire combo into one call when the timing of the inputs matters more than checking between them.
game_time_scale(0.2) buys in-game time when precision at a particular moment is required.
hold=True keeps the input applied between turns so the character keeps moving while the agent is thinking. game_reset_input clears it.
game_state returns the level, world time, pause/time dilation, player pawn transform, velocity, movement mode, camera, and the distance and normalized screen position of the nearest actor. These are the same 0–1 coordinates game_mouse receives, so you can immediately aim at a target you find in the status report. The cost is a fraction of image processing, and it will not misread HUD numbers.
game_observe(state=True) folds this into what an observation, and `game_pad(..., state=True)' folds it into an action, so movement and verification still take only one round trip.
On a real map, most nearest actors are background objects, so the report also includes class_counts (a survey of all objects within the radius). Read it once, and you can narrow it down with class_filter="Enemy".
The built-in report needs no game-side query. For anything only that project knows — health, score, quest flags, etc. — implement IRemoteGameStateProvider (Plugins/RemoteConsole2/Source/RemoteConsole2/RemoteGameState.h) on any actor and return the string of a JSON object:
FString AMyGameMode::GetRemoteGameState_Implementation()
{
return FString::Printf( TEXT("{\"score\":%d,\"wave\":%d}"), Score, Wave );
}Since it's a BlueprintNativeEvent, it can be overridden even in Blueprint-only projects. All returned values are stored under custom, keyed by actor name. Providers are collected regardless of the distance filter, so even a scorekeeper handled at the origin will send a report. Text that is not valid JSON is not discarded; it is passed through as its raw string, so even simple Printf debugging while starting up is helpful.
game_ui_dump + game_ui_click — exact and fast, but only knows widgets registered through UMG. Games with a custom Slate UI get nothing back, and the tool explicitly says so instead of hanging.
Pad navigation — game_pad(buttons=["DOWN"]), game_pad(buttons=["A"]). Works with almost any game.
Look & click — game_observe(grid=True) overlays a labeled 0–1 coordinate grid on the capture. Read the target from the image and pass the same numbers to game_mouse(x=..., y=...). It is independent of user resolution and works no matter how the UI is built.
Sticks follow UE's specification: ly=+1 is forward. (The protocol flips the Y axis, but our code flips it back, so the tool API matches what the game's own axis mapping means.)
Mouse coordinates are normalized to 0–1 with the top-left as origin, and are translated to pixels using the game's actual back-buffer size. Since the capture reports its resized dimensions, this source size is probed separately.
game_observe returns only new log lines since the previous observation, so the same output is never resent, even during a long session.
When hosting the game from the editor binary, console commands are routed through Python. FGameAppInterface::ExecConsoleCommand dispatches to the implementation of IConsoleCommandExecutor [0]; if the editor's Python plugin is loaded, that slot is Python instead of Cmd. So a plain stat fpscomes back as aSyntaxError. On first use the server probes this, and if it detects the problem, it wraps the command with unreal.SystemLibrary.execute_console_command. Packaged builds have no Python executor, so this solution is not needed. You can also override it with game_console(via="cmd")`.
Screen clicks assume the capture fills the entire game window. This is true with -game -windowed. In a letterboxed fullscreen mode, the reported source size includes the black bars, so the mapping is off.
game_state traverses every actor in the level on each call. At normal map scale this is fine, but a streaming open world would probably need a spatial query instead of TActorIterator.
uv run test/smoke_test.pyCommunicates directly with the game and outputs smoke_*.jpg, allowing you to check the capture and grid overlays visually.
uv run test/mcp_client_test.pyStarts the server via standard input/output (stdio) as a real MCP client, and runs and tests all tools, including error paths.
Connect to the running game and report what is on the other end.
Call this first, or after the game has been restarted. reconnect=True
drops the existing sockets and dials again.
| Name | Required | Description | Default |
|---|---|---|---|
| reconnect | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of disclosing behavior. It does disclose a key behavioral aspect: that `reconnect=True` 'drops the existing sockets and dials again.' But it does not describe what happens on a normal call (when reconnect is False) if an existing connection is already present—whether it reuses it, errors, or silently changes state. It also omits any mention of prerequisites, side effects on other game tools, or authentication requirements. The description adds some behavioral context but leaves significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise—two sentences—and every sentence contributes value. It front-loads the purpose, then provides usage timing, then explains the parameter behavior. There is no redundancy or filler, making it an exemplar of efficient communication.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema, the description does not need to detail return values. It covers the core aspects: purpose, when to use, and the effect of its only parameter. However, it could be more complete on edge cases, such as behavior when no game is running or when already connected without reconnect. Still, for a simple connection tool with one parameter, it provides sufficient context for an agent to call it correctly in the common flow, meriting a slightly above-average score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does explain the sole parameter `reconnect`: 'reconnect=True drops the existing sockets and dials again.' This adds meaning beyond the bare schema, which only provides a title and default. However, the description does not clarify the default behavior (when reconnect is False) or what happens if the tool is called without an existing connection. Thus, semantics are only partially covered, warranting a middle score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('connect') and resource ('running game') and an action ('report what is on the other end'). It clearly establishes the tool's role as an initial connection step, and 'Call this first' distinguishes it from sibling tools that likely operate post-connection. However, 'report what is on the other end' is somewhat vague and could be more explicit about what the report contains, preventing a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit timing guidance is provided: 'Call this first, or after the game has been restarted.' This clearly indicates when the tool should be invoked. It also explains the behavior of the `reconnect` parameter for re-establishing a connection. However, it does not mention when *not* to use it or point to alternatives, but the purpose and timing are clear enough for an agent to select it correctly in most cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run a console command and return the log lines it produced.
Useful beyond cheats: stat fps, showdebug ai, and especially
getall <ClassName> <PropertyName> to read gameplay values out of the game
until a structured state API exists.
via picks the executor and should stay on "auto": when the game is hosted
by the editor binary the console routes through Python, which auto mode
detects once and works around. Force it with "cmd" or "python" if needed.
| Name | Required | Description | Default |
|---|---|---|---|
| via | No | auto | |
| settle | No | ||
| command | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It does disclose important runtime behavior: the 'via' parameter's auto mode and the Python routing workaround when the game is hosted by the editor binary. However, it does not mention that running arbitrary console commands may mutate game state, have side effects, or require special permissions—significant gaps for a command-execution tool with no annotation hints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description front-loads the primary purpose, then provides usage examples, and finally explains the 'via' parameter. It is moderately long but every section adds meaningful context. The structure is logical and scannable, though the third paragraph on 'via' could be tightened without losing content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main usage scenarios, gives concrete command examples, and explains the critical 'via' routing logic. Since an output schema is present, return-value details are not required here. However, it omits the semantics of 'settle' and does not warn about potential state changes from console commands, which an agent would need to safely invoke the tool in all contexts.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It thoroughly explains 'via' (auto vs. cmd vs. python) and why 'auto' should be the default, which adds value beyond the bare schema. But it says nothing about 'settle' (a numeric parameter with a default of 0.15) or the expected format of 'command' beyond examples. With one parameter entirely undocumented, the description fails to fully compensate for the zero-coverage schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear verb-resource pair: 'Run a console command and return the log lines it produced.' It distinguishes itself from siblings like game_state or game_log by referencing command execution and log output, and it provides concrete examples ('stat fps', 'getall') that sharpen its purpose as a generic command runner rather than a structured-state reader.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context: it is useful 'beyond cheats' and specifically for reading values 'until a structured state API exists,' implying it is a fallback when game_state is not sufficient. However, it does not explicitly name sibling alternatives (e.g., game_state) or state when not to use it, so the guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tap a keyboard key for duration seconds.
key is a single character ("w", "e") or a name: ENTER, ESC, SPACE, TAB,
SHIFT, CTRL, ALT, UP/DOWN/LEFT/RIGHT, F1-F12, TILDE.
| Name | Required | Description | Default |
|---|---|---|---|
| key | Yes | ||
| observe | No | ||
| show_ui | No | ||
| duration | No | ||
| max_size | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It explains the key format and duration, but does not mention side effects, requirements (e.g., an active game connection), or what happens after the key is tapped. It also does not explain how the observe, show_ui, and max_size parameters affect behavior. This is a significant gap for a tool that performs an input action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, consisting of two sentences, and is well-structured with a clear format for key names. Every sentence provides value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With five parameters, no annotations, and no output schema, the description is incomplete. It does not explain the purpose of the observe/show_ui/max_size parameters, nor does it clarify prerequisites or return behavior. More context is needed for an agent to use this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description covers only two of five parameters: key (with valid values) and duration (in seconds). The other three parameters (observe, show_ui, max_size) are entirely undocumented. Since schema description coverage is 0%, the description must compensate but does not, leaving most parameters unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: 'Tap a keyboard key for `duration` seconds.' The verb 'tap' and resource 'keyboard key' are specific, and the mention of valid key names (ENTER, ESC, etc.) distinguishes it from siblings like game_pad (gamepad) or game_mouse. It is immediately clear what this tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for keyboard input by specifying a keyboard key, but it does not explicitly state when to use it over alternatives like game_pad or game_mouse. No exclusions or conditions are given. The guidance is adequate but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
New log lines since the last read. pattern is a regex filter.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| since | No | ||
| pattern | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It states 'since the last read' which implies stateful consumption (reading marks progress), and mentions the regex filter. However, it does not clarify whether logs are cleared, how 'last read' is tracked, or any side effects of calling this tool repeatedly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at two sentences, front-loading the core purpose. It is efficient and free of fluff, though it sacrifices necessary detail. This is good conciseness, but for a tool with three parameters, it leans toward under-specification rather than optimal economy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has three parameters and an existing output schema, the description still fails to explain the semantics of `limit` and `since` or how they interact with the 'last read' pointer. It does not specify whether `since` is a timestamp or line number, nor the default behavior when null. The output schema exists, so return value details are not required, but the parameter semantics and stateful behavior are essential and missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for all parameters. Only `pattern` is explained as a regex filter. The meanings of `limit` and `since` are left entirely to the schema defaults and titles, providing no additional context about their units, semantics, or interactions with the 'last read' behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns new log lines since the last read, which is a specific verb (read) and resource (game logs). It is not a tautology and is distinguishable from siblings like game_state or game_observe, though it doesn't explicitly name an alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a hint that it is used to fetch fresh log lines, but provides no explicit when-to-use or when-not-to-use guidance. There is no mention of alternatives such as game_wait_for_log for waiting on logs or game_observe for observations, leaving the agent to infer the appropriate context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Move or click the mouse at normalized screen coordinates.
x/y are 0..1 with the origin at the top left -- exactly the numbers printed
on a game_observe(grid=True) capture, so you can read a target off the
image and pass it straight in. They are converted to pixels here using the
game's real resolution.
action: click, down, up, double, move, setpos, wheel button: L, M, R, T1, T2
This is the fallback that works regardless of how the game builds its UI, when game_ui_click cannot see the widget tree.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| grid | No | ||
| wheel | No | ||
| action | No | click | |
| button | No | L | |
| observe | No | ||
| show_ui | No | ||
| duration | No | ||
| max_size | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry behavioral burden. It does explain the normalized coordinate system and pixel conversion, plus the available actions/buttons. However, it omits any mention of side effects (e.g., whether this moves the OS cursor vs. simulated game input), the effect of 'observe'/'duration', or failure modes. The core behavior is described, but not comprehensively.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured logically: first the operation, then the coordinate details, then the parameter lists, then usage context. Every sentence adds information, with no fluff. It could be slightly condensed, but the length is justified given the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a 10-parameter tool with no output schema and no annotations. The description explains only the essential core (x/y and action/button) but leaves many parameters undefined, no return value description, and no explanation of how 'observe' or 'show_ui' affect output. An agent would likely struggle to invoke this correctly for anything beyond basic clicks.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains x/y normalization and lists action/button values, covering only 4 of the 10 parameters. Parameters like grid, wheel, observe, show_ui, duration, and max_size are entirely unexplained, leaving significant ambiguity for an agent trying to use them correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb ('Move or click') and resource ('the mouse'), and immediately distinguishes itself from the sibling tool 'game_ui_click' by positioning itself as the fallback for when the widget tree is unavailable. This explicit differentiator prevents confusion in tool selection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The final sentence clearly defines when to use this tool: 'when game_ui_click cannot see the widget tree.' It also explains the coordinate system and conversion, giving enough context for an agent to decide between this and alternatives. The 'when-not' is implied by the fallback framing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Look at the game: a screenshot plus the log lines since the last look.
grid=True overlays a labelled 0..1 coordinate grid -- turn it on when you
intend to click something with game_mouse. show_ui=False hides the
HUD/UMG overlays. log_pattern is a regex filter over the new log lines.
state=True folds in game_state, so one call answers what the world looks
like and what it actually contains.
| Name | Required | Description | Default |
|---|---|---|---|
| log | No | ||
| grid | No | ||
| image | No | ||
| level | No | ||
| state | No | ||
| quality | No | ||
| show_ui | No | ||
| max_size | No | ||
| log_limit | No | ||
| log_pattern | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full responsibility. It discloses that logs are 'since the last look' (incremental), that grid overlays a coordinate system, that show_ui hides overlays, and that state folds in game_state. It does not mention read-only status, rate limits, or side effects, but the 'look' verb implies non-destructive. Key behavioral nuances are surfaced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact: a one-line purpose followed by four backtick parameter hints. The essential information is front-loaded, and each sentence adds value. It avoids redundancy and reads naturally. Minor gap: it could include a note about output format, but overall it is well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 10 optional parameters, no output schema, and no annotations, the description is only partially complete. It explains the primary behavior and four parameters, but does not address quality, max_size, log_limit, or level, nor what the returned screenshot/log looks like. For an observation tool with many knobs, more detail would be expected.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must add meaning to parameters. It explains grid, show_ui, log_pattern, and state, but leaves log, image, level, quality, max_size, and log_limit unexplained. The core parameters affecting output and interaction (grid for clicks, show_ui for overlay) are covered, but the rest are ambiguous to the agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns a screenshot plus log lines, combining visual and textual state. It distinguishes itself from siblings like game_log (logs only) and game_state (state only) by merging both, and hints at optional state inclusion. The verb 'observe' and resource 'game' are specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit usage hints for the grid parameter ('turn it on when you intend to click something with game_mouse') and explains when to use state=True. It implies this tool replaces separate calls to game_log and game_state, but does not explicitly say when not to use it. The guidance is concrete and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Press pad buttons and push sticks for duration seconds, then observe.
buttons: any of A B X Y UP DOWN LEFT RIGHT L1 R1 L2 R2 L3 R3 SELECT START HOME TOUCHPAD (aliases: CROSS/CIRCLE/SQUARE/TRIANGLE, LB/RB/LT/RT, BACK/OPTIONS/MENU). sticks: lx/ly left stick, rx/ry right stick (camera), l2/r2 analog triggers. Range -1..1, UE convention -- ly=+1 is forward. hold: keep this input applied after the call returns, so the character keeps moving between your turns. Clear it with game_reset_input.
Example -- run forward for half a second and look at the result: game_pad(ly=1.0, duration=0.5) Example -- jump while running: game_pad(buttons=["A"], ly=1.0, duration=0.3) Example -- move and get exact positions back, not just a picture: game_pad(ly=1.0, duration=0.5, state=True)
| Name | Required | Description | Default |
|---|---|---|---|
| l2 | No | ||
| lx | No | ||
| ly | No | ||
| r2 | No | ||
| rx | No | ||
| ry | No | ||
| grid | No | ||
| hold | No | ||
| state | No | ||
| buttons | No | ||
| observe | No | ||
| show_ui | No | ||
| duration | No | ||
| max_size | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It clearly discloses input conventions (range -1..1, UE forward = +1), the hold behavior (keeps input after return, cleared by game_reset_input), and the state parameter's effect (returns exact positions instead of a picture). It also implies that without hold, inputs are released after the call. No annotation contradiction exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core action and result, followed by organized parameter details and three concrete examples. Every sentence serves a purpose—no fluff. It is substantial but well-structured, making it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 14 parameters, no output schema, and no annotations, the description is remarkably complete. It covers input specification, conventions, hold lifecycle, and how to get positional data. The examples give direct call patterns. The few omitted parameters are minor and do not hinder an agent from using the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate for 14 parameters. It thoroughly explains the core ones: buttons (with full list and aliases), sticks (lx/ly, rx/ry, l2/r2), range, hold, state, and demonstrates duration via examples. However, it omits explanations for grid, observe, show_ui, and max_size, which remain ambiguous. This is a minor gap given they are likely less critical for typical usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear, specific verb ('Press pad buttons and push sticks') and a resource ('pad'), and explicitly ties it to a duration and observation step. It distinguishes itself from sibling tools like game_pad_sequence (sequence vs single action), game_key/game_mouse (different input types), and game_observe (observation without input) by stating both action and result.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
While not explicitly naming alternatives, the description provides multiple examples covering typical scenarios (run forward, jump while running, and requesting exact state) and explains the hold parameter with a clear instruction to clear it via game_reset_input. It implicitly guides when to use this versus other tools, but does not explicitly state 'use this instead of X when...'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run several pad states back to back in a single round trip.
Each step is {"buttons": [...], "sticks": {"ly": 1.0}, "duration": 0.2, "gap": 0.05}. Use this for combos and for anything where the timing between inputs matters more than looking in between.
Example -- a three hit combo: [{"buttons": ["X"], "duration": 0.1, "gap": 0.25}, {"buttons": ["X"], "duration": 0.1, "gap": 0.25}, {"buttons": ["Y"], "duration": 0.1}]
| Name | Required | Description | Default |
|---|---|---|---|
| steps | Yes | ||
| observe | No | ||
| show_ui | No | ||
| max_size | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the 'single round trip' behavior, which is useful, and details the step structure. However, it does not disclose potential gaps (e.g., partial execution on failure), error behavior, or how it interacts with other input tools. The information is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized: it starts with a one-line purpose, then defines the step format, gives a usage context, and ends with a concrete example. It is not overly verbose, and the example is efficient and illustrative, earning its place. The front-loading of purpose is effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 4 parameters and no output schema, the description is incomplete. It explains 'steps' thoroughly but omits the purpose and behavior of 'observe', 'show_ui', and 'max_size'. It also does not describe what the tool returns or how the sequence result is delivered. This leaves agents uncertain about side effects and required configurations, making the description insufficient for correct invocation in many scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains the 'steps' parameter in detail, including the exact JSON structure for each step and an example. However, the schema coverage is 0%, and the description fails to document the other three parameters (observe, show_ui, max_size). It compensates partially but leaves significant gaps, especially for optional parameters that affect the round trip.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs 'several pad states back to back in a single round trip' and provides a concrete example of a three-hit combo. It distinguishes itself from sibling tools like game_pad by emphasizing sequence and timing, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly recommends using the tool 'for combos and for anything where the timing between inputs matters more than looking in between,' providing clear context for when it is appropriate. It implies that for cases requiring observation between inputs, one would use separate calls, though it does not explicitly name the alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Release every held button, centre the sticks, lift the mouse buttons.
Use this after hold=True actions, or any time the character seems stuck
walking into a wall.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the exact physical actions performed (release, centre, lift) and implies the effect of unsticking the character. While it does not mention side effects or broader scope, the reset behavior is clear and sufficient for this simple tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences, with the core action front-loaded in the first sentence and a crisp usage note in the second. Every word earns its place with no redundant or vague phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and an output schema present, the description fully addresses how to use the tool, when to use it, and what it accomplishes. Even includes a troubleshooting scenario ('stuck walking into a wall'). Nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so there is no parameter information to add. The description correctly contains no parameter details, and the schema coverage is trivially 100%. The baseline score for zero-parameter tools is 4, and no deduction is warranted.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action on a clear resource: 'Release every held button, centre the sticks, lift the mouse buttons.' This distinguishes it from sibling tools like game_key, game_mouse, and game_pad, which handle individual inputs. It is not a tautology and clearly communicates the tool's function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this after `hold=True` actions, or any time the character seems stuck walking into a wall.' This gives clear, concrete when-to-use conditions. However, it does not state when not to use it or mention alternative tools, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Structured world state -- no screenshot, cheap to call often.
Returns the level name, world time and pause/dilation, the player pawn's
transform, velocity and movement mode, the camera, and the nearest actors
with their distance and their normalized screen position (the same 0..1
coordinates game_mouse takes, so you can aim at what you find here).
class_filter keeps only actors whose class name contains that substring,
e.g. "Enemy". radius is in centimetres. Games can add their own values --
health, score, quest flags -- by implementing IRemoteGameStateProvider on an
actor; those show up under "custom".
Prefer this over reading numbers off the screen: it is exact, and it costs a fraction of the tokens an image does.
| Name | Required | Description | Default |
|---|---|---|---|
| actors | No | ||
| radius | No | ||
| max_actors | No | ||
| class_filter | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It discloses that the call is cheap and can be made often, and it details the exact return contents, including custom provider values. It does not mention side effects or failure modes, but as a read-only state query, it is sufficiently transparent and contradicts nothing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with a summary of the tool's essence. It then details return values, parameter semantics, and a usage recommendation, with no wasted sentences. Formatting with backticks and a clear logical flow makes it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the major return fields and custom provider values, and an output schema exists. It likely suffices for an agent to understand what to expect. Minor gaps include lack of error or edge-case handling, but these are not critical for a state query tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains `class_filter` (substring filter) and `radius` (in centimetres) explicitly. It does not explain `actors` or `max_actors`, but they are inferable from names and defaults. Since schema coverage is 0%, the description only partially compensates, leaving two parameters underspecified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns a structured world state, enumerating specific elements: level name, time, player transform, velocity, movement mode, camera, and nearby actors. It distinguishes itself from a screenshot and explicitly frames itself as a precise alternative to reading numbers off the screen, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit guidance to prefer this tool over reading numbers off the screen, indicating a use case. It also references game_mouse coordinates, implying how to use the output for aiming. However, it does not name specific alternative tools or state when not to use this one, leaving some room for interpretation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Connection state, current level, and any input still being held.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of disclosing behavioral traits such as side effects or safety. It only lists the data returned and does not state whether the call is read-only, modifies any state, or has any side effects. This is a significant gap for a status tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no fluff. It is front-loaded with the core status elements. It earns its place, though it could be more explicit about being a read-only status query.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters) and the existence of an output schema, the description does not need to explain return values. However, it omits any guidance on prerequisites, such as requiring an established connection (relevant given sibling game_connect). This leaves minor ambiguity but is largely sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema fully covers parameter semantics by definition. The baseline for 0 parameters is 4. The description adds no parameter information, but none is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly indicates the tool reports status information: connection state, current level, and held input. The verb is implied (returns/reports), and the resource is specifically game status. It does not differentiate from siblings like game_state, but the purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention game_state, game_observe, or any conditions for selection. There is no context about prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Slow down or speed up the world (slomo). 1.0 is normal, 0.0 freezes it.
Drop to 0.1-0.3 when you need to place an action precisely; the game keeps running but each of your turns costs far less game time.
| Name | Required | Description | Default |
|---|---|---|---|
| scale | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosure. It explains the effect of scale values and the consequence that turns cost less game time, which is beyond the name. However, it does not disclose potential side effects like whether the change is global or persistent, or any latency implications, leaving some ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the core semantics and then adding a practical usage tip. No filler or repetition; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with one optional parameter, and the description covers its effect and a use case. It doesn't explain the return value, but an output schema exists, so that is not required. It could mention the scope of the change (e.g., affects all players or just the session) for full completeness, but given its simplicity, it is largely adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides no description for the 'scale' parameter, but the description compensates by defining the value semantics: 1.0 normal, 0.0 freeze, and recommending a slow-motion range of 0.1-0.3. This adds meaning well beyond the schema's bare number type and default.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool's function—slowing down or speeding up the world, likening it to the `slomo` command—and defines the scale semantics (1.0 normal, 0.0 freeze). This clearly distinguishes it from sibling game-control tools like game_pad or game_key, which manipulate input rather than time.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a specific use case: dropping to 0.1-0.3 for precise action placement, noting that the game continues but turns cost less game time. It does not explicitly mention when not to use it or alternative tools, but the guidance is concrete and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Click a UMG widget by name. action: click, press, release, hover, unhover.
Fast and exact when it works; use game_ui_dump first to learn the names.
| Name | Required | Description | Default |
|---|---|---|---|
| action | No | click | |
| widget | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions 'Fast and exact when it works' which hints at reliability but doesn't disclose side effects, whether it can be destructive, or what happens on hover/release. It does not clarify if it modifies game state or if it requires a connected game session. Minimal behavioral context beyond performance.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise: two short sentences. The primary purpose is front-loaded, and the usage tip about game_ui_dump is placed as a secondary note. Every sentence adds value with no redundant text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core usage and mentions the prerequisite dump tool. However, it lacks details on error handling, interaction with game_connect, or specifics about how the action parameter affects behavior. Given the presence of an output schema, return format is not needed, but for a tool with multiple actions and a dependency, more context would help.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must clarify parameters. It lists the valid actions for the action parameter (click, press, release, hover, unhover), which adds meaning beyond the schema. However, it does not explain the widget parameter format (e.g., exact path, case sensitivity) or any constraints. Partial compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool clicks a UMG widget by name, with a specific verb and resource. It lists distinct actions (click, press, release, hover, unhover) which differentiates it from generic input tools like game_mouse or game_pad. However, it does not explicitly contrast with game_ui_focus, which might also interact with UI widgets, so it slightly lacks full sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives a clear prerequisite: use game_ui_dump first to learn widget names. This helps the agent know how to proceed. It does not state when to use this over alternatives like game_ui_focus or game_mouse, nor does it provide exclusions (e.g., if the widget is not clickable). The 'Fast and exact when it works' implies potential failure but does not describe fallback paths.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Dump the UMG widget tree. mode: all, button, debug.
Only sees widgets registered through UMG. A game with custom Slate UI will come back empty or error -- that is expected, fall back to the pad or to game_observe(grid=True) plus game_mouse.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | button |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the tool's limitation (only UMG widgets) and the expected error/empty result for Slate UI, which is valuable behavioral context. However, it does not explain the effect of each mode (all, button, debug) on the output, and there's no mention of side effects or required permissions, though dump operations are generally benign.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is terse and front-loaded with the main action: 'Dump the UMG widget tree.' It immediately lists the modes, then adds a crucial caveat about Slate UI and fallback options in the second sentence. Every sentence serves a purpose with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers the tool's purpose, limitations, and fallbacks, it omits the semantics of the three modes. An agent would need more detail to decide which mode to use (e.g., what 'debug' returns vs. 'all'). The output schema exists but is not described, and a dump tool's output could vary significantly by mode. Overall, it's not fully complete without mode behavior explanations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It lists three possible mode values ('all', 'button', 'debug') but does not explain what each mode does or how they differ. This provides some guidance over the schema's bare string type, but the agent still lacks semantic understanding of each mode. The default 'button' is set, but its meaning is unclear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear verb ('Dump') and resource ('UMG widget tree'), making the purpose unambiguous. It also differentiates itself from sibling tools by explicitly noting that Slate UI won't work and suggesting fallbacks to pad, game_observe, and game_mouse, so an agent can tell when this tool is appropriate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use guidance by stating it only sees UMG widgets and explains the expected failure on custom Slate UI. It also names specific alternative tools (pad, game_observe, game_mouse) as fallbacks, so the agent knows what to do if this tool fails.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Set the game's input focus. mode: game, ui, game_and_ui, window.
Bring the window forward and hand focus back to the game with mode="game" if input seems to be going nowhere.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | game | |
| widget | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full behavioral burden. It does state that the tool can bring the window forward and return focus to the game, which is a useful side effect. However, it omits potential side effects, prerequisites, or reversibility details, leaving some uncertainty for a focus-altering operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact at two sentences, with the core action stated first. The second sentence adds a practical troubleshooting hint. No unnecessary fluff or redundancy is present.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, the description covers the primary purpose and one use case well, but it fails to address the 'widget' parameter and does not describe return values (though an output schema exists). Without annotations, this leaves the tool less than fully specified for all scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains the possible values for the 'mode' parameter ('game, ui, game_and_ui, window') but says nothing about the 'widget' parameter, leaving it completely undocumented. This is a significant gap for an agent needing to pass a widget.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sets the game's input focus and enumerates four distinct modes. This makes its purpose unambiguous and distinguishes it from input-sending tools like game_key or game_mouse, which trigger actions rather than alter focus.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a concrete usage trigger: 'Bring the window forward and hand focus back to the game with mode="game" if input seems to be going nowhere.' This tells the agent when to reach for this tool, albeit without explicitly comparing it to sibling alternatives like game_reset_input.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Block until a log line matches pattern, or the timeout expires.
This is the assertion primitive for automated tests: fire an action, then wait for the line the game prints when the thing you expected happened.
| Name | Required | Description | Default |
|---|---|---|---|
| pattern | Yes | ||
| timeout | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the tool blocks (synchronous) and that a timeout expires, but does not mention what happens on timeout (error vs. return value), whether log lines are consumed, or any side effects. Given that it's a harmless wait operation, the core behavior is clear, but error handling and return semantics are left to the output schema. Not as transparent as it could be.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core functionality in the first sentence, and the second provides valuable usage context. There is no filler or repetition. Every sentence earns its place, making it efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, and an output schema exists, so return values are likely covered. The description explains what it does and when to use it. It does not mention any prerequisite (e.g., game_connect) or error conditions, but for a straightforward wait/assertion tool, the provided information is largely sufficient. A minor gap is the lack of timeout behavior details, but that can be inferred or covered by the output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains 'pattern' as the pattern to match against log lines, and implicitly 'timeout' as the duration via the first sentence. It does not specify pattern syntax (e.g., regex) or default values, but it adds enough semantic meaning for an agent to understand what each parameter does. This is a strong effort given the schema provides no descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Block until a log line matches `pattern`, or the timeout expires.' This is a specific verb (block/wait) and resource (log lines). It further distinguishes itself from sibling tools by positioning as 'the assertion primitive for automated tests,' which separates it from game_log (likely read-only log retrieval) and other action tools. The purpose is unambiguous and well-differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear usage context: 'fire an action, then wait for the line the game prints when the thing you expected happened.' This explicitly tells the agent when to use the tool (after triggering an action, to assert expected behavior). It does not name alternative tools or explicitly state when not to use it, but the context is sufficient for typical selection. No misleading guidance is present.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0game_connectgame_consolegame_keygame_loggame_mousegame_observegame_padgame_pad_sequencegame_reset_inputgame_stategame_statusgame_time_scalegame_ui_clickgame_ui_dumpgame_ui_focusgame_wait_for_logScored across 16 tools
Each tool targets a distinct interaction mode: connection, observation, state, input (pad/key/mouse/UI), logging, console, time scaling, and focus. Even overlapping actions like game_mouse vs game_ui_click are explicitly differentiated by target (screen coordinates vs widget tree) and fallback behavior.
All tools follow the predictable 'game_' prefix with an action verb (connect, log, observe, state, pad, key, mouse, console). The sub-group 'game_ui_*' maintains consistency with a clear sub-domain. No mixed conventions or vague verbs.
16 tools is slightly above the ideal 3-15 range but fully justified for a comprehensive UE5 gameplay API covering connection, observation, multiple input methods, UI interaction, and convenience utilities. Each tool serves a clear practical purpose.
The surface covers the full cycle: connect, observe (screenshot/state), act (pad/key/mouse/UI), read logs, wait for conditions, adjust time scale, and reset input. No obvious dead ends or missing operations for controlling a game; even edge cases like UI failure have explicit fallback guidance.
Drive real devices from your AI Coding tool. Embed a client SDK (Unity, Godot, Flutter, iOS/macOS, Android, React Native, Web) in your app, then capture screenshots, traverse the UI tree, inject taps and key events, and run automated test tasks on the physical device over a secure relay.
Control Unreal Engine to browse assets, import content, and manage levels and sequences. Automate…
remote debug iOS/Android/Unity/Godot/Flutter/RN/Web on real-device.ui-tree/screenshots/taps,tests.
A comprehensive Model Context Protocol (MCP) server that enables AI assistants to control Unreal E…
