ue5-gameplay-mcp
ue5-gameplay-mcp
実行中のUnreal Engine 5のゲームをプレイするMCPサーバーです。 仮想ゲームパッド、キーボード、マウスの入力を受け付け、スクリーンキャプチャ、ログ行、UMGステータスを出力します。
これはエンジンプラグインではなく、クライアントです。2つのプラグインがすでにエンジン内の処理を担い、それぞれ独自のポートを保持しています。 このサーバーはその両方にダイヤルインし、1つのツールインターフェースとして提供します。
プラグイン | ポート | 提供機能 |
RemoteConsole2 | 10101 |
|
RemoteCapturePlugin | 10102 | JPEG/PNGスクリーンキャプチャ(PIEおよびパッケージ化されたビルドで動作) |
両プラグインには、このサーバーのために少量のC++コードが追加されました。 これらは独立性を保ち、独自のプロトコルを維持しています。追加部分は後方互換性があるため、古いクライアントでも再ビルドされたゲームと通信可能です:
FImageMeta.SourceSize— ゲームのバックバッファサイズ。構造体のサイズを24バイトに保つため、以前は予約語だった領域にパックされています。これにより、クライアントはダウンスケールされたキャプチャ上の特定の点をウィンドウのピクセルにマッピングし直すことができます。CMD_GET_GAME_STATE(520) およびIRemoteGameStateProvider— 詳細は後述します。
セットアップ
cd ue5_gameplay_mcp
uv syncMCP Python SDK v2 (mcp.server.MCPServer) が必要です。
Related MCP server: VERA MCP Server
実行方法
まずゲームを起動します。 このプロジェクトでは、Gameターゲットがクックされていないプロジェクトでは即座に終了してしまうため、スタンドアロンゲームはエディタのバイナリから実行されます:
"C:/Program Files/Epic Games/UE_5.8/Engine/Binaries/Win64/UnrealEditor.exe" "<PATH>/MyProject.uproject" -game -windowed -resx=1280 -resy=720 -log -nosplash次にサーバーを登録します。
プロジェクトルートにある.mcp.jsonがすでにこれを行っているため、Claude Codeは自動的にこれを認識します。
手動で行う場合と同等のコマンドは以下の通りです:
claude mcp add ue5-gameplay -- uv run --directory <PATH>/ue5_gameplay_mcp -m ue5_gameplay_mcpサーバーは遅延接続(レイジーコネクト)を行うため、実際の起動順序は重要ではありません。 ゲームより先に起動した場合でも、最初のツール呼び出し時に接続が行われます。
オプション: --host, --console-port, --capture-port, --format, --quality, --max-size, --grid-step, --transport streamable-http --mcp-port 14102。
ツール
セッション — game_connect, game_status, game_reset_input
オブザベーション(観察) — game_observe, game_state, game_log, game_wait_for_log
アクション — game_pad, game_pad_sequence, game_key, game_mouse, game_console, game_time_scale
UMG — game_ui_dump, game_ui_click, game_ui_focus
APIの設計とその理由
エージェントのラウンドトリップには数秒かかりますが、ゲームは60Hzで動作します。フレーム単位でのアクションは非現実的であるため、以下のようになっています:
すべてのアクションツールは
duration(継続時間)を受け取り、押す/押し続ける/離すという一連の動作をローカル環境で通信速度に合わせて実行します。1回のラウンドトリップは1フレームではなく、1つの意図(インテント)を伝達します。アクションツールはデフォルトで観察(Observe)を行います。
game_pad(ly=1.0, duration=0.5)は前進し、その結果のフレームを返します。これにより、行動と確認を別々に行う場合の半分のラウンドトリップで済みます。game_pad_sequenceは、入力間の確認よりも入力のタイミングが重要な場合に、コンボ全体を1回の呼び出しにまとめます。game_time_scale(0.2)は、瞬間の精度が必要な場合にゲーム内時間を稼ぎます。hold=Trueは、エージェントが思考している間もキャラクターが動き続けるように、ターンの間も入力を適用したままにします。game_reset_inputでこれをクリアします。
ピクセルではなく数値
game_stateは、レベル、ワールドタイム、ポーズ/タイムダイレーション、プレイヤーポーンのトランスフォーム、速度、移動モード、カメラ、そして最も近いアクタの距離と正規化された画面位置を返します。
これはgame_mouseが受け取るのと同じ0~1の座標であるため、ステータスレポート内で見つけた対象に即座に狙いを定めることができます。
この処理コストは画像の処理コストのほんの一部であり、HUDの数値を読み違えることもありません。
game_observe(state=True)はこれを観察に組み込み、game_pad(..., state=True)はアクションに組み込むため、移動と確認は依然として1回のラウンドトリップで済みます。
実際のマップでは、最も近いアクタのほとんどが背景オブジェクトであるため、レポートにはclass_counts(半径内のすべてのオブジェクトの調査結果)も含まれます。これを一度読み取り、class_filter="Enemy"で絞り込みます。
ゲーム独自の数値の追加
組み込みのレポートにはゲーム側のコードは必要ありません。ヘルス、スコア、クエストフラグなど、そのプロジェクトのみが知る情報については、任意のアクタにIRemoteGameStateProvider(Plugins/RemoteConsole2/Source/RemoteConsole2/RemoteGameState.h)を実装し、JSONオブジェクトの文字列を返すようにします:
FString AMyGameMode::GetRemoteGameState_Implementation()
{
return FString::Printf( TEXT("{\"score\":%d,\"wave\":%d}"), Score, Wave );
}これはBlueprintNativeEventであるため、Blueprintのみのプロジェクトでもオーバーライド可能です。
返された値はすべて、アクタ名をキーとしてcustomの下に格納されます。
プロバイダは距離フィルタに関係なく収集されるため、原点に配置されたスコアキーパーもレポートを送信します。
有効なJSONではないテキストは破棄されず、文字列としてそのままパススルーされるため、立ち上げ時の簡単なPrintfによるデバッグでも有用です。
メニューを操作する3つの方法(推奨順)
game_ui_dump+game_ui_click— 正確かつ高速ですが、UMG経由で登録されたウィジェットしか認識しません。カスタムのSlate UIを持つゲームでは何も返されませんが、ハングアップする代わりにツールがその旨を伝えます。パッドナビゲーション —
game_pad(buttons=["DOWN"]),game_pad(buttons=["A"])。ほぼすべてのゲームで機能します。ルック&クリック —
game_observe(grid=True)は、ラベル付きの0~1の座標グリッドをオーバーレイ表示します。画像からターゲットを読み取り、同じ数値をgame_mouse(x=..., y=...)に渡します。解像度に依存せず、UIの構築方法に関係なく機能します。
規則(コンベンション)
スティックはUEの仕様に従います:
ly=+1が前進です。(通信プロトコルではY軸が反転しますが、ここではそれを元に戻しているため、ツールAPIはゲーム独自の軸マッピングの意味と一致します)。マウス座標は左上を原点として0~1に正規化されており、ゲームの実際のバックバッファサイズを使用してピクセルに変換されます。キャプチャはリサイズ後の寸法を報告するため、このサイズは別途プローブされます。
game_observeは前回の観察以降の新しいログ行のみを返すため、長時間のセッションでも同じ出力が再送されることはありません。
既知の問題点・未完成な部分
エディタバイナリでゲームをホストしている場合、コンソールコマンドはPythonを経由してルーティングされます。
FGameAccessAPI::ExecConsoleCommandはIConsoleCommandExecutorの実装[0]にディスパッチしますが、エディタのPythonプラグインがロードされている場合、そのスロットはCmdではなくPythonになります。そのため、単なるstat fpsがSyntaxErrorとして返されます。サーバーは初回使用時に一度プローブを行い、これを検出した場合はコマンドをunreal.SystemLibrary.execute_console_commandでラップします。パッケージ化されたビルドにはPythonエグゼキュータがないため、この回避策は不要です。game_console(via="cmd")でオーバーライド可能です。画面のクリックは、キャプチャがゲームウィンドウ全体を満たしていることを前提としています。
-game -windowedの場合はこれが当てはまります。レターボックス化されたフルスクリーンモードの場合、報告されるソースサイズ内に黒帯が含まれることになり、マッピングがずれます。game_stateは、呼び出されるたびにレベル内のすべてのアクタを走査します。 通常のマップの規模であれば問題ありませんが、ストリーミングされるオープンワールドではTActorIteratorの代わりに空間クエリが必要になるでしょう。
テスト
uv run test/smoke_test.pyゲームと直接通信し、smoke_*.jpgを書き出します。これにより、キャプチャやグリッドのオーバーレイを目視で確認できます。
uv run test/mcp_client_test.py標準入出力(stdio)経由でサーバーを実際のMCPクライアントとして起動し、エラーパスを含めてすべてのツールを実行・テストします。
Available Tools
16 toolsgame_connectA
Connect to the running game and report what is on the other end.
Call this first, or after the game has been restarted. reconnect=True
drops the existing sockets and dials again.
| Name | Required | Description | Default |
|---|---|---|---|
| reconnect | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of disclosing behavior. It does disclose a key behavioral aspect: that `reconnect=True` 'drops the existing sockets and dials again.' But it does not describe what happens on a normal call (when reconnect is False) if an existing connection is already present—whether it reuses it, errors, or silently changes state. It also omits any mention of prerequisites, side effects on other game tools, or authentication requirements. The description adds some behavioral context but leaves significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise—two sentences—and every sentence contributes value. It front-loads the purpose, then provides usage timing, then explains the parameter behavior. There is no redundancy or filler, making it an exemplar of efficient communication.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema, the description does not need to detail return values. It covers the core aspects: purpose, when to use, and the effect of its only parameter. However, it could be more complete on edge cases, such as behavior when no game is running or when already connected without reconnect. Still, for a simple connection tool with one parameter, it provides sufficient context for an agent to call it correctly in the common flow, meriting a slightly above-average score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does explain the sole parameter `reconnect`: 'reconnect=True drops the existing sockets and dials again.' This adds meaning beyond the bare schema, which only provides a title and default. However, the description does not clarify the default behavior (when reconnect is False) or what happens if the tool is called without an existing connection. Thus, semantics are only partially covered, warranting a middle score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('connect') and resource ('running game') and an action ('report what is on the other end'). It clearly establishes the tool's role as an initial connection step, and 'Call this first' distinguishes it from sibling tools that likely operate post-connection. However, 'report what is on the other end' is somewhat vague and could be more explicit about what the report contains, preventing a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit timing guidance is provided: 'Call this first, or after the game has been restarted.' This clearly indicates when the tool should be invoked. It also explains the behavior of the `reconnect` parameter for re-establishing a connection. However, it does not mention when *not* to use it or point to alternatives, but the purpose and timing are clear enough for an agent to select it correctly in most cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_consoleA
Run a console command and return the log lines it produced.
Useful beyond cheats: stat fps, showdebug ai, and especially
getall <ClassName> <PropertyName> to read gameplay values out of the game
until a structured state API exists.
via picks the executor and should stay on "auto": when the game is hosted
by the editor binary the console routes through Python, which auto mode
detects once and works around. Force it with "cmd" or "python" if needed.
| Name | Required | Description | Default |
|---|---|---|---|
| via | No | auto | |
| settle | No | ||
| command | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It does disclose important runtime behavior: the 'via' parameter's auto mode and the Python routing workaround when the game is hosted by the editor binary. However, it does not mention that running arbitrary console commands may mutate game state, have side effects, or require special permissions—significant gaps for a command-execution tool with no annotation hints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description front-loads the primary purpose, then provides usage examples, and finally explains the 'via' parameter. It is moderately long but every section adds meaningful context. The structure is logical and scannable, though the third paragraph on 'via' could be tightened without losing content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main usage scenarios, gives concrete command examples, and explains the critical 'via' routing logic. Since an output schema is present, return-value details are not required here. However, it omits the semantics of 'settle' and does not warn about potential state changes from console commands, which an agent would need to safely invoke the tool in all contexts.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It thoroughly explains 'via' (auto vs. cmd vs. python) and why 'auto' should be the default, which adds value beyond the bare schema. But it says nothing about 'settle' (a numeric parameter with a default of 0.15) or the expected format of 'command' beyond examples. With one parameter entirely undocumented, the description fails to fully compensate for the zero-coverage schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear verb-resource pair: 'Run a console command and return the log lines it produced.' It distinguishes itself from siblings like game_state or game_log by referencing command execution and log output, and it provides concrete examples ('stat fps', 'getall') that sharpen its purpose as a generic command runner rather than a structured-state reader.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context: it is useful 'beyond cheats' and specifically for reading values 'until a structured state API exists,' implying it is a fallback when game_state is not sufficient. However, it does not explicitly name sibling alternatives (e.g., game_state) or state when not to use it, so the guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_keyB
Tap a keyboard key for duration seconds.
key is a single character ("w", "e") or a name: ENTER, ESC, SPACE, TAB,
SHIFT, CTRL, ALT, UP/DOWN/LEFT/RIGHT, F1-F12, TILDE.
| Name | Required | Description | Default |
|---|---|---|---|
| key | Yes | ||
| observe | No | ||
| show_ui | No | ||
| duration | No | ||
| max_size | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It explains the key format and duration, but does not mention side effects, requirements (e.g., an active game connection), or what happens after the key is tapped. It also does not explain how the observe, show_ui, and max_size parameters affect behavior. This is a significant gap for a tool that performs an input action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, consisting of two sentences, and is well-structured with a clear format for key names. Every sentence provides value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With five parameters, no annotations, and no output schema, the description is incomplete. It does not explain the purpose of the observe/show_ui/max_size parameters, nor does it clarify prerequisites or return behavior. More context is needed for an agent to use this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description covers only two of five parameters: key (with valid values) and duration (in seconds). The other three parameters (observe, show_ui, max_size) are entirely undocumented. Since schema description coverage is 0%, the description must compensate but does not, leaving most parameters unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: 'Tap a keyboard key for `duration` seconds.' The verb 'tap' and resource 'keyboard key' are specific, and the mention of valid key names (ENTER, ESC, etc.) distinguishes it from siblings like game_pad (gamepad) or game_mouse. It is immediately clear what this tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for keyboard input by specifying a keyboard key, but it does not explicitly state when to use it over alternatives like game_pad or game_mouse. No exclusions or conditions are given. The guidance is adequate but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_logC
New log lines since the last read. pattern is a regex filter.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| since | No | ||
| pattern | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It states 'since the last read' which implies stateful consumption (reading marks progress), and mentions the regex filter. However, it does not clarify whether logs are cleared, how 'last read' is tracked, or any side effects of calling this tool repeatedly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at two sentences, front-loading the core purpose. It is efficient and free of fluff, though it sacrifices necessary detail. This is good conciseness, but for a tool with three parameters, it leans toward under-specification rather than optimal economy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has three parameters and an existing output schema, the description still fails to explain the semantics of `limit` and `since` or how they interact with the 'last read' pointer. It does not specify whether `since` is a timestamp or line number, nor the default behavior when null. The output schema exists, so return value details are not required, but the parameter semantics and stateful behavior are essential and missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for all parameters. Only `pattern` is explained as a regex filter. The meanings of `limit` and `since` are left entirely to the schema defaults and titles, providing no additional context about their units, semantics, or interactions with the 'last read' behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns new log lines since the last read, which is a specific verb (read) and resource (game logs). It is not a tautology and is distinguishable from siblings like game_state or game_observe, though it doesn't explicitly name an alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a hint that it is used to fetch fresh log lines, but provides no explicit when-to-use or when-not-to-use guidance. There is no mention of alternatives such as game_wait_for_log for waiting on logs or game_observe for observations, leaving the agent to infer the appropriate context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_mouseA
Move or click the mouse at normalized screen coordinates.
x/y are 0..1 with the origin at the top left -- exactly the numbers printed
on a game_observe(grid=True) capture, so you can read a target off the
image and pass it straight in. They are converted to pixels here using the
game's real resolution.
action: click, down, up, double, move, setpos, wheel button: L, M, R, T1, T2
This is the fallback that works regardless of how the game builds its UI, when game_ui_click cannot see the widget tree.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| grid | No | ||
| wheel | No | ||
| action | No | click | |
| button | No | L | |
| observe | No | ||
| show_ui | No | ||
| duration | No | ||
| max_size | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry behavioral burden. It does explain the normalized coordinate system and pixel conversion, plus the available actions/buttons. However, it omits any mention of side effects (e.g., whether this moves the OS cursor vs. simulated game input), the effect of 'observe'/'duration', or failure modes. The core behavior is described, but not comprehensively.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured logically: first the operation, then the coordinate details, then the parameter lists, then usage context. Every sentence adds information, with no fluff. It could be slightly condensed, but the length is justified given the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a 10-parameter tool with no output schema and no annotations. The description explains only the essential core (x/y and action/button) but leaves many parameters undefined, no return value description, and no explanation of how 'observe' or 'show_ui' affect output. An agent would likely struggle to invoke this correctly for anything beyond basic clicks.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains x/y normalization and lists action/button values, covering only 4 of the 10 parameters. Parameters like grid, wheel, observe, show_ui, duration, and max_size are entirely unexplained, leaving significant ambiguity for an agent trying to use them correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb ('Move or click') and resource ('the mouse'), and immediately distinguishes itself from the sibling tool 'game_ui_click' by positioning itself as the fallback for when the widget tree is unavailable. This explicit differentiator prevents confusion in tool selection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The final sentence clearly defines when to use this tool: 'when game_ui_click cannot see the widget tree.' It also explains the coordinate system and conversion, giving enough context for an agent to decide between this and alternatives. The 'when-not' is implied by the fallback framing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_observeA
Look at the game: a screenshot plus the log lines since the last look.
grid=True overlays a labelled 0..1 coordinate grid -- turn it on when you
intend to click something with game_mouse. show_ui=False hides the
HUD/UMG overlays. log_pattern is a regex filter over the new log lines.
state=True folds in game_state, so one call answers what the world looks
like and what it actually contains.
| Name | Required | Description | Default |
|---|---|---|---|
| log | No | ||
| grid | No | ||
| image | No | ||
| level | No | ||
| state | No | ||
| quality | No | ||
| show_ui | No | ||
| max_size | No | ||
| log_limit | No | ||
| log_pattern | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full responsibility. It discloses that logs are 'since the last look' (incremental), that grid overlays a coordinate system, that show_ui hides overlays, and that state folds in game_state. It does not mention read-only status, rate limits, or side effects, but the 'look' verb implies non-destructive. Key behavioral nuances are surfaced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact: a one-line purpose followed by four backtick parameter hints. The essential information is front-loaded, and each sentence adds value. It avoids redundancy and reads naturally. Minor gap: it could include a note about output format, but overall it is well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 10 optional parameters, no output schema, and no annotations, the description is only partially complete. It explains the primary behavior and four parameters, but does not address quality, max_size, log_limit, or level, nor what the returned screenshot/log looks like. For an observation tool with many knobs, more detail would be expected.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must add meaning to parameters. It explains grid, show_ui, log_pattern, and state, but leaves log, image, level, quality, max_size, and log_limit unexplained. The core parameters affecting output and interaction (grid for clicks, show_ui for overlay) are covered, but the rest are ambiguous to the agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns a screenshot plus log lines, combining visual and textual state. It distinguishes itself from siblings like game_log (logs only) and game_state (state only) by merging both, and hints at optional state inclusion. The verb 'observe' and resource 'game' are specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit usage hints for the grid parameter ('turn it on when you intend to click something with game_mouse') and explains when to use state=True. It implies this tool replaces separate calls to game_log and game_state, but does not explicitly say when not to use it. The guidance is concrete and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_padA
Press pad buttons and push sticks for duration seconds, then observe.
buttons: any of A B X Y UP DOWN LEFT RIGHT L1 R1 L2 R2 L3 R3 SELECT START HOME TOUCHPAD (aliases: CROSS/CIRCLE/SQUARE/TRIANGLE, LB/RB/LT/RT, BACK/OPTIONS/MENU). sticks: lx/ly left stick, rx/ry right stick (camera), l2/r2 analog triggers. Range -1..1, UE convention -- ly=+1 is forward. hold: keep this input applied after the call returns, so the character keeps moving between your turns. Clear it with game_reset_input.
Example -- run forward for half a second and look at the result: game_pad(ly=1.0, duration=0.5) Example -- jump while running: game_pad(buttons=["A"], ly=1.0, duration=0.3) Example -- move and get exact positions back, not just a picture: game_pad(ly=1.0, duration=0.5, state=True)
| Name | Required | Description | Default |
|---|---|---|---|
| l2 | No | ||
| lx | No | ||
| ly | No | ||
| r2 | No | ||
| rx | No | ||
| ry | No | ||
| grid | No | ||
| hold | No | ||
| state | No | ||
| buttons | No | ||
| observe | No | ||
| show_ui | No | ||
| duration | No | ||
| max_size | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It clearly discloses input conventions (range -1..1, UE forward = +1), the hold behavior (keeps input after return, cleared by game_reset_input), and the state parameter's effect (returns exact positions instead of a picture). It also implies that without hold, inputs are released after the call. No annotation contradiction exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core action and result, followed by organized parameter details and three concrete examples. Every sentence serves a purpose—no fluff. It is substantial but well-structured, making it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 14 parameters, no output schema, and no annotations, the description is remarkably complete. It covers input specification, conventions, hold lifecycle, and how to get positional data. The examples give direct call patterns. The few omitted parameters are minor and do not hinder an agent from using the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate for 14 parameters. It thoroughly explains the core ones: buttons (with full list and aliases), sticks (lx/ly, rx/ry, l2/r2), range, hold, state, and demonstrates duration via examples. However, it omits explanations for grid, observe, show_ui, and max_size, which remain ambiguous. This is a minor gap given they are likely less critical for typical usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear, specific verb ('Press pad buttons and push sticks') and a resource ('pad'), and explicitly ties it to a duration and observation step. It distinguishes itself from sibling tools like game_pad_sequence (sequence vs single action), game_key/game_mouse (different input types), and game_observe (observation without input) by stating both action and result.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
While not explicitly naming alternatives, the description provides multiple examples covering typical scenarios (run forward, jump while running, and requesting exact state) and explains the hold parameter with a clear instruction to clear it via game_reset_input. It implicitly guides when to use this versus other tools, but does not explicitly state 'use this instead of X when...'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_pad_sequenceA
Run several pad states back to back in a single round trip.
Each step is {"buttons": [...], "sticks": {"ly": 1.0}, "duration": 0.2, "gap": 0.05}. Use this for combos and for anything where the timing between inputs matters more than looking in between.
Example -- a three hit combo: [{"buttons": ["X"], "duration": 0.1, "gap": 0.25}, {"buttons": ["X"], "duration": 0.1, "gap": 0.25}, {"buttons": ["Y"], "duration": 0.1}]
| Name | Required | Description | Default |
|---|---|---|---|
| steps | Yes | ||
| observe | No | ||
| show_ui | No | ||
| max_size | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the 'single round trip' behavior, which is useful, and details the step structure. However, it does not disclose potential gaps (e.g., partial execution on failure), error behavior, or how it interacts with other input tools. The information is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized: it starts with a one-line purpose, then defines the step format, gives a usage context, and ends with a concrete example. It is not overly verbose, and the example is efficient and illustrative, earning its place. The front-loading of purpose is effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 4 parameters and no output schema, the description is incomplete. It explains 'steps' thoroughly but omits the purpose and behavior of 'observe', 'show_ui', and 'max_size'. It also does not describe what the tool returns or how the sequence result is delivered. This leaves agents uncertain about side effects and required configurations, making the description insufficient for correct invocation in many scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains the 'steps' parameter in detail, including the exact JSON structure for each step and an example. However, the schema coverage is 0%, and the description fails to document the other three parameters (observe, show_ui, max_size). It compensates partially but leaves significant gaps, especially for optional parameters that affect the round trip.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs 'several pad states back to back in a single round trip' and provides a concrete example of a three-hit combo. It distinguishes itself from sibling tools like game_pad by emphasizing sequence and timing, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly recommends using the tool 'for combos and for anything where the timing between inputs matters more than looking in between,' providing clear context for when it is appropriate. It implies that for cases requiring observation between inputs, one would use separate calls, though it does not explicitly name the alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_reset_inputA
Release every held button, centre the sticks, lift the mouse buttons.
Use this after hold=True actions, or any time the character seems stuck
walking into a wall.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the exact physical actions performed (release, centre, lift) and implies the effect of unsticking the character. While it does not mention side effects or broader scope, the reset behavior is clear and sufficient for this simple tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences, with the core action front-loaded in the first sentence and a crisp usage note in the second. Every word earns its place with no redundant or vague phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and an output schema present, the description fully addresses how to use the tool, when to use it, and what it accomplishes. Even includes a troubleshooting scenario ('stuck walking into a wall'). Nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so there is no parameter information to add. The description correctly contains no parameter details, and the schema coverage is trivially 100%. The baseline score for zero-parameter tools is 4, and no deduction is warranted.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action on a clear resource: 'Release every held button, centre the sticks, lift the mouse buttons.' This distinguishes it from sibling tools like game_key, game_mouse, and game_pad, which handle individual inputs. It is not a tautology and clearly communicates the tool's function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this after `hold=True` actions, or any time the character seems stuck walking into a wall.' This gives clear, concrete when-to-use conditions. However, it does not state when not to use it or mention alternative tools, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_stateA
Structured world state -- no screenshot, cheap to call often.
Returns the level name, world time and pause/dilation, the player pawn's
transform, velocity and movement mode, the camera, and the nearest actors
with their distance and their normalized screen position (the same 0..1
coordinates game_mouse takes, so you can aim at what you find here).
class_filter keeps only actors whose class name contains that substring,
e.g. "Enemy". radius is in centimetres. Games can add their own values --
health, score, quest flags -- by implementing IRemoteGameStateProvider on an
actor; those show up under "custom".
Prefer this over reading numbers off the screen: it is exact, and it costs a fraction of the tokens an image does.
| Name | Required | Description | Default |
|---|---|---|---|
| actors | No | ||
| radius | No | ||
| max_actors | No | ||
| class_filter | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It discloses that the call is cheap and can be made often, and it details the exact return contents, including custom provider values. It does not mention side effects or failure modes, but as a read-only state query, it is sufficiently transparent and contradicts nothing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with a summary of the tool's essence. It then details return values, parameter semantics, and a usage recommendation, with no wasted sentences. Formatting with backticks and a clear logical flow makes it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the major return fields and custom provider values, and an output schema exists. It likely suffices for an agent to understand what to expect. Minor gaps include lack of error or edge-case handling, but these are not critical for a state query tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains `class_filter` (substring filter) and `radius` (in centimetres) explicitly. It does not explain `actors` or `max_actors`, but they are inferable from names and defaults. Since schema coverage is 0%, the description only partially compensates, leaving two parameters underspecified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns a structured world state, enumerating specific elements: level name, time, player transform, velocity, movement mode, camera, and nearby actors. It distinguishes itself from a screenshot and explicitly frames itself as a precise alternative to reading numbers off the screen, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit guidance to prefer this tool over reading numbers off the screen, indicating a use case. It also references game_mouse coordinates, implying how to use the output for aiming. However, it does not name specific alternative tools or state when not to use this one, leaving some room for interpretation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_statusB
Connection state, current level, and any input still being held.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of disclosing behavioral traits such as side effects or safety. It only lists the data returned and does not state whether the call is read-only, modifies any state, or has any side effects. This is a significant gap for a status tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no fluff. It is front-loaded with the core status elements. It earns its place, though it could be more explicit about being a read-only status query.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters) and the existence of an output schema, the description does not need to explain return values. However, it omits any guidance on prerequisites, such as requiring an established connection (relevant given sibling game_connect). This leaves minor ambiguity but is largely sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema fully covers parameter semantics by definition. The baseline for 0 parameters is 4. The description adds no parameter information, but none is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly indicates the tool reports status information: connection state, current level, and held input. The verb is implied (returns/reports), and the resource is specifically game status. It does not differentiate from siblings like game_state, but the purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention game_state, game_observe, or any conditions for selection. There is no context about prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_time_scaleA
Slow down or speed up the world (slomo). 1.0 is normal, 0.0 freezes it.
Drop to 0.1-0.3 when you need to place an action precisely; the game keeps running but each of your turns costs far less game time.
| Name | Required | Description | Default |
|---|---|---|---|
| scale | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosure. It explains the effect of scale values and the consequence that turns cost less game time, which is beyond the name. However, it does not disclose potential side effects like whether the change is global or persistent, or any latency implications, leaving some ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the core semantics and then adding a practical usage tip. No filler or repetition; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with one optional parameter, and the description covers its effect and a use case. It doesn't explain the return value, but an output schema exists, so that is not required. It could mention the scope of the change (e.g., affects all players or just the session) for full completeness, but given its simplicity, it is largely adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides no description for the 'scale' parameter, but the description compensates by defining the value semantics: 1.0 normal, 0.0 freeze, and recommending a slow-motion range of 0.1-0.3. This adds meaning well beyond the schema's bare number type and default.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool's function—slowing down or speeding up the world, likening it to the `slomo` command—and defines the scale semantics (1.0 normal, 0.0 freeze). This clearly distinguishes it from sibling game-control tools like game_pad or game_key, which manipulate input rather than time.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a specific use case: dropping to 0.1-0.3 for precise action placement, noting that the game continues but turns cost less game time. It does not explicitly mention when not to use it or alternative tools, but the guidance is concrete and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_ui_clickB
Click a UMG widget by name. action: click, press, release, hover, unhover.
Fast and exact when it works; use game_ui_dump first to learn the names.
| Name | Required | Description | Default |
|---|---|---|---|
| action | No | click | |
| widget | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions 'Fast and exact when it works' which hints at reliability but doesn't disclose side effects, whether it can be destructive, or what happens on hover/release. It does not clarify if it modifies game state or if it requires a connected game session. Minimal behavioral context beyond performance.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise: two short sentences. The primary purpose is front-loaded, and the usage tip about game_ui_dump is placed as a secondary note. Every sentence adds value with no redundant text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core usage and mentions the prerequisite dump tool. However, it lacks details on error handling, interaction with game_connect, or specifics about how the action parameter affects behavior. Given the presence of an output schema, return format is not needed, but for a tool with multiple actions and a dependency, more context would help.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must clarify parameters. It lists the valid actions for the action parameter (click, press, release, hover, unhover), which adds meaning beyond the schema. However, it does not explain the widget parameter format (e.g., exact path, case sensitivity) or any constraints. Partial compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool clicks a UMG widget by name, with a specific verb and resource. It lists distinct actions (click, press, release, hover, unhover) which differentiates it from generic input tools like game_mouse or game_pad. However, it does not explicitly contrast with game_ui_focus, which might also interact with UI widgets, so it slightly lacks full sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives a clear prerequisite: use game_ui_dump first to learn widget names. This helps the agent know how to proceed. It does not state when to use this over alternatives like game_ui_focus or game_mouse, nor does it provide exclusions (e.g., if the widget is not clickable). The 'Fast and exact when it works' implies potential failure but does not describe fallback paths.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_ui_dumpA
Dump the UMG widget tree. mode: all, button, debug.
Only sees widgets registered through UMG. A game with custom Slate UI will come back empty or error -- that is expected, fall back to the pad or to game_observe(grid=True) plus game_mouse.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | button |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the tool's limitation (only UMG widgets) and the expected error/empty result for Slate UI, which is valuable behavioral context. However, it does not explain the effect of each mode (all, button, debug) on the output, and there's no mention of side effects or required permissions, though dump operations are generally benign.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is terse and front-loaded with the main action: 'Dump the UMG widget tree.' It immediately lists the modes, then adds a crucial caveat about Slate UI and fallback options in the second sentence. Every sentence serves a purpose with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers the tool's purpose, limitations, and fallbacks, it omits the semantics of the three modes. An agent would need more detail to decide which mode to use (e.g., what 'debug' returns vs. 'all'). The output schema exists but is not described, and a dump tool's output could vary significantly by mode. Overall, it's not fully complete without mode behavior explanations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It lists three possible mode values ('all', 'button', 'debug') but does not explain what each mode does or how they differ. This provides some guidance over the schema's bare string type, but the agent still lacks semantic understanding of each mode. The default 'button' is set, but its meaning is unclear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear verb ('Dump') and resource ('UMG widget tree'), making the purpose unambiguous. It also differentiates itself from sibling tools by explicitly noting that Slate UI won't work and suggesting fallbacks to pad, game_observe, and game_mouse, so an agent can tell when this tool is appropriate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use guidance by stating it only sees UMG widgets and explains the expected failure on custom Slate UI. It also names specific alternative tools (pad, game_observe, game_mouse) as fallbacks, so the agent knows what to do if this tool fails.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_ui_focusB
Set the game's input focus. mode: game, ui, game_and_ui, window.
Bring the window forward and hand focus back to the game with mode="game" if input seems to be going nowhere.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | game | |
| widget | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full behavioral burden. It does state that the tool can bring the window forward and return focus to the game, which is a useful side effect. However, it omits potential side effects, prerequisites, or reversibility details, leaving some uncertainty for a focus-altering operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact at two sentences, with the core action stated first. The second sentence adds a practical troubleshooting hint. No unnecessary fluff or redundancy is present.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, the description covers the primary purpose and one use case well, but it fails to address the 'widget' parameter and does not describe return values (though an output schema exists). Without annotations, this leaves the tool less than fully specified for all scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains the possible values for the 'mode' parameter ('game, ui, game_and_ui, window') but says nothing about the 'widget' parameter, leaving it completely undocumented. This is a significant gap for an agent needing to pass a widget.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sets the game's input focus and enumerates four distinct modes. This makes its purpose unambiguous and distinguishes it from input-sending tools like game_key or game_mouse, which trigger actions rather than alter focus.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a concrete usage trigger: 'Bring the window forward and hand focus back to the game with mode="game" if input seems to be going nowhere.' This tells the agent when to reach for this tool, albeit without explicitly comparing it to sibling alternatives like game_reset_input.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
game_wait_for_logA
Block until a log line matches pattern, or the timeout expires.
This is the assertion primitive for automated tests: fire an action, then wait for the line the game prints when the thing you expected happened.
| Name | Required | Description | Default |
|---|---|---|---|
| pattern | Yes | ||
| timeout | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the tool blocks (synchronous) and that a timeout expires, but does not mention what happens on timeout (error vs. return value), whether log lines are consumed, or any side effects. Given that it's a harmless wait operation, the core behavior is clear, but error handling and return semantics are left to the output schema. Not as transparent as it could be.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core functionality in the first sentence, and the second provides valuable usage context. There is no filler or repetition. Every sentence earns its place, making it efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, and an output schema exists, so return values are likely covered. The description explains what it does and when to use it. It does not mention any prerequisite (e.g., game_connect) or error conditions, but for a straightforward wait/assertion tool, the provided information is largely sufficient. A minor gap is the lack of timeout behavior details, but that can be inferred or covered by the output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains 'pattern' as the pattern to match against log lines, and implicitly 'timeout' as the duration via the first sentence. It does not specify pattern syntax (e.g., regex) or default values, but it adds enough semantic meaning for an agent to understand what each parameter does. This is a strong effort given the schema provides no descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Block until a log line matches `pattern`, or the timeout expires.' This is a specific verb (block/wait) and resource (log lines). It further distinguishes itself from sibling tools by positioning as 'the assertion primitive for automated tests,' which separates it from game_log (likely read-only log retrieval) and other action tools. The purpose is unambiguous and well-differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear usage context: 'fire an action, then wait for the line the game prints when the thing you expected happened.' This explicitly tells the agent when to use the tool (after triggering an action, to assert expected behavior). It does not name alternative tools or explicitly state when not to use it, but the context is sufficient for typical selection. No misleading guidance is present.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
16 tool updates
v0.1.0- First observed
game_connect - First observed
game_console - First observed
game_key - First observed
game_log - First observed
game_mouse - First observed
game_observe - First observed
game_pad - First observed
game_pad_sequence - First observed
game_reset_input - First observed
game_state - First observed
game_status - First observed
game_time_scale - First observed
game_ui_click - First observed
game_ui_dump - First observed
game_ui_focus - First observed
game_wait_for_log
TDQS
Scored across 16 tools
Each tool targets a distinct interaction mode: connection, observation, state, input (pad/key/mouse/UI), logging, console, time scaling, and focus. Even overlapping actions like game_mouse vs game_ui_click are explicitly differentiated by target (screen coordinates vs widget tree) and fallback behavior.
All tools follow the predictable 'game_' prefix with an action verb (connect, log, observe, state, pad, key, mouse, console). The sub-group 'game_ui_*' maintains consistency with a clear sub-domain. No mixed conventions or vague verbs.
16 tools is slightly above the ideal 3-15 range but fully justified for a comprehensive UE5 gameplay API covering connection, observation, multiple input methods, UI interaction, and convenience utilities. Each tool serves a clear practical purpose.
The surface covers the full cycle: connect, observe (screenshot/state), act (pad/key/mouse/UI), read logs, wait for conditions, adjust time scale, and reset input. No obvious dead ends or missing operations for controlling a game; even edge cases like UI failure have explicit fallback guidance.
Maintenance
Related MCP Connectors
Drive real devices from your AI Coding tool. Embed a client SDK (Unity, Godot, Flutter, iOS/macOS, Android, React Native, Web) in your app, then capture screenshots, traverse the UI tree, inject taps and key events, and run automated test tasks on the physical device over a secure relay.
Control Unreal Engine to browse assets, import content, and manage levels and sequences. Automate…
remote debug iOS/Android/Unity/Godot/Flutter/RN/Web on real-device.ui-tree/screenshots/taps,tests.
A comprehensive Model Context Protocol (MCP) server that enables AI assistants to control Unreal E…
Related MCP Servers
- AlicenseBqualityAmaintenanceEnables AI assistants to control Unreal Engine via Remote Control API for game development automation, including asset management, actor control, level editing, animation, physics, visual effects, and cinematics creation through natural language.1544 npm913MIT
- AlicenseNot gradedqualityBmaintenanceEnables AI clients like Claude Code, Cursor, or VS Code to drive the Unreal Editor: execute Python, capture screenshots, tail logs, check status, and run VERA commands.18MIT
- AlicenseAqualityDmaintenanceEnables AI assistants to interact with Unreal Engine via Remote Control API for actor, asset, level, and editor operations.229 npmMIT
- AlicenseNot gradedqualityBmaintenanceEnables an agent to drive a running Unreal Engine game via UE4SS by reading/writing UObject properties and calling UFunction, with verdicts that verify whether the requested state change actually happened.MIT