Universal Game MCP
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
Universal Game MCP (UGM) - V2.0
English | Tiếng Việt (Vietnamese Guide) | HOI4 Guide | Skill Packs
Universal Game MCP (UGM) is a lightweight, CPU-friendly Model Context Protocol (MCP) server that empowers external AI agents (Claude, Gemini, GPT-4, etc.) to autonomously explore, navigate, learn, and play offline strategy and simulation games without requiring manual scripting, game-specific logic, secondary agent runtimes, or model fine-tuning.
The AI model provides high-level reasoning and decision-making. Universal Game MCP provides:
Eyes: DXGI/GDI screen capture, event-driven OCR (RapidOCR PP-OCRv4), icon detection, and adaptive UI element exploration.
Hands: DirectInput mouse and keyboard control with bounded coordinate safety and dry-run simulation.
Memory: Persistent SQLite knowledge store with WAL mode, perceptual hashing (dHash), point-in-time checkpoints, and rollback.
Navigation: Multi-signal UI state classification and Dijkstra shortest-path menu traversal.
Self-Learning & Exploration: Autonomous interactive UI probing, 4-state knowledge lifecycle (
candidate→verified→suspect→rejected), and cross-game concept generalization.
⚡ Measured Benchmark & Performance Profile
Benchmarked on AMD Ryzen 5 5500U (6 Cores / 12 Threads, Windows 11 x64, integrated Radeon graphics) under pure CPU execution:
Subsystem | Measured Latency | Throughput / Notes |
Synthetic In-Memory Fixture Latency | 1.79 ms | In-memory memcpy baseline buffer grab |
Visual Diff Verification | 5.48 ms | Real-time state verification ( |
ROI Template Match (OpenCV) | 43.97 ms | 13.2x faster than full screen (Multi-scale + NMS) |
Full-Screen Template Match | 579.94 ms | Full 1080p scan |
Event-driven ROI OCR | 831.89 ms | PP-OCRv4 CPU on target bounding box |
Full-Screen 1080p OCR | 2,185.15 ms | Heavy; avoided at runtime via ROI crop |
RAM Footprint (Steady State) | 136.12 MB | Entire engine, OCR runtime & SQLite in RAM (< 150 MB) |
Synthetic in-memory fixture benchmarks are explicitly distinguished from live DXGI desktop duplication captures to prevent misleading performance claims.
Related MCP server: local-game-mcp
🏗 Architecture (V2 Autonomous Exploration Infrastructure)
+-----------------------------------+
| AI Client / LLM Agent |
+-----------------+-----------------+
| (MCP SDK v2 JSON-RPC)
v
+-----------------------------------+
| Universal Game MCP Server |
| (ugm.server) |
+-----------------+-----------------+
|
+----------------------------+----------------------------+
| | |
v v v
+------------------+ +------------------+ +------------------+
| ugm.core | | ugm.memory | | ugm.navigation |
| - capture (live) | | - database.py | | - recognizer.py |
| - input (direct) | | (SQLite WAL) | | (dHash + OCR) |
| - ocr (rapidocr) | | - storage.py | | - graph.py |
| - icon (opencv) | | - checkpoints | | (Dijkstra) |
| - explorer (v2) | | - cross_game | | - navigator.py |
| - safety & stop | | - dHash & pruner | +------------------+
+------------------+ +------------------+ |
| | |
+----------------------------+----------------------------+
|
v
+------------------+
| ugm.skills |
| - engine.py |
| - learner.py |
| - loader.py |
+------------------+Core Subsystems
Autonomous Exploration (
ugm.core.explorer): Generic element discovery, interactive contour and text bounding box grouping, non-destructive probing actions, and layout zone segmentation.Persistent Knowledge & Lifecycle (
ugm.memory.database,ugm.memory.storage): SQLite with WAL mode, point-in-time snapshot checkpoints, versioning, rollback, perceptual hash indexing (dHash), and 4-state knowledge promotion.Multi-Signal UI State Recognizer (
ugm.navigation.recognizer): Adaptive layout classifier combining perceptual dHash, prominent text anchors, panel geometry, and confidence metrics (known,ambiguous,unknown).Navigation Graph (
ugm.navigation.graph,ugm.navigation.navigator): Dynamic directed graph where nodes are UI states and edges are verified executable actions. Plans lowest-cost routes via Dijkstra with automatic replanning on mismatch.Verified Self-Learning Engine (
ugm.skills.learner): Procedural learning pipeline: Observe → Identify → Plan → Act → Verify → Learn → Reuse. Curated packs remain read-only; learned skills are incrementally validated and promoted.Skill Execution Engine (
ugm.skills.engine): Sandboxed runtime with allowlisted action primitives (press_keys,click_target,locate_and_click,click_selector,hover,wait,verify_state), precondition/postcondition checks, and emergency killswitch.Cross-Game Concept Store (
ugm.memory.storage): Registers and queries abstract strategy patterns (e.g., economy loops, pause toggles, production queues) shared across multiple games.
🛠 Complete MCP Tool Reference (34 Tools)
1. Perception, Input & Safety Tools (12 Tools)
Tool Name | Parameters | Description |
|
| Finds and attaches to target game window; returns geometry, DPI scale, and client rect. |
|
| Captures current game frame with client rect coordinates. |
|
| Event-driven RapidOCR reading on normalized ROI. |
|
| Multimodal target resolution with ambiguity detection and escalation. |
|
| Multi-scale OpenCV template matching with NMS. |
|
| Hovers mouse at normalized |
|
| Clicks target dynamically; verifies visual state before confirming. |
|
| Sends DirectInput keystrokes (e.g. |
|
| Computes visual difference and validates state transition. |
|
| Verifies active UI element selection and text status. |
|
| Instant global emergency killswitch halting all input dispatch. |
|
| Opt-in recording of annotated game frames to |
2. Memory, Navigation & Skill Execution Tools (11 Tools)
Tool Name | Parameters | Description |
| none | Lists discovered curated packs and registered games in persistent memory. |
|
| Classifies current game UI state (e.g. |
|
| Searches persistent SQLite memory across states, transitions, skills, or action history. |
|
| Calculates optimal multi-step navigation route using Dijkstra algorithm. |
|
| Safely traverses menus step-by-step with state verification and replanning. |
|
| Executes a curated or learned Game Skill with precondition/postcondition checks. |
|
| Saves newly discovered action sequence as candidate skill in persistent memory. |
|
| Lists procedurally learned skills, validation track records, and status. |
|
| Reports persistent memory statistics (states, transitions, action history, skills). |
|
| Safely deletes scoped memory or individual learned skill. |
|
| Read-only extraction of UI metadata and sprites from installed game files into local cache. |
3. V2 Autonomous Exploration & Knowledge Lifecycle Tools (11 Tools)
Tool Name | Parameters | Description |
|
| Auto-detects active foreground/named game, registers baseline resolution, and initializes empty knowledge store. |
|
| Scans active screen/ROI for interactive elements (buttons, icons, tabs, inputs) with bounding boxes. |
|
| Non-destructive probing action with pre/post capture diff analysis and isolated change tracking. |
|
| Updates lifecycle state ( |
|
| Creates an atomic point-in-time snapshot checkpoint of game knowledge. |
|
| Restores knowledge state to a previous point-in-time checkpoint. |
|
| Lists all available checkpoints with creation timestamps and metadata. |
|
| Queries reusable abstract concepts and gameplay patterns applicable across titles. |
|
| Registers an abstract gameplay pattern into cross-game knowledge store. |
|
| Optimizes execution latency of a verified skill by removing redundant delays and probes. |
|
| Safely cleans unpromoted candidates, rejected items, and transient exploration data. |
🎮 Game Skill Packs (Curated & Learned)
game_skills/
├── README.md # Skill pack architecture & JSON schemas
├── _template/ # Starter template for new games
└── hoi4/ # Hearts of Iron IV Reference Pack
├── manifest.json # Process names, resolutions, dpi-awareness
├── skills/ # Curated skills (pause_unpause, open_production, increase_speed, select_army_group)
├── selectors/ # Visual and OCR selectors (topbar_factories, political_power, pause_button)
└── templates/ # Reference PNG icons for template matchingCurated Skill Protection: Curated skills in
game_skills/are read-only at runtime and can never be overwritten by self-learning calls.Procedural Promotion: Learned skills start as
candidateand promote toverifiedafter ≥ 3 successful validations with confidence ≥ 0.85.Security Sandboxing: Execution of arbitrary shell scripts or code is blocked; only allowlisted primitives are permitted.
🚀 Quickstart
1. Installation
Requires Python 3.12 on Windows 10/11 x64:
# Clone repository
git clone https://github.com/QuafdaPoet/universal-game-mcp.git
cd universal-game-mcp
# Create Python 3.12 virtualenv
uv venv --python 3.12 .venv
# or: python -m venv .venv
# Activate virtualenv
.\.venv\Scripts\activate
# Install dependencies in editable mode
pip install -e .2. Generate Test Fixtures & Run Offline Tests
# Generate offline test fixtures
python scripts/generate_fixtures.py
# Run automated tests (100% offline, no game required)
pytest -v3. Run CPU Latency & RAM Profiling Benchmark
python scripts/benchmark_cpu_ram.py4. Starting the Universal Game MCP Server
# Run with stdio transport (Claude Desktop, Cursor, or Antigravity IDE)
python -m ugm.server
# Or enforce dry-run server-wide (inviolable hardware simulation)
python -m ugm.server --dry-runMCP Client Configuration (claude_desktop_config.json):
{
"mcpServers": {
"universal-game-mcp": {
"command": "C:\\Users\\<USER>\\...\\universal-game-mcp\\.venv\\Scripts\\python.exe",
"args": ["-m", "ugm.server"]
}
}
}🔒 Safety and Anti-Cheat Policies
Offline & Single-Player Only: Exclusively intended for single-player, offline strategy/simulation games.
Strict Anti-Cheat Blocker: Active process checks immediately abort and trigger emergency stop if anti-cheat services (EAC, BattlEye, Vanguard, Ricochet, etc.) are detected.
Server-Wide Dry-Run: When launched with
--dry-run, tool calls cannot override it withdry_run=False.Emergency Stop Killswitch: The
emergency_stoptool instantly locks out mouse and keyboard inputs.
📄 License & Attribution
Source code authored for UGM is licensed under the MIT License.
Third-party dependencies are acknowledged in THIRD_PARTY_LICENSES.md.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Connect AI assistants to your approved computer for files, commands, browser and desktop tasks.
Savecraft serves real save game data and expert game knowledge to AI assistants.
Make games with AI: one MCP server that connects your agent to Unity, Unreal and Godot editors.
AI agents on real phones: run tasks, read the screen, tap, type and install apps.
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to remotely control gaming PCs for automated gameplay with tools for screen capture, mouse/keyboard control, workflows, and system operations.2MIT
- FlicenseNot gradedqualityDmaintenanceEnables AI agents to play local Windows games through low-level keyboard/mouse input, screen capture, OCR, and per-game profiles for semantic actions.-
- AlicenseNot gradedqualityBmaintenanceEnables AI assistants to autonomously control Windows 11 and 10 desktops via sub-10ms screen capture, native UI Automation element inspection, and zero-lag keyboard and mouse input. Combines a visual plane with a semantic plane and stall detection so agents can operate real applications reliably without vision-only guessing.262 npm171 PyPI7MIT
- AlicenseAqualityCmaintenanceEnables AI assistants to interact with running games in real time — introspecting game state, simulating keyboard and mouse input to actually play, capturing screenshots, executing Lua code, hot-reloading files, and receiving push notifications when state changes.8MIT