EZComputerCtrl MCP

English

A visual-first MCP for Windows desktop GUI control, allowing agents to operate on semantic objects rather than fragile screen coordinates.

EZComputerCtrl first organizes the current desktop into structured, executable interface facts, then lets the higher-level agent select objects and actions, and finally handles positioning, execution, and result return within the MCP.

Current Capabilities

Currently exposed MCP tools:

1. see

2. click

3. scroll

4. move_to

5. type_text

6. hotkey

Currently supported transport methods:

1. stdio

2. streamable-http

3. sse

Default transport method: streamable-http

Related MCP server: OmniFlow

Installation

git clone https://github.com/JucieOvo/ezcomputerctrl-mcp.git
cd ezcomputerctrl-mcp
pip install -e .

Included Skills

The repository includes a universal assembly skill:

skills/install-ezcomputerctrl-mcp/SKILL.md

This skill is designed to allow different Agent clients to quickly complete local configuration in their respective environments, rather than being bound to a specific client format.

Typical usage:

1. Open the cloned repository in the Agent client.

2. Call /install-ezcomputerctrl-mcp, or have the Agent use this skill.

3. Let the Agent complete port selection, local environment writing, background startup, and MCP connection configuration.

Local Runtime Configuration

Machine-specific local runtime configurations should be written to:

.runtime/ezcomputerctrl.env.ps1

An example file is located at:

scripts/ezcomputerctrl.env.example.ps1

The startup script will automatically load this file before launching the service.

Common environment variables:

1. EZCTRL_TRANSPORT

2. EZCTRL_SERVER_HOST

3. EZCTRL_SERVER_PORT

4. EZCTRL_MODEL_NAME

5. EZCTRL_MODEL_BASE_URL

6. EZCTRL_MODEL_API_KEY

VLM Configuration

Recommended path: Qwen3.5/3.6 35B-A3B

Do not hardcode fictional model names here. Please always use the ones actually deployed:

1. EZCTRL_MODEL_NAME

2. EZCTRL_MODEL_BASE_URL

3. EZCTRL_MODEL_API_KEY

The current default expectation is a VLM interface compatible with the OpenAI protocol.

Start and Stop

Hidden background startup:

cscript //nologo scripts\start_ezcomputerctrl_hidden.vbs

This entry starts a keep-alive guardian process with a hidden window by default.

1. The guardian process PID will be written to .runtime/ezcomputerctrl.guardian.pid.

2. The current business process PID will be written to .runtime/ezcomputerctrl.pid.

3. As long as the stop script has not been executed and the current Windows session has not ended, the guardian process will automatically restart the business process after it exits.

Stop background service:

powershell -NoProfile -ExecutionPolicy Bypass -File scripts\stop_ezcomputerctrl.ps1

The stop script will write a stop signal, terminate the current business process, and clean up the runtime PID files in .runtime/.

Execute startup script directly:

powershell -NoProfile -ExecutionPolicy Bypass -File scripts\run_ezcomputerctrl_http.ps1

Execute keep-alive mode directly:

powershell -NoProfile -ExecutionPolicy Bypass -File scripts\run_ezcomputerctrl_http.ps1 -KeepAlive

Single-run mode will only launch the MCP service once; keep-alive mode will additionally maintain the guardian process PID file .runtime/ezcomputerctrl.guardian.pid.

MCP Address

Under the default host and port, the streamable-http address is:

http://127.0.0.1:8765/mcp

If you have changed the port locally, please replace this address with the actual port.

Directory Structure

.
|-- .runtime/
|-- pyproject.toml
|-- README.md
|-- README_EN.md
|-- scripts/
|   |-- ezcomputerctrl.env.example.ps1
|   |-- run_ezcomputerctrl_http.ps1
|   |-- start_ezcomputerctrl_hidden.vbs
|   `-- stop_ezcomputerctrl.ps1
|-- skills/
|   `-- install-ezcomputerctrl-mcp/
|       `-- SKILL.md
`-- src/
    `-- ezcomputerctrl/

Windows Notes

1. Startup and shutdown scripts are primarily compatible with Windows PowerShell 5.1.

2. Script content is kept ASCII-first as much as possible to reduce encoding issues.

3. Hidden startup uses VBScript + PowerShell to avoid accidental closure caused by exposing the command line window.

Current Limitations

1. The current implementation focuses on Windows desktop GUI and is not intended for game control scenarios.

2. Current capabilities rely on a real desktop environment and a real VLM service.

3. This is a vision-driven control layer and does not guarantee zero-error recognition in complex interfaces.

4. The current external tool surface is kept focused, prioritizing stability and controllability.

5. After this tool is started, if you need to run games with R0-level anti-cheat (such as Tencent ACE), there may be a risk of triggering a ban. Please be sure to close this tool and restart your computer before entering such games. The author and the community are not responsible for any game account bans, restrictions, or other losses resulting from the use of this tool.