SideDesk AI

繁體中文

Download the Windows technical preview

A Windows background workspace for AI agents, using applications installed on the host. Pre-alpha 0.1.0a1. This milestone proves a real hidden-desktop control loop with a deterministic Win32 application. It does not promise universal GUI support.

What works

• Create, start, inspect, stop and locally remove workspaces from a desktop manager.

• Launch approved executables on a private Win32 Desktop Object, with owned process jobs and checked suspended-launch containment.

• Inspect standard Win32 controls, replace Edit text with readback, and send background button clicks. A click remains unverified until its application effect is checked.

• Capture real window pixels using a disposable, bounded PrintWindow helper. PNGs include quality and hash metadata; the MCP tool also returns image content.

• Share live workspaces between the GUI and MCP clients through an authenticated, local-only named-pipe broker. No control TCP listener is started.

• Export files with hash verification. Cleanup refuses running workspaces, links and unexported files, and rechecks exported copies before removing source results.

There is no global keyboard/mouse injection or desktop-switch fallback. Raw Input, arbitrary shortcuts, drag-and-drop, GPU capture, UAC interaction and Sandboxie isolation are not supported by this milestone. See the compatibility matrix.

A desktop is not a file sandbox. Approved applications run with your Windows user's host access. TEMP redirection only organizes files. Register only trusted programs with a tested independent-instance launch strategy. A single-instance application may contact a pre-existing host process before SideDesk can detect it.

Related MCP server: local-assistant-mcp

Run

Requires Windows 11 x64, an interactive user session and Python 3.12 with Tcl/Tk for source execution. No administrator permissions or driver installation are required.

uv sync --locked
uv run sidedesk doctor
uv run sidedesk gui

For the portable build, extract the whole archive and open SideDesk.exe. sidedesk-cli.exe and _internal/ must stay alongside it. A portable build does not need a separate Python installation. The first build is unsigned; clean-machine installation and SmartScreen behavior have not been accepted yet.

1. In Applications, select a trusted executable and its fixed JSON argument list. Review and explicitly acknowledge host access and experimental background launch.

2. Create a workspace, select its approved applications, and start it.

3. Launch an application, list its windows, inspect controls, or capture a preview.

4. Stop the workspace, export any results, then reset or delete it when permitted.

Agent tools cannot register programs, change launch arguments, select arbitrary export destinations, force stop, reset or delete workspaces. These actions belong to the local manager. See installation and MCP setup.

Agent connection

Configure an MCP client to start the console bridge with stdio:

{
  "mcpServers": {
    "sidedesk": {
      "command": "C:/path/to/SideDesk/sidedesk-cli.exe",
      "args": ["mcp"]
    }
  }
}

For a source checkout use an absolute path to .venv/Scripts/sidedesk-mcp.exe. The GUI and bridge must use the same --data-root if overriding the default. Tools use underscore names such as workspace_create, window_inspect, and window_capture; results identify the workspace, backend and verification status.

Evidence and development

Windows acceptance, MCP acceptance, GUI acceptance, and lifecycle acceptance record the actual checks. Supervisor crash and packaged MCP have separate live evidence. Synthetic screenshots contain no host desktop captures. Stationary-host sampling is not evidence for concurrent typing or fullscreen games.

uv run pytest -q
uv run ruff check src script tests
uv run ruff format --check src script tests
uv run pip-audit --local
uv run python script/verify_windows.py
uv run python script/verify_mcp.py
uv run python script/build_windows.py

Live scripts run controlled applications on hidden desktops. Ordinary CI runs contract/policy tests and builds packages; it does not certify interactive GUI support. Read testing, architecture, backend research, platform research, roadmap, and contributing.

SideDesk source is MIT licensed. See third-party notices and security boundaries. MATLAB and Sandboxie are not bundled.