EDM
Enables AI agents to observe and control AutoCAD as a Windows desktop application, using screen capture, region zoom, UIA/OCR perception, and mouse/keyboard input including smooth drawing paths for CAD curves and dense ribbon interactions.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
EDM — Everything Desktop on Me
High-resolution Windows desktop vision, region snipping, zoom, UIA/OCR perception, and full mouse/keyboard control for AI agents via MCP and CLI.
Status: Planning & architecture complete (
v0.1specification ready). Source implementation begins at Phase 0.
Türkçe: README.tr.md · Design & Plan: docs/plan/00-TASARIM.md
Why EDM Exists
AI agents trying to drive real Windows desktop apps (AutoCAD, WinRAR, Chrome dialogs, AOC G-Menu, etc.) repeatedly fail on the first attempt and loop because:
Fragile window binding: Window-scoped capture (
WGC/PrintWindow) times out or returns blank frames on DirectX/OpenGL viewports, splash screens, popups, or system tray menus.UI Automation blind spots: Standard controls expose accessibility trees, while CAD viewports and custom hardware UIs appear as opaque black boxes.
DPI & coordinate drift: Scaled monitors (125% / 150% / 4K) cause pixel guesses to miss small toolbar icons.
Silent OS blocks (UIPI & isolated desktops): Elevated apps silently drop standard
SendInput, and sandboxed agent shells run on an isolated desktop (WinSta0\exebox-...) instead of the user's visibleWinSta0\Defaultsession.
EDM solves this with an observation-bound, PrintScreen-first, closed-loop architecture:
See everything actually on screen: Virtual screen, specific monitor, visible window crop (GDI composite), or rectangular region — with WGC and
PrintWindowfallbacks for occluded windows and blank-frame detection.Coarse-to-fine
zoom: Inspect full-screen layout, then crop/zoom any dense ribbon or tiny button at native pixel resolution (preserving lossless PNG on small CAD crops) while EDM automatically maps coordinates back to the physical virtual screen (pixelornorm1000).Human-like full input & drawing control: Single/double/right/middle clicks, straight drags, smooth multi-point continuous strokes (
draw_pathfor Paint/CAD curves), held buttons/keys (mouse_button,key_statewith automatic release-on-error), Unicode typing, shortcuts, and validated multi-step sequences.Closed-loop safety: Per-Monitor-V2 DPI lock, UIPI elevation detection (
ELEVATION_REQUIRED), stale-observation guard (window geometry + 24×24 pixel patch verification), inter-agent input lock, and instant global kill switch (Ctrl+Alt+Shift+F12+(0,0)corner).Multi-client ready: One core engine (
src/edm) exposed as both an MCP server (stdio+ user-started loopback HTTP) and a JSON CLI (edm), designed for Antigravity, Codex Desktop, Claude Code, Gemini CLI (agy), and local scripts.
Related MCP server: com-use-mcp
Planned Architecture (v0.1)
AI Clients (Antigravity / Codex / Claude Code / Gemini CLI) ──MCP──▶ mcp_server.py / http_server.py
User / Local Scripts ──CLI──▶ cli.py
│
service.Edm (single facade)
│
┌──────────────────────┬──────────────────────┬───────────────────────┴──────────────────────┐
▼ ▼ ▼ ▼
capture/ observe/ perception/ actions/
gdi → wgc → pw store, render, diff uia (SoM), winrt ocr (tr-TR) guard, killswitch, inputKey design & planning documents (in Turkish):
docs/plan/00-TASARIM.md — System design, coordinate contract, 20 MCP tools, error taxonomy.
docs/plan/01-UYGULAMA-PLANI.md — TDD task-by-task implementation plan (Phases 0–6).
docs/plan/02-TEST-VE-KABUL.md — Automated gates, live acceptance suite (
L1–L11), and real app matrix (M1–M7).docs/plan/PROGRESS.md — Live execution tracker.
AGENTS.md — Strict agent start/stop, anti-loop, and safety rules.
Security & Privacy Model
Local-only: No cloud telemetry or outbound network calls. Screenshots and action logs stay strictly under
%LOCALAPPDATA%\EDM\(auto-pruned at 7 days / 2 GB) and are never committed to Git.Explicit boundaries: EDM never switches Windows desktops, patches sandboxes, or bypasses UAC. If an elevated app (e.g., G-Menu) requires admin integrity, EDM detects it and asks the user to launch EDM elevated if desired.
Screenshot sensitivity: Anything visible on your screen when a tool runs can be sent as tool output to whichever AI client invoked EDM. Do not leave secrets or banking sessions uncovered during agent runs.
Acknowledgments & Prior Art
EDM is written cleanly from scratch for Windows, informed by lessons from open-source GUI agents and local Windows automation work; see THIRD_PARTY_NOTICES.md.
License
MIT © 2026 Hüseyin Günaydın
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Connect ChatGPT, Claude, Cursor or VS Code to your Windows PC: files, terminal, browser, apps.
Eyes and hands on real Windows PCs — observe, click, type via Glasswarp API.
Real desktops for AI agents, on machines you choose: screenshot, type, click, hire more agents.
Connect AI assistants to your approved computer for files, commands, browser and desktop tasks.
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceEnables AI assistants to autonomously control Windows 11 and 10 desktops via sub-10ms screen capture, native UI Automation element inspection, and zero-lag keyboard and mouse input. Combines a visual plane with a semantic plane and stall detection so agents can operate real applications reliably without vision-only guessing.262 npm171 PyPI7MIT
- FlicenseNot gradedqualityCmaintenanceEnables AI agents to see and operate a Windows PC through mouse, keyboard, screenshots, OCR, window control, and shell commands.-
- AlicenseAqualityBmaintenanceEnables any AI agent to perceive and control a Windows desktop through live screen capture, OCR, and safety-gated mouse/keyboard actions via a local HTTP API and MCP.81610MIT
- AlicenseAqualityBmaintenanceEnables agents to perceive and control a Windows desktop, a connected local Chromium browser, or a cloud Playwright browser through human-like mouse moves, clicks, drags, scrolling, and screenshots, forming an observe-locate-act computer-use loop. Includes safety guards such as random pauses, emergency stop, and approval prompts for destructive actions.11MIT