Skip to main content
Glama

VisumWay

Give your AI eyes and a body.

VisumWay gives a local AI agent ways to see what’s on its computer, read what apps expose, and check what happened after it acted. Set it up once and any compatible local agent can use the same tools through the CLI or MCP.

Out of the Screen. A robot takes shape from a monitor's circuits, connected to a keyboard and mouse.

Download v0.2.6 for Windows x64 · Agent setup · Platform support

See the result of your own work

In a live Windows test, Codex saw a clipped Launch button in a running game, fixed the layout using an authorized tool, and looked again to verify the fix. In the same test series, a buffered window feed caught a visual state that lasted just 552 ms between requests. The agent could go back and see it. Live test record.

  • Read the app. Windows accessibility inspection returns labels, states, and bounds for controls the app exposes. In two matched Calculator trials, Codex completed the full task in 23.3 s vs. 34.3 s with screenshots using the same keyboard actions. That’s 32% less time in this benchmark. Method and raw timings.

  • See what pixels can tell you. Request an image when you need to judge a layout, check rendering, or see a game. On Windows, follow a single window in an active session and retrieve changed frames from a short local buffer.

  • Follow the evidence. Connect a window to its process and inspect local events. An agent can check whether an app responded, changed, or crashed after it used its authorized tools.

Related MCP server: Open Computer Use

Quick start

Give this repository to an agent that can run local commands and say:

“Use VisumWay and get yourself set up on this computer.”

The agent follows SETUP.md. VisumWay checks the host, discovers available connections, and writes a machine-specific PLAN.md. The agent uses that plan to connect and verify what works on your computer.

On Windows x64, the offline bundle includes the native capture and accessibility helper. You can also install the shared core from a checkout on Windows, macOS, or Linux with Python 3.9+:

python -m pip install .
visumway setup
visumway verify

Try it on Windows. Open an app you’re working on and ask your agent: “Use VisumWay to inspect this window. Tell me what you can read directly and capture the window if you need to check the layout.”

Use the same setup from a second agent

Say “Use VisumWay” in the second agent. If it can run commands locally, it can use the shared CLI immediately. For native MCP calls, point both hosts at the same installation:

codex mcp add visumway -- visumway mcp --agent codex
claude mcp add --transport stdio --scope user visumway -- visumway mcp --agent claude

If the agent cannot find visumway, use its absolute path. For a custom installation, put the same --home PATH before mcp in both entries. Use list_capture_targets to find a window, inspect_window for Windows controls, and capture_screen for pixels. Windows capture and buffered watch.

Optional crash alerts, one agent in charge

The first named installer (visumway setup --agent NAME) reserves notification ownership. Other local agents can look through the same connections and use the same event history. The first agent keeps the job of replying to background events.

Automatic wakes start paused. If a supported adapter is bound to your existing sign-in, the owner can review visumway wakes preview and enable alerts with visumway wakes enable --agent NAME. Local rules and a wake budget control when an event gets an agent turn. A verified Windows crash woke signed-in Codex and produced a native alert that reopened the chat with its response. Windows walkthrough.

VisumWay uses zero model tokens while idle. A visual watch runs on an explicit lease and ends on stop, expiry, or disconnect. Event history is local. An enabled wake sends a bounded report to the selected agent, which may send it to its provider. Agent actions use the host's existing permissions. Security and privacy.

Platform support

The shared setup targets Windows, macOS, and Linux. The prebuilt bundle, direct control reads, buffered window capture, and live Codex desktop tests are Windows results. macOS and Linux have their own capture and event backends and still need live desktop validation. The platform matrix and verification record show which paths have been exercised.

Build with VisumWay

MIT License

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      Not graded
      quality
      D
      maintenance
      Enables AI assistants to control Windows GUI by listing and focusing windows, capturing element snapshots via UIA/OCR/CDP, performing clicks/inputs/scrolls, verifying changes, waiting for screen updates, taking screenshots, and obtaining visual descriptions.
      2
      -
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to observe and control the user's live desktop by listing apps, reading accessibility trees, and performing clicks, typing, scrolling, dragging, and other input actions with per-action approvals and local audit archives.
      1
      MIT
    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables AI agents to inspect and drive native Windows desktop applications (WinForms, WPF, WinUI 3, and WebView2 hybrids) through Microsoft UI Automation by capturing interactive control snapshots, then clicking, filling, and asserting on elements via ephemeral refs or semantic screen-map targets. Mutating actions are dry-run by default and gated behind explicit arming, allow-lists, and a kill switch for lawful, assisted QA testing.
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to read what is on a Windows desktop as text — open windows, their contents, and what changed — using Win32 and UI Automation with optional NPU vision. Secrets, password fields, and private windows are withheld and personal data scrubbed on the user's PC before anything reaches the agent.
      AGPL 3.0