Skip to main content
Glama
Sign In
OpenMouse MCP

OpenMouse MCP

by osca5771666
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Hybrid

OpenMouse MCP

内置式视觉鼠标操控 MCP 插件。本地 Agent 直接以 stdio 启动，云部署可暴露 Streamable HTTP。插件内部自动选择执行面：

  • 前台是已连接的 Chromium 浏览器时，使用页面内 JS 虚拟光标、DOM Pointer/Mouse 事件和当前标签页截图。

  • 前台不是该浏览器时，在 Windows 上使用 PyAutoGUI 操控真实系统鼠标，可操作桌面、窗口和终端。

  • 在 Linux/云端或显式启用云浏览器时，使用 MCP 进程内 Playwright Chromium，提供可信浏览器输入。

浏览器端不会穿透网页去移动系统鼠标；Windows 端可以操作桌面任意软件。

能力

工具

作用

mouse_move

贝塞尔轨迹移动，带轻微抖动和中途停顿

mouse_click

左键单击、左键双击、右键单击

mouse_drag

人类化拖拽与浏览器文本选择

mouse_scroll

鼠标滚轮上下滚动

mouse_position

读取当前虚拟光标或真实光标坐标

mouse_screenshot

浏览器当前标签页或 Windows 全屏截图

computer_observe

全屏截图 + 任务栏、窗口、显示器、浏览器视口等视觉元数据

browser_open

快速打开本地默认浏览器/Edge/Chrome，或导航云端 Playwright 页面

app_open

快速恢复已运行应用窗口，或从别名/开始菜单启动本地应用

mouse_stop

紧急停止、释放按键、清空动作队列

mouse_status

查看自动路由、扩展心跳和失败状态

所有鼠标动作共用同一套 MCP 参数，Agent 不传环境参数。computer_observe 提供截图给模型做视觉识别，形成“观察 → 定位 → 动作 → 再观察”的 computer-use 闭环。

Codex 插件配置使用 MOUSE_MCP_SPEED_PROFILE=fast：本地轨迹约为 0.1 到 0.38 秒，普通动作后停顿约为 0.04 到 0.16 秒，同时保留曲线和非瞬移。高风险动作仍使用更慢的确认后节奏。

Related MCP server: Open Computer Use

安全机制

  • 每次移动、点击、拖拽和滚动都使用曲线轨迹。fast 档附加 0.04 到 0.16 秒停顿，balanced 档使用 0.2 到 1.3 秒停顿。

  • mouse_click 最多自动重试 2 次。连续 3 次操作失败后直接紧急停止。

  • 全局 ESC 监听会设置当前动作取消标志、清空排队任务，并通知浏览器扩展停止。

  • Windows 坐标限制在虚拟屏幕范围内。浏览器点击、拖拽、滚动的目标越界时直接拒绝；普通移动会夹取到视口边界。

  • 删除、清空、格式化、批量提交、关闭全部等语义会触发 MCP elicitation。没有用户明确批准就不执行。

  • 浏览器页面还可用 data-openmouse-risk="high" 标记高风险控件。

目录

src/mouse_mcp/           Python MCP 服务、云浏览器、桌面视觉观察、自动路由
browser-extension/       Chromium MV3 扩展、网页虚拟鼠标、截图
demo/                    浏览器人工测试页
docs/                    架构、部署、Agent 提示词、工具与风险文档
examples/                OpenClaw 和通用 MCP 配置
scripts/                 Windows 安装、注册和检查脚本
tests/                   单元测试、浏览器 E2E、stdio E2E
codex/                   Codex Skill 源文件
Dockerfile               云端 Playwright Chromium 镜像

快速开始

作为 Codex 插件安装

先安装 Python 依赖：

python -m pip install "git+https://github.com/osca5771666/openmouse-mcp.git"

再从 GitHub marketplace 安装插件：

powershell -ExecutionPolicy Bypass -File .\scripts\install-codex-plugin.ps1

等价命令：

codex plugin marketplace add osca5771666/openmouse-mcp --ref main
codex plugin add openmouse@openmouse

插件会同时注册 openmouse MCP 和 openmouse-computer-use Skill。新开 Codex 会话后生效。

从源码安装

在 PowerShell 中运行：

powershell -ExecutionPolicy Bypass -File .\scripts\install.ps1

然后：

  1. 打开 chrome://extensions，启用开发者模式，选择“加载已解压的扩展程序”，目录选 browser-extension。

  2. 打开扩展选项，填入安装脚本输出的浏览器令牌。

  3. 运行 .\.venv\Scripts\python.exe -m mouse_mcp --diagnose 验证桥状态。

  4. 运行 scripts\install-codex-plugin.ps1 安装 Codex 插件，或按 OpenClaw 部署说明 注册其他 MCP 宿主。

  5. 将 Agent 系统提示词 加入 Agent 的系统提示词或工作区说明。

  6. 云端使用 CLOUD.md 的 Streamable HTTP 或 Docker 配置。

已验证

  • Python 单元测试：40 项通过。

  • Node 人类化轨迹测试：3 项通过。

  • Chromium 真实扩展 E2E：单击、双击、拖拽选择、滚动、截图、高风险拦截通过。

  • MCP stdio E2E：初始化、11 个工具发现、桌面观察和截图通过。

  • 云端 Playwright E2E：页面打开、可信点击、截图通过。

  • 云 HTTP E2E：Bearer 鉴权、Streamable HTTP、云端页面操作通过。

  • Windows 桌面观察：任务栏矩形、活动窗口、显示器、窗口清单和 1920×1200 截图通过。

  • Codex 0.147.0：真实 tool_search 发现和 mouse_status / computer_observe 调用通过。

文档

许可

MIT

Available Tools

11 tools
app_openOpen or focus a local applicationA
Idempotent

Fast path for local applications. Focuses an existing running window, or launches a known executable/Start Menu application. Supports aliases such as feishu, notepad, explorer, cmd, powershell, calculator, edge, and chrome.

Inputs: name (required). Returns: result

browser_openOpen a URL directly in a browserA
Idempotent

Fast path for opening a known http:// or https:// URL. On Windows this launches the default browser, Edge, or Chrome directly without screenshots or mouse movement. In cloud mode it navigates the embedded Playwright browser and blocks private addresses by default.

Inputs: url (required), browser. Returns: result

computer_observeObserve the active computer surfaceA
Read-onlyIdempotent

Capture the active desktop/browser and return richer context. On Windows this includes the active window, taskbar rectangles, monitors and visible application windows. Use this before visual targeting and again for verification.

Inputs: no named parameters.

mouse_clickClick with the mouseA
Destructive

Move to x/y if supplied, then perform one left/right click or a left double click. intent is mandatory and must state the expected real-world effect. Destructive actions, bulk submissions, and close-all operations trigger an explicit user prompt.

Inputs: x (required), y (required), button (required), clicks (required), +2 more. Returns: result

mouse_dragDrag with the mouseA
Destructive

Drag from one point to another using a human-like curved path. intent is mandatory. High-risk semantic intent triggers an explicit user prompt.

Inputs: to_x (required), to_y (required), button (required), from_x (required), +3 more. Returns: result

mouse_moveMove the mouse cursorA

Move the active mouse cursor along a human-like curved path. Browser coordinates use CSS viewport pixels; Windows coordinates use physical desktop pixels. Out-of-range move targets are clamped to the surface.

Inputs: x (required), y (required). Returns: result

mouse_positionRead the mouse positionA
Read-onlyIdempotent

Read the current virtual browser cursor or real Windows cursor position.

Inputs: no named parameters. Returns: result

mouse_screenshotCapture the active mouse surfaceA
Read-onlyIdempotent

Capture the active local browser tab, the cloud browser, or the full Windows desktop. The text content includes image/viewport dimensions and the coordinate space. Use that metadata to map a visual target into mouse tool coordinates.

Inputs: no named parameters.

mouse_scrollScroll with the mouse wheelA

Scroll up or down at the current cursor location. One click is one wheel notch. intent is mandatory; high-risk semantic intent is confirmed first.

Inputs: clicks (required), intent (required), direction (required), x, +2 more. Returns: result

mouse_statusInspect mouse environment statusA
Read-onlyIdempotent

Read automatic routing state, browser companion freshness, viewport, platform, and consecutive failure count. This does not move the mouse.

Inputs: no named parameters. Returns: result

mouse_stopEmergency stopA
Idempotent

Immediately cancel the current mouse action, drain queued mouse actions, release held buttons, and send stop to the browser companion.

Inputs: no named parameters. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Deploy Server
    One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

    TDQS

    A3.9/5.0

    Scored across 11 tools

    Disambiguation4/5

    Most tools are cleanly separated: browser_open vs app_open split on URL vs executable, and the mouse_* family divides cleanly into action (move/click/drag/scroll), read (position/status), and control (stop). The only real overlap is computer_observe vs mouse_screenshot, both of which capture the active desktop/browser, though descriptions differentiate structural metadata from image capture.

    Naming Consistency4/5

    Eight of eleven tools follow a uniform resource_action pattern with the mouse_ prefix, and browser_open/app_open extend the same pattern to other resources. computer_observe is the one outlier in prefix family naming, but it remains readable and unambiguous.

    Tool Count5/5

    Eleven tools sit squarely in the well-scoped range for a desktop/browser automation server. Each mouse primitive (move, click, drag, scroll, stop, position, status, screenshot) earns its place without redundancy.

    Completeness3/5

    The pointer surface is thorough, but there is no keyboard/typing or key-press tool, which severely limits any workflow requiring text entry, form filling, or shortcuts in the browser or apps. Browser coverage also stops at opening a URL, with no navigation, tab, or element-query operations, leaving agents at dead ends for interactive pages.

    Maintenance

    ActivityMaintained
    ResponsivenessOutdated data

    Resources

    Related MCP Connectors

    Related MCP Servers

    • dsh-desktop-operator

      zjh02249
      A
      license
      Not graded
      quality
      B
      maintenance
      Enables safe Windows desktop automation and computer use through natural language, including window observation, UI Automation, and execution of verified actions like clicking, typing, and scrolling.
      2
      MIT

    • Open Computer Use

      aibo204
      A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to observe and control the user's live desktop by listing apps, reading accessibility trees, and performing clicks, typing, scrolling, dragging, and other input actions with per-action approvals and local audit archives.
      1
      MIT

    • ORVEX

      ujjawalsol
      A
      license
      Not graded
      quality
      C
      maintenance
      Enables AI assistants to drive Windows desktop automation — opening and interacting with applications, native UI controls, keyboard/mouse input, files, processes, clipboard, and Chrome/Edge browsing via UI Automation and CDP. All actions pass through a safety policy with human approval, emergency stop, and deterministic verification.
      MIT