TarsUITarsBridge
This MCP server bridges to UI-TARS desktop automation, letting you remotely control a desktop environment through five MCP tools:
screenshot— Capture the current screen as PNG or JPEG (default PNG).click— Click at specified (x, y) screen coordinates.type— Type text into the focused input field.scroll— Scroll up/down/left/right by a given pixel amount (default 300).browser_open— Open a URL in the default browser.
Provides tools for interacting with ByteDance's UI-TARS desktop automation client, enabling remote control of desktop environments via screenshot, click, type, scroll, and browser open operations through MCP.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@TarsUITarsBridge open https://github.com"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
TarsUITarsBridge
MCP Bridge for UI-TARS — connects TSG gateway to UI-TARS desktop automation via Model Context Protocol.
关于作者
你好!我是这个项目的作者,一名正在读小学的学生。 这个项目从规划到代码实现,完全由我自己主导设计,并借助 AI 工具完成。每一行代码、每一个功能模块,都凝聚了我对技术的热爱与坚持。 如果你认可这个项目,或者觉得它对你有帮助,希望能给我一个 Star 支持,这对我意义重大!
Related MCP server: desktop-touch-mcp
欢迎支持
如果你在使用过程中遇到问题,欢迎提交 Issue 或参与讨论
如果你觉得项目不错,请点一个 Star,这是对我最大的鼓励
也欢迎分享给身边有需要的朋友,让更多人看到这个项目
后续功能迭代会继续推进,你的支持是我持续更新的动力
⚠️ 维护说明
本项目依赖 AI 服务套餐,若因套餐额度耗尽或服务商调整,可能导致更新暂时停滞。 如遇较长时间未更新,请理解这是 AI 服务限制所致,并非项目放弃维护。 恢复服务后会第一时间继续推进,感谢你的耐心与包容。
Overview
TarsUITarsBridge wraps the UI-TARS desktop automation client as an MCP server with stdio transport, enabling TSG (TarsSecureGuard) gateway to remotely control desktop environments through a standardized protocol.
Independent repository — evolves separately from TSG gateway
Auto-include ready — TSG gateway can mount this bridge automatically via configuration
MCP-native — speaks standard Model Context Protocol (stdio transport in v0.1.0)
Quick Start
# Install dependencies
npm install
# Build
npm run build
# Run (mock mode if no UI-TARS endpoint configured)
npm startConfiguration
Environment Variable | Description | Default |
| UI-TARS API endpoint URL | (mock mode) |
| API key for the endpoint | — |
| Model name |
|
| Request timeout |
|
MCP Tools (v0.1.0)
Tool | Description |
| Capture screen (PNG/JPEG) |
| Click at (x, y) coordinates |
| Type text into focused input |
| Scroll in direction (up/down/left/right) |
| Open a URL in the default browser |
Roadmap
v0.1.0 — MVP: stdio transport + 5 tools + mock endpoint ✅
v0.2.0 — HTTP transport dual-mode + official MCP server mount
v0.3.0 — TSG gateway auto-include protocol
v0.4.0 — Tauri sidecar integration (cross-platform)
v0.5.0 — Self-hosted vLLM deployment docs
v1.0.0 — 10+ tools, multi-device routing, penetration test report
Integration with TSG Gateway
In TSG gateway configuration, enable the bridge:
{
"bridges": {
"ui_tars": {
"enabled": true,
"command": "npx tars-ui-tars-bridge",
"env": {
"UI_TARS_ENDPOINT": "http://localhost:8080"
}
}
}
}License
Apache-2.0 — see LICENSE
Changelog
v1.0.1
Version bump with stability improvements
Available Tools
5 toolsbrowser_openC
Open a URL in the default browser
Inputs: url (required).
clickC
Click at the specified screen coordinates
Inputs: x (required), y (required).
screenshotC
Capture a screenshot of the current screen
Inputs: format.
scrollC
Scroll the screen in the specified direction
Inputs: direction (required), amount.
typeA
Type the given text into the focused input field
Inputs: text (required).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 5 tools
Each tool performs a clearly distinct action: screenshot, click, type, scroll, and browser_open. There is no overlap between their purposes.
Most tool names are single verbs (screenshot, click, type, scroll), which is consistent and readable. browser_open uses a noun_verb pattern, a minor deviation from the verb-centric convention.
The 5 tools are well-scoped for a UI automation bridge, covering essential interactions without excess.
Provides core UI automation functions (capture, click, type, scroll, open URL) but lacks operations like key presses, drag, or window management, which are common in such bridges.
Maintenance
Related MCP Connectors
Securely control computers you explicitly pair through files, terminals, processes, screenshots, desktop UI/input, clipboard, browser automation, diagnostics, and document tools.
A cloud Linux desktop for your AI: browse, click, type, read the screen and run commands.
Operate Linux, macOS and Windows from your LLM. Every action runs through an auditable allowlist.
Connect AI assistants to your approved computer for files, commands, browser and desktop tasks.
Related MCP Servers
- AlicenseBqualityDmaintenanceEnables AI assistants to automate and test Tauri desktop applications through the Model Context Protocol. It provides tools for app management, UI interaction, and state inspection across multiple platforms without requiring CDP dependencies.141MIT
- AlicenseAqualityCmaintenanceAllows AI clients to see and control Windows 10/11 desktops via MCP, with screenshots, UI Automation, Chrome CDP, keyboard/mouse, and terminal using semantic element targeting.30MIT
- AlicenseNot gradedqualityCmaintenanceEnables Windows GUI automation through the Model Context Protocol, providing tools, workflows, and generative capabilities for natural language-driven interaction with desktop applications.MIT
- AlicenseNot gradedqualityCmaintenanceEnables an AI client to inspect and control an interactive Windows desktop, including screenshots, mouse and keyboard input, Windows UI Automation, PowerShell commands with managed jobs, and authenticated file uploads/downloads. Connects over remote MCP Streamable HTTP or local stdio without a separate host-side gateway.Apache 2.0