🤖 Automation MCP

Automation MCP is a Model Context Protocol (MCP) server that provides AI models with complete desktop automation capabilities on macOS. It enables AI assistants to:

• 🖱️ Control your mouse (click, move, scroll, drag)

• ⌨️ Type and send keyboard input (including system shortcuts)

• 📸 Take screenshots and analyze screen content

• 🪟 Manage windows (focus, move, resize, minimize)

• 🎯 Interact with UI elements through coordinates

• 🎨 Analyze screen colors and highlight regions

• 🔍 Wait for images to appear on screen

🚀 Quick start

Make sure you have furi installed, and then run the following command:

furi add ashwwwin/automation-mcp

followed by:

furi start ashwwwin/automation-mcp

and you're done! (or you can just use the furi desktop app for no cli).

🥲 Normal start (without furi)

Prerequisites

• Bun runtime - Install with: curl -fsSL https://bun.sh/install | bash

1. Clone and Install

git clone https://github.com/ashwwwin/automation-mcp.git
cd automation-mcp
bun install

2. Start the Server

# Start with HTTP transport (recommended for web apps)
bun run index.ts

# Or start with stdio transport (for command line tools)
bun run index.ts --stdio

3. Grant Permissions

On first run, macOS will ask for permissions. You must grant these for full functionality:

1. Accessibility - Allows keyboard/mouse control

2. Screen Recording - Enables screenshots and screen analysis

Or manually enable in: System Settings → Privacy & Security → Accessibility/Screen Recording

🛠️ Available Tools

🖱️ Mouse Control

• mouseClick - Click at coordinates with left/right/middle button

• mouseDoubleClick - Double-click at coordinates

• mouseMove - Move cursor to position

• mouseGetPosition - Get current cursor location

• mouseScroll - Scroll in any direction

• mouseDrag - Drag from current position to target

• mouseButtonControl - Press/release mouse buttons

• mouseMovePath - Follow a smooth path with multiple points

⌨️ Keyboard Input

• type - Type text or press key combinations

• keyControl - Advanced key press/release control

• systemCommand - Common shortcuts (copy, paste, undo, save, etc.)

📸 Screen Capture & Analysis

• screenshot - Capture full screen, regions, or specific windows

• screenInfo - Get screen dimensions

• screenHighlight - Highlight screen regions visually

• colorAt - Get color of any pixel

• waitForImage - Wait for images to appear (template matching)

🪟 Window Management

• getWindows - List all open windows

• getActiveWindow - Get current active window

• windowControl - Focus, move, resize, minimize windows

🔒 Security & Permissions

1. Accessibility - Required for:

   • Mouse clicks and movement

   • Keyboard input simulation

   • Window management

2. Screen Recording - Required for:

   • Taking screenshots

   • Screen analysis

   • Color detection

🚀 Integration Examples

With Claude Desktop + furi

If you've already configured furi with Claude Desktop, you don't need to do anything.

Add to your MCP configuration:

{
  "mcpServers": {
    "furi": {
      "command": "furi",
      "args": ["connect"]
    }
  }
}

With Claude Desktop

Add to your MCP configuration:

{
  "mcpServers": {
    "automation": {
      "command": "bun",
      "args": ["run", "/path/to/automation-mcp/index.ts", "--stdio"]
    }
  }
}

🐛 Troubleshooting

Common Issues

Permission Denied Errors

• Ensure Accessibility and Screen Recording permissions are granted

• Ensure Xcode Command Line Tools: xcode-select --install

🙋‍♂️ Support

Having issues? Check the troubleshooting section above or open an issue with:

• Your operating system and version

• Error messages

• Steps to reproduce