Skip to main content
Glama
LaAlquimia

AIsistent

by LaAlquimia
OverviewSchemaRelated ServersScoreDiscussions
Python

AIsistent

MCP server for non-intrusive RDP automation. OCR, YOLO button detection, and click injection — zero footprint on the remote machine.

macOS

Windows

Linux

Capture

RDP WindowID

MSS fullscreen

MSS fullscreen

OCR

Apple Vision (~0.05s)

EasyOCR (CPU/CUDA)

EasyOCR (CPU/CUDA)

Detection

YOLO MPS GPU

YOLO CUDA GPU

YOLO CPU

Click

pyautogui

pyautogui

pyautogui

Quick Start

# macOS (Apple Silicon)
pip install aistent[apple]

# Windows / Linux (CPU)
pip install aistent[cpu]

# Windows (NVIDIA CUDA)
pip install aistent[cuda]

# Everything
pip install aistent[all]

aisistent

From source

git clone https://github.com/LaAlquimia/AIsistent
cd AIsistent
pip install -e ".[all]"
aisistent

Related MCP server: Computer Use MCP

Tools

Tool

Description

capture_rdp_screen

Capture RDP window (macOS) or full screen (Win/Linux)

run_apple_ocr

OCR: Apple Vision (Mac) or EasyOCR (CPU/CUDA)

detect_rdp_buttons

YOLOv8 button detection on CUDA / MPS / CPU

inject_rdp_click

Click injection at percentage-based coordinates

benchmark

Run performance benchmark on OCR + YOLO (returns JSON)

Configuration

Env var

Default

Description

AISISTENT_YOLO_WEIGHTS

models/weights/best.pt

Path to YOLO weights file

AISISTENT_TEMP_DIR

temp_captures/

Screenshot temp directory

Cross-Platform Hardware Detection

Hardware is auto-detected at import time in aisistent/platform.py:

Backend

Detection

dtype

Use Case

CUDA (NVIDIA)

torch.cuda.is_available()

float16

Windows/Linux with NVIDIA GPU

MPS (Apple)

torch.backends.mps.is_available()

float16

macOS Apple Silicon (M1–M4)

CPU

fallback

float32

Any OS, no GPU

Install GPU backends

# CUDA (NVIDIA)
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124

# MPS (Apple) — included in default torch on macOS
pip install torch torchvision

Benchmark

Run a quick performance test from the command line:

aisistent-bench                                 # captures a real screenshot & benchmarks
aisistent-bench --synthetic                     # use synthetic image (no screen capture)
aisistent-bench --skip-ocr                      # YOLO only
aisistent-bench --image screenshot.png          # use your own image
aisistent-bench --device cpu                    # force CPU backend

Or via MCP tool call:

benchmark(image_base64: "")              # empty = real screenshot, or pass base64

Real-world performance (Apple MacBook M5 — MPS GPU)

Benchmark on a real 1920×1080 desktop screenshot with text, buttons, and UI elements:

Platform : macOS (Apple Silicon M5)
Device   : MPS
──────────────────────────────────────
Capture  :  0.22s
OCR      :  0.43s  —  102 texts detected
YOLO     :  0.76s  —   59 buttons detected
──────────────────────────────────────
Total    : ~1.4s

Step

Time

Throughput

Screen capture

~0.22s

—

Apple Vision OCR (Neural Engine)

~0.43s

~237 texts/sec

YOLOv8 inference (MPS float16)

~0.76s

~78 detections/sec

End-to-end

~1.4s

—

These numbers are from the same machine running both the MCP server and the benchmark — no overhead from network or RDP. On NVIDIA CUDA, YOLO inference is typically 0.3–0.5s (RTX 3060+).

MCP Client Integration

AIsistent implements the standard MCP (Model Context Protocol), so it works with any MCP client. Below are detailed setup instructions for each platform.

Hermes MCP

Hermes is an AI agent that uses MCP tools to interact with your computer.

1. Install AIsistent

# macOS (Apple Silicon — Apple Vision OCR + MPS GPU)
pip install aistent[apple]

# Windows/Linux CPU
pip install aistent[cpu]

# Windows with NVIDIA GPU
pip install aistent[cuda]

2. Locate Hermes config file

OS

Path

macOS

~/.config/hermes/config.json

Windows

%APPDATA%\hermes\config.json

Linux

~/.config/hermes/config.json

3. Add AIsistent to Hermes config

{
  "mcpServers": {
    "aisistent": {
      "command": "aisistent",
      "type": "stdio"
    }
  }
}

If AIsistent is not on your PATH, use the full path:

{
  "mcpServers": {
    "aisistent": {
      "command": "/path/to/venv/bin/aisistent",
      "type": "stdio"
    }
  }
}

4. Start Hermes

hermes

Hermes will auto-discover AIsistent's tools on startup. You should see:

👁️ AIsistent — capture_rdp_screen, run_apple_ocr, detect_rdp_buttons, inject_rdp_click, benchmark

Example: Hermes asks AIsistent to read the screen

> What's on my screen right now?

Hermes will:

  1. Call capture_rdp_screen → gets screenshot

  2. Call run_apple_ocr(image) → extracts all text

  3. Call detect_rdp_buttons(image) → finds buttons

  4. Returns a structured summary of what's on screen

Example: Hermes clicks a button via AIsistent

> Open Chrome and go to youtube.com

Hermes will:

  1. Call capture_rdp_screen → sees desktop

  2. Call detect_rdp_buttons(image) → finds Chrome icon coordinates

  3. Call inject_rdp_click(12.5, 8.3) → clicks Chrome

  4. Repeats capture → detect → click until done

OpenCode

OpenCode is an agentic CLI that also supports MCP tools.

1. Install AIsistent

pip install aistent[all]

2. Add to OpenCode config

Create or edit ~/.config/opencode/opencode.jsonc:

{
  "mcpServers": {
    "aisistent": {
      "command": "aisistent",
      "type": "stdio"
    }
  }
}

Or per-project, add to .opencode.jsonc in your project root:

{
  "mcpServers": {
    "aisistent": {
      "command": "aisistent",
      "type": "stdio"
    }
  }
}

3. Verify it works

opencode

Then ask:

capture the screen and tell me what applications are open

OpenCode will call capture_rdp_screen → run_apple_ocr and return the result.

Claude Desktop

Claude Desktop supports MCP tools via its config file.

1. Locate Claude Desktop config

OS

Path

macOS

~/Library/Application Support/Claude/claude_desktop_config.json

Windows

%APPDATA%\Claude\claude_desktop_config.json

2. Add AIsistent

{
  "mcpServers": {
    "aisistent": {
      "command": "aisistent",
      "type": "stdio"
    }
  }
}

3. Restart Claude Desktop

Claude will show a hammer icon with AIsistent's available tools.

Cursor

Cursor IDE supports MCP tools.

1. Open Cursor settings

Settings → Features → MCP Servers

2. Add server

Name: AIsistent
Type: stdio
Command: aisistent

3. Use in chat

In Cursor's AI chat, type:

@aisistent capture the screen and detect buttons

Any MCP Client (generic stdio)

If your MCP client uses stdio transport, the configuration is always the same pattern:

{
  "mcpServers": {
    "aisistent": {
      "command": "aisistent",
      "type": "stdio"
    }
  }
}

For HTTP/SSE transport instead of stdio:

# Start AIsistent as an SSE server on port 8100
python -c "from aisistent.server import mcp; mcp.run(transport='sse', port=8100)"

Then configure:

{
  "mcpServers": {
    "aisistent": {
      "url": "http://localhost:8100/sse",
      "type": "sse"
    }
  }
}

winremote-mcp Integration

AIsistent works alongside winremote-mcp for comprehensive Windows remote management. Run both MCP servers:

aisistent &                              # AIsistent (stdio)
winremote-mcp --transport sse --port 8100 # winremote-mcp (SSE)

AIsistent handles the visual layer (OCR, detection, clicks) while winremote-mcp handles system operations (registry, services, processes, files, etc.).

Docs

Project Structure

AIsistent/
├── aisistent/
│   ├── __init__.py      # Version
│   ├── __main__.py      # Entry point
│   ├── server.py        # MCP server + tools
│   ├── platform.py      # OS + device detection
│   ├── config.py        # Settings management
│   ├── capture.py       # Screen capture (RDP / MSS)
│   ├── ocr.py           # OCR (Apple Vision / EasyOCR)
│   ├── detection.py     # YOLOv8 button detection
│   ├── action.py        # Click injection
│   └── benchmark.py     # Performance benchmark
├── docs/                # Documentation
├── pyproject.toml
└── README.md

License

MIT

🇪🇸 AIsistent

Servidor MCP para automatización RDP no intrusiva. OCR, detección de botones con YOLOv8 e inyección de clics — sin instalar nada en la máquina remota.

macOS

Windows

Linux

Captura

RDP WindowID

MSS pantalla completa

MSS pantalla completa

OCR

Apple Vision (~0.05s)

EasyOCR (CPU/CUDA)

EasyOCR (CPU/CUDA)

Detección

YOLO GPU MPS

YOLO GPU CUDA

YOLO CPU

Click

pyautogui

pyautogui

pyautogui

Inicio Rápido

# macOS (Apple Silicon)
pip install aistent[apple]

# Windows / Linux (CPU)
pip install aistent[cpu]

# Windows (NVIDIA CUDA)
pip install aistent[cuda]

aisistent

Benchmark

aisistent-bench                          # pantallazo real
aisistent-bench --synthetic              # imagen sintética
aisistent-bench --image captura.png      # imagen propia

Resultados reales (MacBook M5 — MPS)

Paso

Tiempo

Elementos

Captura

~0.22s

—

OCR (Apple Vision)

~0.43s

102 textos

YOLO (MPS float16)

~0.76s

59 botones

Total

~1.4s

—

Integración con MCP Clients

Hermes MCP

Añade AIsistent como servidor MCP en ~/.config/hermes/config.json:

{
  "mcpServers": {
    "aisistent": {
      "command": "aisistent",
      "type": "stdio"
    }
  }
}

Luego inicia Hermes: hermes

OpenCode

Añade en ~/.config/opencode/opencode.jsonc:

{
  "mcpServers": {
    "aisistent": {
      "command": "aisistent",
      "type": "stdio"
    }
  }
}

Claude Desktop

Añade en ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "aisistent": {
      "command": "aisistent",
      "type": "stdio"
    }
  }
}

Licencia

MIT

Install Server
A
license - permissive license
A
quality
B
maintenance

How are these scores calculated?

Maintenance

–Maintainers
–Response time
–Release cycle
–Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/LaAlquimia/AIsistent'

If you have feedback or need assistance with the MCP directory API, please join our Discord server