Skip to main content
Glama
senseisven

MCP Remote macOS Control Server

by senseisven
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Hybrid

remote_macos_mouse_click

Click at specified coordinates on a remote macOS machine with automatic coordinate scaling for accurate pointer control.

Instructions

Perform a mouse click at specified coordinates on a remote MacOs machine, with automatic coordinate scaling. Uses environment variables for connection details.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
xYesX coordinate for mouse position (in source dimensions)
yYesY coordinate for mouse position (in source dimensions)
source_widthNoWidth of the reference screen for coordinate scaling
source_heightNoHeight of the reference screen for coordinate scaling
buttonNoMouse button (1=left, 2=middle, 3=right)

Implementation Reference

  • src/action_handlers.py:156-219 (handler)
    The core handler function implementing the remote_macos_mouse_click tool. Connects to VNC server, scales input coordinates to target screen resolution, performs the mouse click via VNCClient.send_mouse_click, and returns status details.
    def handle_remote_macos_mouse_click(arguments: dict[str, Any]) -> list[types.TextContent | types.ImageContent | types.EmbeddedResource]:
    """Perform a mouse click action on a remote MacOs machine."""
    # Use environment variables
    host = MACOS_HOST
    port = MACOS_PORT
    password = MACOS_PASSWORD
    username = MACOS_USERNAME
    encryption = VNC_ENCRYPTION

    # Get required parameters from arguments
    x = arguments.get("x")
    y = arguments.get("y")
    source_width = int(arguments.get("source_width", 1366))
    source_height = int(arguments.get("source_height", 768))
    button = int(arguments.get("button", 1))

    if x is None or y is None:
        raise ValueError("x and y coordinates are required")

    # Ensure source dimensions are positive
    if source_width <= 0 or source_height <= 0:
        raise ValueError("Source dimensions must be positive values")

    # Initialize VNC client
    vnc = VNCClient(host=host, port=port, password=password, username=username, encryption=encryption)

    # Connect to remote MacOs machine
    success, error_message = vnc.connect()
    if not success:
        error_msg = f"Failed to connect to remote MacOs machine at {host}:{port}. {error_message}"
        return [types.TextContent(type="text", text=error_msg)]

    try:
        # Get target screen dimensions
        target_width = vnc.width
        target_height = vnc.height

        # Scale coordinates
        scaled_x = int((x / source_width) * target_width)
        scaled_y = int((y / source_height) * target_height)

        # Ensure coordinates are within the screen bounds
        scaled_x = max(0, min(scaled_x, target_width - 1))
        scaled_y = max(0, min(scaled_y, target_height - 1))

        # Single click
        result = vnc.send_mouse_click(scaled_x, scaled_y, button, False)

        # Prepare the response with useful details
        scale_factors = {
            "x": target_width / source_width,
            "y": target_height / source_height
        }

        return [types.TextContent(
            type="text",
            text=f"""Mouse click (button {button}) from source ({x}, {y}) to target ({scaled_x}, {scaled_y}) {'succeeded' if result else 'failed'}
Source dimensions: {source_width}x{source_height}
Target dimensions: {target_width}x{target_height}
Scale factors: {scale_factors['x']:.4f}x, {scale_factors['y']:.4f}y"""
        )]
    finally:
        # Close VNC connection
        vnc.close()
  • src/mcp_remote_macos_use/server.py:188-202 (schema)
    Input schema definition for the remote_macos_mouse_click tool, specifying parameters for coordinates, scaling, and button.
    types.Tool(
    name="remote_macos_mouse_click",
    description="Perform a mouse click at specified coordinates on a remote MacOs machine, with automatic coordinate scaling. Uses environment variables for connection details.",
    inputSchema={
        "type": "object",
        "properties": {
            "x": {"type": "integer", "description": "X coordinate for mouse position (in source dimensions)"},
            "y": {"type": "integer", "description": "Y coordinate for mouse position (in source dimensions)"},
            "source_width": {"type": "integer", "description": "Width of the reference screen for coordinate scaling", "default": 1366},
            "source_height": {"type": "integer", "description": "Height of the reference screen for coordinate scaling", "default": 768},
            "button": {"type": "integer", "description": "Mouse button (1=left, 2=middle, 3=right)", "default": 1}
        },
        "required": ["x", "y"]
    },
),
  • src/mcp_remote_macos_use/server.py:274-275 (registration)
    Dispatch logic in the MCP server's call_tool handler that routes requests for 'remote_macos_mouse_click' to the handler function.
    elif name == "remote_macos_mouse_click":
    return handle_remote_macos_mouse_click(arguments)
  • src/mcp_remote_macos_use/server.py:130-251 (registration)
    The tool is registered in the list_tools method by including it in the returned list of available tools.
    async def handle_list_tools() -> list[types.Tool]:
    """List available tools"""
    return [
        types.Tool(
            name="remote_macos_get_screen",
            description="Connect to a remote MacOs machine and get a screenshot of the remote desktop. Uses environment variables for connection details.",
            inputSchema={
                "type": "object",
                "properties": {}
            },
        ),
        types.Tool(
            name="remote_macos_mouse_scroll",
            description="Perform a mouse scroll at specified coordinates on a remote MacOs machine, with automatic coordinate scaling. Uses environment variables for connection details.",
            inputSchema={
                "type": "object",
                "properties": {
                    "x": {"type": "integer", "description": "X coordinate for mouse position (in source dimensions)"},
                    "y": {"type": "integer", "description": "Y coordinate for mouse position (in source dimensions)"},
                    "source_width": {"type": "integer", "description": "Width of the reference screen for coordinate scaling", "default": 1366},
                    "source_height": {"type": "integer", "description": "Height of the reference screen for coordinate scaling", "default": 768},
                    "direction": {
                        "type": "string",
                        "description": "Scroll direction",
                        "enum": ["up", "down"],
                        "default": "down"
                    }
                },
                "required": ["x", "y"]
            },
        ),
        types.Tool(
            name="remote_macos_send_keys",
            description="Send keyboard input to a remote MacOs machine. Uses environment variables for connection details.",
            inputSchema={
                "type": "object",
                "properties": {
                    "text": {"type": "string", "description": "Text to send as keystrokes"},
                    "special_key": {"type": "string", "description": "Special key to send (e.g., 'enter', 'backspace', 'tab', 'escape', etc.)"},
                    "key_combination": {"type": "string", "description": "Key combination to send (e.g., 'ctrl+c', 'cmd+q', 'ctrl+alt+delete', etc.)"}
                },
                "required": []
            },
        ),
        types.Tool(
            name="remote_macos_mouse_move",
            description="Move the mouse cursor to specified coordinates on a remote MacOs machine, with automatic coordinate scaling. Uses environment variables for connection details.",
            inputSchema={
                "type": "object",
                "properties": {
                    "x": {"type": "integer", "description": "X coordinate for mouse position (in source dimensions)"},
                    "y": {"type": "integer", "description": "Y coordinate for mouse position (in source dimensions)"},
                    "source_width": {"type": "integer", "description": "Width of the reference screen for coordinate scaling", "default": 1366},
                    "source_height": {"type": "integer", "description": "Height of the reference screen for coordinate scaling", "default": 768}
                },
                "required": ["x", "y"]
            },
        ),
        types.Tool(
            name="remote_macos_mouse_click",
            description="Perform a mouse click at specified coordinates on a remote MacOs machine, with automatic coordinate scaling. Uses environment variables for connection details.",
            inputSchema={
                "type": "object",
                "properties": {
                    "x": {"type": "integer", "description": "X coordinate for mouse position (in source dimensions)"},
                    "y": {"type": "integer", "description": "Y coordinate for mouse position (in source dimensions)"},
                    "source_width": {"type": "integer", "description": "Width of the reference screen for coordinate scaling", "default": 1366},
                    "source_height": {"type": "integer", "description": "Height of the reference screen for coordinate scaling", "default": 768},
                    "button": {"type": "integer", "description": "Mouse button (1=left, 2=middle, 3=right)", "default": 1}
                },
                "required": ["x", "y"]
            },
        ),
        types.Tool(
            name="remote_macos_mouse_double_click",
            description="Perform a mouse double-click at specified coordinates on a remote MacOs machine, with automatic coordinate scaling. Uses environment variables for connection details.",
            inputSchema={
                "type": "object",
                "properties": {
                    "x": {"type": "integer", "description": "X coordinate for mouse position (in source dimensions)"},
                    "y": {"type": "integer", "description": "Y coordinate for mouse position (in source dimensions)"},
                    "source_width": {"type": "integer", "description": "Width of the reference screen for coordinate scaling", "default": 1366},
                    "source_height": {"type": "integer", "description": "Height of the reference screen for coordinate scaling", "default": 768},
                    "button": {"type": "integer", "description": "Mouse button (1=left, 2=middle, 3=right)", "default": 1}
                },
                "required": ["x", "y"]
            },
        ),
        types.Tool(
            name="remote_macos_open_application",
            description="Opens/activates an application and returns its PID for further interactions.",
            inputSchema={
                "type": "object",
                "properties": {
                    "identifier": {
                        "type": "string",
                        "description": "REQUIRED. App name, path, or bundle ID."
                    }
                },
                "required": ["identifier"]
            },
        ),
        types.Tool(
            name="remote_macos_mouse_drag_n_drop",
            description="Perform a mouse drag operation from start point and drop to end point on a remote MacOs machine, with automatic coordinate scaling.",
            inputSchema={
                "type": "object",
                "properties": {
                    "start_x": {"type": "integer", "description": "Starting X coordinate (in source dimensions)"},
                    "start_y": {"type": "integer", "description": "Starting Y coordinate (in source dimensions)"},
                    "end_x": {"type": "integer", "description": "Ending X coordinate (in source dimensions)"},
                    "end_y": {"type": "integer", "description": "Ending Y coordinate (in source dimensions)"},
                    "source_width": {"type": "integer", "description": "Width of the reference screen for coordinate scaling", "default": 1366},
                    "source_height": {"type": "integer", "description": "Height of the reference screen for coordinate scaling", "default": 768},
                    "button": {"type": "integer", "description": "Mouse button (1=left, 2=middle, 3=right)", "default": 1},
                    "steps": {"type": "integer", "description": "Number of intermediate points for smooth dragging", "default": 10},
                    "delay_ms": {"type": "integer", "description": "Delay between steps in milliseconds", "default": 10}
                },
                "required": ["start_x", "start_y", "end_x", "end_y"]
            },
        ),
    ]

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden. It mentions 'automatic coordinate scaling' and 'environment variables for connection details', which adds some behavioral context, but fails to disclose critical aspects like required permissions, network dependencies, error handling, or what happens if the remote machine is unavailable. For a remote control tool with zero annotation coverage, this is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core action and includes key behavioral details (coordinate scaling, environment variables) without unnecessary elaboration. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a remote control tool with no annotations and no output schema, the description is incomplete. It lacks information on error conditions, return values, security implications, or performance characteristics, leaving significant gaps for an AI agent to understand tool behavior fully.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, providing full parameter documentation. The description adds minimal value beyond the schema by mentioning 'automatic coordinate scaling', which relates to source_width and source_height parameters, but doesn't explain scaling mechanics or environmental variable usage. Baseline 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('perform a mouse click') and target ('on a remote MacOS machine'), distinguishing it from siblings like mouse_move or mouse_drag_n_drop by specifying click behavior. However, it doesn't explicitly differentiate from mouse_double_click, which is a similar click action.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions 'automatic coordinate scaling' and 'environment variables for connection details', which provides some context, but offers no explicit guidance on when to use this tool versus alternatives like mouse_double_click or mouse_drag_n_drop, nor any prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools