remote_macos_get_screen

Instructions

Implementation Reference

• src/action_handlers.py:43-76 (handler)

  Main handler function for the 'remote_macos_get_screen' tool. Uses environment variables to connect via VNC, calls capture_vnc_screen helper, base64 encodes the PNG screenshot, and returns ImageContent with dimensions text.

  async def handle_remote_macos_get_screen(arguments: dict[str, Any]) -> list[types.TextContent | types.ImageContent | types.EmbeddedResource]:
      """Connect to a remote MacOs machine and get a screenshot of the remote desktop."""
      # Use environment variables
      host = MACOS_HOST
      port = MACOS_PORT
      password = MACOS_PASSWORD
      username = MACOS_USERNAME
      encryption = VNC_ENCRYPTION
  
      # Capture screen using helper method
      success, screen_data, error_message, dimensions = await capture_vnc_screen(
          host=host, port=port, password=password, username=username, encryption=encryption
      )
  
      if not success:
          return [types.TextContent(type="text", text=error_message)]
  
      # Encode image in base64
      base64_data = base64.b64encode(screen_data).decode('utf-8')
  
      # Return image content with dimensions
      width, height = dimensions
      return [
          types.ImageContent(
              type="image",
              data=base64_data,
              mimeType="image/png",
              alt_text=f"Screenshot from remote MacOs machine at {host}:{port}"
          ),
          types.TextContent(
              type="text",
              text=f"Image dimensions: {width}x{height}"
          )
      ]

• src/mcp_remote_macos_use/server.py:262-263 (registration)

  Tool call dispatch in @server.call_tool() that routes requests for 'remote_macos_get_screen' to the handler function.

  if name == "remote_macos_get_screen":
      return await handle_remote_macos_get_screen(arguments)

• src/mcp_remote_macos_use/server.py:133-140 (schema)

  MCP Tool registration in @server.list_tools(), defining name, description, and empty input schema (no parameters required).

  types.Tool(
      name="remote_macos_get_screen",
      description="Connect to a remote MacOs machine and get a screenshot of the remote desktop. Uses environment variables for connection details.",
      inputSchema={
          "type": "object",
          "properties": {}
      },
  ),

• src/vnc_client.py:19-91 (helper)

  Key helper function that performs VNC connection using Apple Authentication, captures the screen framebuffer (handling RAW/COPY_RECT encodings), scales the image to 1366x768 PNG, and returns bytes with dimensions.

  async def capture_vnc_screen(host: str, port: int, password: str, username: Optional[str] = None,
                               encryption: str = "prefer_on") -> Tuple[bool, Optional[bytes], Optional[str], Optional[Tuple[int, int]]]:
      """Capture a screenshot from a remote MacOs machine.
  
      Args:
          host: remote MacOs machine hostname or IP address
          port: remote MacOs machine port
          password: remote MacOs machine password
          username: remote MacOs machine username (optional)
          encryption: Encryption preference (default: "prefer_on")
  
      Returns:
          Tuple containing:
          - success: True if the operation was successful
          - screen_data: PNG image data if successful, None otherwise
          - error_message: Error message if unsuccessful, None otherwise
          - dimensions: Tuple of (width, height) if successful, None otherwise
      """
      logger.debug(f"Connecting to remote MacOs machine at {host}:{port} with encryption: {encryption}")
  
      # Initialize VNC client
      vnc = VNCClient(host=host, port=port, password=password, username=username, encryption=encryption)
  
      try:
          # Connect to remote MacOs machine
          success, error_message = vnc.connect()
          if not success:
              detailed_error = f"Failed to connect to remote MacOs machine at {host}:{port}. {error_message}\n"
              detailed_error += "This VNC client only supports Apple Authentication (protocol 30). "
              detailed_error += "Please ensure the remote MacOs machine supports this protocol. "
              detailed_error += "For macOS, enable Screen Sharing in System Preferences > Sharing."
              return False, None, detailed_error, None
  
          # Capture screen
          screen_data = vnc.capture_screen()
  
          if not screen_data:
              return False, None, f"Failed to capture screenshot from remote MacOs machine at {host}:{port}", None
  
          # Save original dimensions for reference
          original_dims = (vnc.width, vnc.height)
  
          # Scale the image to FWXGA resolution (1366x768)
          target_width, target_height = 1366, 768
  
          try:
              # Convert bytes to PIL Image
              image_data = io.BytesIO(screen_data)
              img = Image.open(image_data)
  
              # Resize the image to the target resolution
              scaled_img = img.resize((target_width, target_height), Image.Resampling.LANCZOS)
  
              # Convert back to bytes
              output_buffer = io.BytesIO()
              scaled_img.save(output_buffer, format='PNG')
              output_buffer.seek(0)
              scaled_screen_data = output_buffer.getvalue()
  
              logger.info(f"Scaled image from {original_dims[0]}x{original_dims[1]} to {target_width}x{target_height}")
  
              # Return success with scaled screen data and target dimensions
              return True, scaled_screen_data, None, (target_width, target_height)
  
          except Exception as e:
              logger.warning(f"Failed to scale image: {str(e)}. Returning original image.")
              # Return the original image if scaling fails
              return True, screen_data, None, original_dims
  
      finally:
          # Close VNC connection
          vnc.close()