GhostDesk

Capture the screen, optionally cropped to a region.

Args: region: Area to capture (full screen if omitted). format: "webp" (default, smaller payload) or "png" (lossless). stabilize: Wait for the page to stop moving before capturing (max 2.5 s). Useful right after navigation.

Click at screen coordinates. Use coordinates from screen_shot() or inspect().

Returns a dict with:

• action: description of what was performed.

• screen_changed: whether the 200x200 px zone around the click visibly changed within 2 s. If false the click likely missed its target — retry with adjusted coordinates or take a new screen_shot().

• reaction_time_ms: how quickly the change was detected (ms).

Double-click at screen coordinates. Use for opening files or selecting words.

Returns a dict with:

• action: description of what was performed.

• screen_changed: whether the 200x200 px zone around the click visibly changed within 2 s. If false the click likely missed its target.

• reaction_time_ms: how quickly the change was detected (ms).

Drag from one position to another. Use for selecting text, moving items, or resizing.

Returns a dict with:

• action: description of what was performed.

• screen_changed: whether the 200x200 px zone around the drop point visibly changed within 2 s.

• reaction_time_ms: how quickly the change was detected (ms).

Scroll at a position. direction: up/down/left/right. amount: number of scroll steps (max 5).

Returns a dict with:

• action: description of what was performed.

• screen_changed: whether the 200x200 px zone around the scroll point visibly changed within 2 s. If false the page may already be at the scroll boundary.

• reaction_time_ms: how quickly the change was detected (ms).

Type text. Handles Unicode, newlines, and tabs.

Returns the standard {action, screen_changed, reaction_time_ms} feedback. If screen_changed is false, the text field probably didn't have focus — click on it first and retry.

Press a key or key combination.

Friendly names accepted: Tab, Return, Escape, BackSpace, Left, Page_Up, F4, Ctrl, Alt, Shift, Super. Single printable characters stay as-is (a, c, 5).

Examples: Tab, Ctrl+c, Alt+F4, Ctrl+Shift+Tab.

Returns the standard {action, screen_changed, reaction_time_ms} feedback.

Return installed GUI apps.

Scans .desktop entries in /usr/share/applications/. Call this before choosing which app to use for a task, or after installing new software during the session.

Returns a list of dicts, each with:

• name: human-readable application name.

• exec: the executable to pass to app_launch().

Launch a desktop GUI application and return its PID and log file path.

Only applications listed by app_list() are accepted. The process runs in the background; its stdout and stderr are captured in a log file under /tmp/ghostdesk/proc-<pid>.log. Use app_status(pid) to check whether it is still running and to read its output.

Returns a dict with:

• pid: the process ID of the launched application.

• log_file: path to the file capturing stdout and stderr.

• action: description of what was launched.

On failure, returns a dict with a single error key describing what went wrong (not a GUI app, invalid syntax, command not found).

Check whether a launched app is still running and read its logs.

Only PIDs returned by app_launch() in this session are accepted.

Args: pid: Process ID returned by app_launch(). lines: Number of trailing log lines to return (default 50).

Returns a dict with:

• pid: the process ID.

• running: whether the process is still alive.

• log_file: path to the log file.

• tail: the last lines lines of stdout/stderr output.

On failure, returns a dict with a single error key.

Read the current clipboard text.

Write text to the clipboard. Use with key_press("ctrl+v") to paste.