astra-linux-agent

Linux computer-use capability layer for GPT-6 Astra running through Codex with ChatGPT subscription authentication — Wayland/Hyprland first.

astra-linux-agent is a local MCP server that gives Astra eyes and hands on a Linux desktop: screenshots and structured desktop state, pointer and keyboard input, semantic window/workspace control, a Chromium browser lane, and structured Linux tools (filesystem, git, processes, systemd, containers, clipboard) — all behind one cross-tool policy engine that judges actions by consequence, whichever route the model takes.

Not an official OpenAI product. This is an independent open-source project. It is not affiliated with, endorsed by, or supported by OpenAI. "ChatGPT", "Codex" and "Astra" are OpenAI's names for their products.

How it connects (no API key)

GPT-6 Astra
    │  your ChatGPT subscription
Codex CLI  — "Sign in with ChatGPT"
    │  MCP over stdio (Codex launches `astra-agent mcp`)
┌───▼─────────────────────────────────────────┐
│  astra-linux-agent                          │
│   policy engine  → capability router        │
│   ┌────────────┬────────────┬─────────────┐ │
│   │ structured │  desktop   │  computer   │ │
│   │ shell/git  │ Hyprland   │ screenshot  │ │
│   │ fs/systemd │ windows    │ click/type  │ │
│   │ containers │ workspaces │ scroll/drag │ │
│   │ browser    │ events     │ visual QA   │ │
│   └────────────┴────────────┴─────────────┘ │
└──────────────────┬──────────────────────────┘
              Wayland / Hyprland

• An eligible ChatGPT plan is required to use Astra through Codex. Codex authenticates with Sign in with ChatGPT.

• No OpenAI API credentials are required or used. This project never calls the OpenAI API and has no place to put an API key.

• It does not proxy, spoof, scrape, or circumvent ChatGPT authentication; it uses only supported Codex mechanisms (ChatGPT login, MCP servers, MCP elicitation, Codex approval modes).

Verified with codex-cli 0.153.4 and gpt-6-astra: screenshots returned as MCP image content are seen by the model (image-only, text+image and image+structuredContent), a screenshot → click → screenshot loop completes, and view_image on a saved PNG works as a fallback. gpt-6-astra refuses codex-cli 0.151.0 ("requires a newer version of Codex").

Related MCP server: kwin-mcp

Why semantic tools before pixels

Clicking through a GUI is slow, brittle, and hard to audit. The router therefore exposes the most semantic and deterministic interface available and only falls back to vision when the GUI is intrinsically required:

Vision (computer_screenshot, computer_click, computer_type, …) remains first-class for GUI-only dialogs, graphical editors, canvases, drag-and-drop, and verifying that a semantic action actually produced the visible result. Every computer_* action returns a fresh screenshot plus DESKTOP_STATE (windows with handles, workspaces, monitors, cursor, lock state), so Astra always has pixels + structured truth.

The policy engine sits above every lane

Policy classifies intent by consequence, not by tool name:

The defining property: if policy denies shell.run("rm -rf ~/important"), it also denies Astra typing rm -rf ~/important + Enter into a terminal window it opened visually. Typed text and key chords aimed at a terminal-class window are inferred into the same shell.run intent and judged by the same rules; the input backend never receives the keystrokes of a denied action.

Honest boundary. Terminal-intent inference is heuristic. It cannot see a command assembled from clipboard pastes, shell-history recall, aliases, GUI "run command" dialogs, or a terminal emulator it does not recognise. Defence in depth comes from the access stages below, Codex's own MCP approval mode, the audit log, and the emergency stop. See docs/policy.md.

Access stages (start small)

Quick start

Requirements: Linux, Wayland, Hyprland ≥ 0.56 (tested 0.56.2), Python ≥ 3.11, grim, wl-clipboard, and Codex CLI ≥ 0.153 signed in with ChatGPT. No root, no /dev/uinput, no daemon.

# 1. Codex, signed in with your ChatGPT account
codex login            # choose "Sign in with ChatGPT"

# 2. astra-linux-agent (from source until the first PyPI release)
uv tool install git+https://github.com/novique-ai/astra-linux-agent   # or: pipx install git+https://github.com/novique-ai/astra-linux-agent
astra-agent doctor                       # checks compositor, grim, Wayland protocols, Codex auth
astra-agent config init                  # writes ~/.config/astra-linux-agent/config.toml (observe-only)

# 3. Register with Codex
astra-agent codex install                # runs `codex mcp add …` and sets the approval mode

# 4. Use it
codex                                    # then: "Take a screenshot and tell me which windows are open."

Non-interactive runs: codex exec --approve-for-me "…" (Codex forces approval_policy = never in exec, so MCP calls need its automatic reviewer).

Widen access deliberately, one step at a time:

# ~/.config/astra-linux-agent/config.toml
stage = "headless"                 # observe → headless → workspace → desktop
[display.headless]
enabled = true                     # isolated virtual output for the agent

See docs/install.md and docs/headless.md.

Safety posture (read this)

This software gives an AI agent eyes and hands on your computer.

• Start in observe; move to headless before desktop.

• Never run it as root; it needs no privileges.

• Keep Codex's MCP approval mode at prompt or writes until you trust your policy; auto skips Codex-side prompts.

• Emergency stop: astra-agent stop (or touch "$XDG_RUNTIME_DIR/astra-linux-agent/STOP"); bind it to a key. Human input wins: if the cursor moves without the agent moving it, the agent pauses itself.

• The browser lane binds CDP to loopback on an ephemeral port with a throwaway profile; never expose it.

• The audit log (~/.local/state/astra-linux-agent/audit.jsonl) records every decision with secrets redacted; screenshots and clipboard contents are never logged.

• Screenshots leave your machine: they go to OpenAI through your Codex session, like any image you attach in Codex.

What is in the box

Runtime dependency: httpx (browser lane only). Everything else is the Python standard library.

Current limitations (0.1.0)

• Hyprland only. The compositor seam (desktop/backend.py) is designed for wlroots/KWin/GNOME/X11 backends, none exist yet.

• Monitor-profile daemons (Omarchy's hyprmoncfg, kanshi) disable a new headless output within seconds; pause them for headless mode (hyprmoncfg unmanage), or use the workspace stage. Measured: 2.4 s.

• Terminal-intent inference is heuristic (see above).

• Confirmations rely on Codex's MCP elicitation support; other MCP hosts without elicitation get fail-closed denials.

• No X11 / XWayland-specific input handling beyond what the compositor provides.

Development

git clone https://github.com/novique-ai/astra-linux-agent && cd astra-linux-agent
uv venv && uv pip install -e '.[dev]'
pytest -q            # unit + integration; live_desktop / live_browser / live_codex are opt-in markers
ruff check . && mypy
python -m testsurface.server   # the synthetic GUI benchmark page

Architecture and decisions: docs/architecture.md, docs/adr/. Contributing: CONTRIBUTING.md. Security reports: SECURITY.md.

License

Apache-2.0 — see LICENSE. Chosen for its explicit patent grant and broad compatibility with both open-source and commercial use.