wisp
by grok-insider
README.md
<!-- docs-owner: usage -->
<p align="center">
<img src="./assets/wisp.png" width="112" alt="wisp logo">
</p>
<h1 align="center">wisp</h1>
<p align="center">
Safety-first automation and app QA through CLI and MCP, with an experimental desktop Hub.
</p>
<p align="center">
<a href="https://github.com/grok-insider/wisp/actions/workflows/ci.yml"><img src="https://github.com/grok-insider/wisp/actions/workflows/ci.yml/badge.svg" alt="CI status"></a>
<a href="./LICENSE"><img src="https://img.shields.io/badge/license-Apache--2.0-blue.svg" alt="Apache 2.0 license"></a>
</p>
wisp gives agents bounded, observable access to graphical apps, browsers, terminals, Android devices, virtual machines, and SSH hosts. Use its provider-backed CLI for autonomous tasks or expose deterministic tools to an MCP client that already supplies the model.
> The release-qualified host is NixOS x86_64 with Hyprland and Wayland. Other Linux environments may work when their runtime tools and permissions are equivalent, but are not release-qualified. See [Support](./SUPPORT.md) for the exact platform policy.
## Product status
There is no published general-availability release yet. The signed `v0.0.2` source and its draft artifacts are the current release candidate. Publishing that draft is a deliberate maintainer decision; Windows headless artifacts remain an unsigned preview until the live qualification and signing gates pass.
| Interface | Status | Intended use |
|---|---|---|
| `wisp` CLI and `wisp-mcp` (Linux) | **Supported and release-tested** | Autonomous tasks and model-controlled tools on the qualified Linux configuration |
| Companion CLI/MCP, `wisp-panel`, and `wisp-swarm` | **Supported and release-tested** | SSH operations, observable headless runs, and bounded parallel sandboxes on Linux |
| `wisp` CLI and `wisp-mcp` (Windows 11 x64, real-desktop) | **Unsigned preview** | Headless package with native broker capture, UI Automation, and guarded input |
| `wisp-gui` Electron Hub | **Experimental** | Optional desktop presentation for the same underlying runtimes |
“Experimental GUI” refers only to the Electron Hub and its React presentation layer. Visual automation through the CLI or MCP is part of the supported headless product on Linux and the Windows real-desktop preview.
## Choose an interface
| Start here | When to use it |
|---|---|
| `wisp "task"` | Let wisp plan and run one bounded task with your configured model provider |
| `wisp-mcp` | Give an MCP-capable agent explicit GUI, browser, mobile, VM, terminal, CLI, and inspection tools |
| `wisp-companion` / `wisp-companion-mcp` | Inspect and operate SSH hosts through a text-first, approval-aware path |
| `wisp-panel` | Watch a headless run through a capability-protected loopback panel |
| `wisp-swarm` | Run a bounded pool of isolated nested-sway tasks |
| `wisp-gui` | Evaluate the experimental all-in-one desktop Hub |
Prefer semantic interfaces over pixels: `browser_*` for websites, `tui_*` for terminal applications, and `cli_*` for bounded commands. Use visual tools when the actual interface is graphical.
## Install with Nix
The flake provides the packaged CLI, supporting binaries, an experimental `gui` app, and a NixOS module.
Try the packaged CLI without installing it:
```bash
nix run github:grok-insider/wisp -- --providers
```
Install the command suite into the current Nix profile:
```bash
nix profile install github:grok-insider/wisp
```
Add wisp to a NixOS configuration:
```nix
{
inputs.wisp.url = "github:grok-insider/wisp";
outputs = { self, nixpkgs, wisp, ... }: {
nixosConfigurations.my-host = nixpkgs.lib.nixosSystem {
modules = [
wisp.nixosModules.default
{
programs.wisp.enable = true;
}
];
};
};
}
```
The module installs the Wayland and terminal runtime tools and can configure ydotool/uinput, AT-SPI, Android tools, and libvirt integration. Review its options before enabling real-desktop or guest control. Group membership changes require a new login session.
## Safe quick start
1. Configure one provider from the table below. Wisp reads documented credential variables from the private `~/.config/wisp/env` file as well as the process environment; never place that file in the repository.
2. Confirm the resolved configuration without driving anything:
```bash
wisp --providers
```
3. Start with an isolated nested-sway sandbox and the deterministic grid grounder:
```bash
WISP_TARGET=nested WISP_GROUNDER=grid \
wisp "open the terminal and type hello from wisp"
```
The nested target cannot move the host cursor or type into host applications. Keep `WISP_TARGET=real` for deliberate runs on an idle desktop after reading the [safety model](#safety-model).
## Providers
Select a credential provider with `WISP_PROVIDER`.
| Provider ID | Credential source | Transport |
|---|---|---|
| `xai-apikey` | `XAI_API_KEY` | OpenAI-compatible chat |
| `openrouter` | `OPENROUTER_API_KEY` | OpenAI-compatible chat |
| `anthropic-key` | `ANTHROPIC_API_KEY` | Anthropic Messages |
| `ollama` | Local endpoint | OpenAI-compatible chat |
| `custom` | `WISP_API_KEY` and an explicit `WISP_BASE_URL` | Configurable |
| `xai-oauth` | Existing Grok CLI credential store | Responses |
| `codex-oauth` | Existing Codex CLI credential store | Responses |
| `claude-oauth` | Existing Claude Code credential store | Anthropic Messages |
API-key and local providers are the clean default. OAuth-reuse providers are explicit opt-ins: they reuse another CLI's subscription credentials and client identity, may conflict with provider terms, and remain pinned to their official hosts.
```bash
WISP_PROVIDER=xai-apikey WISP_MODEL=grok-4.3 WISP_TARGET=nested \
wisp "inspect the sandbox"
WISP_PROVIDER=openrouter WISP_MODEL=provider/model WISP_TARGET=nested \
wisp "inspect the sandbox"
WISP_PROVIDER=ollama WISP_MODEL=qwen3-vl:8b WISP_BRAIN_MODE=json \
WISP_TARGET=nested wisp "inspect the sandbox"
```
Provider-native base URLs are selected automatically. `WISP_BASE_URL` can override API-key and custom providers; OAuth transports cannot be redirected. See the [configuration reference](./docs/reference/configuration.md) for models, endpoints, timeouts, evidence, and safety gates.
## Capabilities and prerequisites
| Capability | Primary path | External prerequisite |
|---|---|---|
| Isolated Linux GUI | Nested sway with AT-SPI, VLM, or grid grounding | Qualified Nix package and a model provider for autonomous runs |
| Linux real desktop | Hyprland capture plus AT-SPI/VLM grounding and guarded input | Wayland session, ydotool/uinput permissions, and an idle desktop |
| Browser | Text-first accessibility snapshots and current element references | Chromium-compatible agent-browser runtime and an allowed host |
| CLI and TUI | Bounded processes, settled structured terminal frames, exact text grids, optional PNG evidence and private recordings | `tmux` for interactive terminal applications |
| Android | adb capture/input and uiautomator grounding | Reachable, awake, user-unlocked device |
| Virtual machine | libvirt capture, QMP input, and optional QGA inspection | Running domain; absolute tablet for pointer input; QGA for guest execution |
| SSH and Companion | OpenSSH control channel plus optional interactive tmux terminal | Reachable SSH host and user-authorized credentials |
The experimental Hub maps these runtimes to internal surface kinds: `sandbox-app`, `real-desktop`, `android-device`, `vm-domain`, `tui-pane`, `cli-run`, `ssh-host`, `ssh-terminal`, and `browser-tab`. Those identifiers are architecture vocabulary, not separate product tiers.
## MCP
`wisp-mcp` speaks stdio MCP. The client supplies the model, presents its own per-tool approvals, and pays for its own model usage; direct MCP tool calls do not invoke wisp's autonomous provider.
```json
{
"mcpServers": {
"wisp": {
"command": "wisp-mcp",
"env": {
"WISP_MCP_ALLOW_REAL": "0",
"WISP_MCP_ALLOW_VM": "0",
"WISP_MCP_ALLOW_GUEST_EXEC": "0"
}
}
}
}
```
Real-desktop, VM, and guest-execution capabilities remain unavailable until their corresponding gates are explicitly enabled. MCP resources expose redacted session evidence for inspection and debugging.
### Terminal and TUI QA
`tui_open` runs an interactive command in an isolated named tmux pane. Use `tui_wait_for`, `tui_wait_idle`, or `tui_wait_exit` instead of fixed sleeps; `tui_status` reports lifecycle, exit code, geometry, cursor, idle time, and recording state. `tui_capture` returns backward-compatible text by default and can also return a versioned styled-cell frame or PNG evidence. Resizing is confirmed through `tui_resize`, and `tui_send` accepts legacy text/keys or ordered typed input atoms.
Terminal evidence is explicit because it may contain prompts, output, or input secrets. `tui_save` writes owner-private text, JSON, ANSI, or PNG artifacts. `tui_record_start`, `tui_mark`, and `tui_record_stop` create a bounded private NDJSON timeline; `tui_export_video` renders it through `ffmpeg` when available. The experimental Hub exposes the same local pane as a live xterm with capture and recording controls.
## SSH Companion
Companion provides text-first, auditable remote operations without making the experimental Hub a requirement.
```bash
wisp-companion user@example.com "explain why the service is unhealthy"
wisp-companion --write web01 "restart the service"
wisp-companion-mcp
```
Commands, risk decisions, approvals, redacted output, and rollback hints remain visible. Interactive SSH uses one visible `ssh -tt` tmux pane plus a hidden control channel for context and integration discovery. Structured Docker, Kubernetes, and Coolify reads are offered only after the integration is detected and enabled. See the [Companion architecture](./docs/architecture/companion.md).
## Headless panel and swarm
```bash
wisp-panel --hold --open "inspect the app"
wisp-swarm --open --task "check login" --task "check settings"
```
The panel prints a capability URL; the bare loopback address is intentionally insufficient. Swarm workers use isolated nested targets and retain the same approval and time limits as a single run.
## Safety model
- Real-desktop approval defaults to `confirm-risky`; missing, expired, or disconnected approval channels deny.
- `auto` is not a supported real-desktop operating mode.
- Human keyboard or pointer input pauses real automation before the next action.
- A process-wide lease prevents concurrent processes from controlling the real desktop.
- Actions, steps, runs, approvals, browser operations, and child processes have deadlines and active cancellation.
- Model values are clamped, and untrusted strings never enter a host shell through interpolation.
- The HUD, control socket, panel, and Hub provide independent stop paths where available.
- Browser authentication sharing is off by default, loopback-CDP only, origin-scoped, and temporary.
- Screenshots persist only when evidence logging is explicitly enabled.
Read [Security](./SECURITY.md) and [Privacy](./PRIVACY.md) before enabling real-desktop input, guest execution, or browser authentication sharing.
## Experimental desktop Hub
`wisp-gui` is an optional Electron and React interface over the same visual, browser, terminal, and Companion runtimes. It brings sessions, approvals, evidence, provider settings, and SSH connections into one window, but its end-to-end workflows and presentation layer are still experimental and are not part of the current supported-interface guarantee.
```bash
nix run github:grok-insider/wisp#gui
```
Do not use the Hub as the sole stop or approval path for safety-critical evaluation. Report GUI findings separately from CLI/MCP runtime issues. The architecture and promotion criteria live in [Unified Hub architecture](./docs/architecture/unified-hub.md) and the [roadmap](./PLAN.md).
## Windows 11 unsigned preview
The primary Windows artifact is the **headless** zip (`wisp-<version>-win-x64.zip`): CLI, MCP, companion entry points, and the native Rust broker for focused-monitor capture, UI Automation grounding, and guarded `SendInput`. An experimental Electron Hub package may also be built; it is optional and does not define Windows support.
On a Windows host only **real-desktop** is available. Nested sandbox, local tmux TUI/CLI, browser-tab, Android, and libvirt/Hyper-V host surfaces remain unavailable. Driving a Windows guest window from Linux with the real target exercises the **Linux** backend and does not qualify Windows.
Treat Windows support as a preview, verify artifact checksums, and use only a dedicated disposable machine or VM. Lab procedure: [Windows lab](./docs/reference/windows-lab.md). Maturity and promotion: [Support](./SUPPORT.md).
```bat
REM on Windows, from the extracted headless package:
wisp.cmd --help
wisp-selftest.cmd winreal
```
## Documentation
- [Support and platform status](./SUPPORT.md)
- [Windows 11 headless lab](./docs/reference/windows-lab.md)
- [Configuration reference](./docs/reference/configuration.md)
- [Debugging and safe QA](./DEBUGGING.md)
- [Security policy](./SECURITY.md)
- [Privacy policy](./PRIVACY.md)
- [Contributing and release gates](./CONTRIBUTING.md)
- [Roadmap and decisions](./PLAN.md)
- [Changelog](./CHANGELOG.md)
- [License](./LICENSE)
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessSyncing