<p align="center">
  <img src="assets/icon.svg" width="128" alt="planchette icon">
</p>

<h1 align="center">planchette</h1>

<p align="center">
  Point at any window, then let a coding agent drive it — click, type,
  screenshot, and control another app's window.
</p>

<p align="center">
  <a href="https://pypi.org/project/planchette/"><img src="https://img.shields.io/pypi/v/planchette" alt="PyPI"></a>
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue" alt="MIT license"></a>
</p>

---

No API key: the agent (Claude Code, Codex, Cursor, …) is the loop, through a
small CLI of window-control primitives.

## Platforms

- **macOS** — the polished path: per-window capture, live-overlay picker.
- **Windows / Linux (X11)** — portable backend (mss + pynput + pywinctl);
  select windows with `pick --name` / `pick --list` instead of the overlay.
- **Wayland** — unsupported (it blocks synthetic input and capture by design).

Requires Python ≥3.10.

## Install

```bash
# 1. Put the `planchette` CLI on PATH
curl -LsSf https://raw.githubusercontent.com/shahriarshm/planchette/main/install.sh | sh
#    (or, if you already have uv: `uv tool install planchette`)
```

Then wire it to your agent:

```text
# 2a. Claude Code: install as a plugin
#     /plugin marketplace add shahriarshm/planchette
#     /plugin install planchette@planchette-marketplace
# 2b. Any MCP-capable agent (Codex, Cursor, …): register `planchette mcp`
#     — see "MCP mode" below.
# 2c. Any other agent tool: point it at AGENTS.md in this repo —
#     it documents the capture → look → act loop.
```

## MCP mode — any MCP-capable agent

`planchette mcp` serves the same primitives over MCP (stdio). Screenshots
come back **inline in the tool result**, so the agent needs no file access —
this is the recommended door for Codex (no sandbox bypass needed), Cursor,
and anything else that speaks MCP.

| agent | register with |
|---|---|
| Claude Code | `claude mcp add planchette -- planchette mcp` |
| Codex | `codex mcp add planchette -- planchette mcp` |
| Gemini CLI | `gemini mcp add planchette planchette mcp` |
| Cursor | add to `.cursor/mcp.json`: `{"mcpServers": {"planchette": {"command": "planchette", "args": ["mcp"]}}}` |
| opencode | add to `opencode.json`: `{"mcp": {"planchette": {"type": "local", "command": ["planchette", "mcp"]}}}` |

Then ask the agent to pick a window (`pick_window`, or `pick_window
name="Telegram"`) and drive it. macOS permissions (Screen Recording +
Accessibility) must be granted to whatever app hosts the agent.

## Skill mode — agents that read SKILL.md / AGENTS.md

The `skills/control-window/` folder is a standard Agent Skill: the same
folder works in Claude Code, Codex, Cursor, and other SKILL.md readers.

| agent | install |
|---|---|
| Claude Code | `/plugin` → add this repo (also brings the `/planchette` command) |
| Codex | `cp -r skills/control-window ~/.codex/skills/` (or `.codex/skills/` in a project) |
| other SKILL.md readers | copy `skills/control-window/` into the agent's skills directory |
| anything else | point the agent at `AGENTS.md` — it documents the whole loop |

## Required macOS permissions

Grant these to **the app you run your agent from** (Terminal, iTerm, VS Code…),
in **System Settings → Privacy & Security**:

1. **Screen Recording** — to capture the target window (else screenshots are black).
2. **Accessibility** — to move the mouse, click, type, and raise windows.

After granting, fully quit and reopen the terminal app.

## Use

In Claude Code:

```
/planchette open a new tab and search for the weather
```

…or just ask in chat: "control my Telegram window and message Saved Messages
'hi'." Claude runs the picker (hover + click the window you want), then loops:
screenshot → decide → act, until the goal is done.

- **Background control (macOS):** input is delivered straight to the target
  app's process — the window needn't be frontmost, your focus and cursor stay
  put, and you can keep chatting with Claude in the terminal while it drives.
  (Exceptions: modifier combos briefly activate the target and hand focus back;
  scroll hops the cursor there and back.) On Windows/Linux input follows real
  focus — don't fight it while it runs.
- **One display** in v1.

## CLI (what the agent drives)

```
planchette pick                    # live-overlay picker (macOS) → select a window
planchette pick --list             # print candidate windows
planchette pick --name "Telegram"  # select frontmost match by app/title substring
planchette capture [--out PATH]    # screenshot it → PNG, prints "PATH  WIDTHxHEIGHT"
planchette capture --crop X Y W H  # native-res zoom of a region (for small text)
planchette click X Y [--double] [--right]  # X Y are captured-image pixels
planchette drag X1 Y1 X2 Y2        # select text, sliders, drag & drop
planchette move X Y
planchette type "text"
planchette key "cmd+t" [--repeat N]
planchette scroll X Y up|down [N]
planchette raise
planchette status
planchette agent [--agent claude|gemini]  # macOS: pick a window → floating panel drives it
planchette mcp                     # serve window control over MCP (stdio)
planchette hotkey [--combo C]      # macOS: global hotkey (default ctrl+cmd+p) → agent panel
planchette hotkey --install        # …as a login LaunchAgent; --uninstall removes it
```

Window bounds are re-read from the live window before every action, so a
window that moved after `pick` still gets clicked in the right place.

## Global hotkey → agent panel (macOS)

`planchette hotkey --install` registers ctrl+cmd+P system-wide (like the ⌘⇧5
screenshot overlay) and keeps it across logins. Pressing it runs
`planchette agent`: pick a window under the overlay, then a small floating
panel appears — type a goal ("reply ok to the last message") and a headless
agent session (`claude` or `gemini`, whichever is installed — force one with
`planchette agent --agent gemini`) drives that window, streaming its actions
into the panel. Enter again for follow-ups in the same session; Esc closes it.
A dropdown on the input row lists the installed agent CLIs — switch anytime
between turns; switching starts a fresh conversation for the new agent.

The spawned session is scoped to window control only (claude:
`--allowedTools "Bash(planchette:*)" Read`; gemini: a generated
`~/.planchette/.gemini/settings.json` capping its tools the same way). The
agent CLI and `planchette` must be installed; the panel finds them on PATH
plus the usual install dirs (`~/.local/bin`, `/opt/homebrew/bin`,
`/usr/local/bin`). Gemini also needs auth the spawned process can see, e.g.
`GEMINI_API_KEY` in `~/.gemini/.env`.

Note: the LaunchAgent runs outside your terminal, so macOS asks for
Accessibility again — grant it to the listed Python in System Settings →
Privacy & Security; the daemon retries on its own. Pre-picking for a terminal
session still works: pick via the hotkey, then just Esc the panel.