# gwagent

Let an AI agent use **Gmail, Google Calendar and Google Drive** on an Android phone or emulator,
through adb, the way a person would: by driving the apps' own UI (Gmail, Drive, Calendar app
screens) and, where Android allows it, Android's calendar content provider.

It is meant for a Google Workspace account that can only be used from the phone (no API
access): the agent gets about 150 structured operations, each returning JSON, instead of raw taps.

- **One registry, two front ends**: every operation is a Python function declared with `@op`;
  the CLI (`gwagent ...`) and the MCP server (`gwagent mcp`) are generated from it.
- **Structured results**: rows with indexes, events with ids, files with paths, so an agent can
  chain calls (search -> read -> reply, list -> download, events -> update one occurrence).
- **Safety built in**: destructive or outward actions need `confirm=true`; index-based targets
  are checked against the list they came from; a Gmail send is never repeated blind.
- **Fast path**: a small on-phone server (a `.dex` run with `app_process`, not an installed app)
  makes UI dumps take ~50 ms instead of 4-10 s and lets any Unicode text be typed.

Documents:

| File | For |
|---|---|
| [AGENT_GUIDE.md](AGENT_GUIDE.md) | An AI agent that receives this system: workflows, ids, errors, etiquette |
| [docs/OPERATIONS.md](docs/OPERATIONS.md) | Every operation with its parameters (generated from the registry) |
| [PORTING.md](PORTING.md) | Moving the system to another machine or phone; `scripts/bootstrap.sh` |
| [DEVELOPING.md](DEVELOPING.md) | Rules for adding or changing operations |

## Requirements

- macOS or Linux host with Python >= 3.12 and [uv](https://docs.astral.sh/uv/).
- Android platform-tools (`adb`) on `PATH`, or pointed to with `GWAGENT_ADB`.
- An Android device reachable by adb: the Android emulator with a Google Play system image
  (developed on Android 15 / API 35, x86_64), or a real phone with USB debugging enabled.
  The phone's UI language must be **English** (ops match English labels).
- A Google (Workspace) account signed in on the phone, with Gmail, Calendar and Drive installed.
  A Workspace account usually forces a screen lock: gwagent unlocks it with a PIN (see below).
- Optional: `pdftotext` (poppler) on the host, for `drive.read_text` on PDFs.
- To rebuild the on-phone server only: a JDK and Android build-tools (see `dumpserver/build.sh`).

## Install

```sh
git clone <this repo> gwagent && cd gwagent
uv sync                                  # creates .venv with gwagent and its dependencies
uv run gwagent device.status --pretty    # check that the phone is reachable
```

`uv run gwagent ...` works from the checkout. To have a `gwagent` command everywhere:
`uv tool install --editable .`.

On a new machine or phone, follow [PORTING.md](PORTING.md) and run `scripts/bootstrap.sh`.

## Configuration (environment variables)

All configuration comes from the environment; nothing personal is stored in the repository.

| Variable | Default | Meaning |
|---|---|---|
| `GWAGENT_SERIAL` | `emulator-5554` | adb serial of the phone to drive |
| `GWAGENT_ADB` | `adb` on `PATH`, then `~/android-ufficio-sdk/platform-tools/adb`, `~/Library/Android/sdk/platform-tools/adb` | adb binary |
| `GWAGENT_PIN` | - | Screen-lock PIN (prefer the PIN file) |
| `GWAGENT_PIN_FILE` | `~/.android-ufficio-pin` | File holding the PIN (mode 600); never print or commit it |
| `GWAGENT_OUT_DIR` | `~/gwagent-downloads` | Where downloads, attachments and screenshots go on the host |
| `GWAGENT_STATE_DIR` | `~/.cache/gwagent` | Host state: last Gmail list, last Drive location, last Gmail action (for undo) |
| `GWAGENT_TZ` | `Europe/Rome` | Time zone for calendar output and for times given without an offset |
| `GWAGENT_CALENDAR_ID` | primary calendar | Default calendar id for `calendar.create` |
| `GWAGENT_LOCK_FILE` | `/tmp/gwagent-ui.lock` | UI lock file (see [UI lock](#ui-lock)) |
| `GWAGENT_LOCK_TIMEOUT` | `300` | Seconds to wait for the UI lock before failing with `busy` |
| `GWAGENT_NO_LOCK` | unset | `1` disables the UI lock: **unit tests only**, never with a real phone |
| `GWAGENT_SLEEP_AFTER` | unset | `1` turns the screen off after every UI op |
| `GWAGENT_KEEPALIVE` | `1` | Keep the screen awake during long ops; `0` disables |
| `GWAGENT_AUTO_RECOVER` | `1` | Wait for an offline phone and repeat retry-safe ops once; `0` disables |
| `GWAGENT_BOOT_TIMEOUT` | `300` | Max seconds to wait for the phone to come back |
| `GWAGENT_RECOVER_CMD` | unset | Operator hook (shell command) run once when the phone stays unreachable for 90 s after `adb reconnect offline` and `adb -s <serial> reconnect` did not help, e.g. a command that restarts the emulator (see PORTING.md); gets `GWAGENT_SERIAL` and a fresh `GWAGENT_BOOT_TIMEOUT`. Without it gwagent never restarts the emulator |
| `GWAGENT_RECOVER_CMD_TIMEOUT` | `120` | Max seconds the `GWAGENT_RECOVER_CMD` hook may run |
| `GWAGENT_FAST_DUMP` | `1` | `0` disables the on-phone server (falls back to `uiautomator dump` / `input`) |
| `GWAGENT_FAST_IDLE` | `45` | Seconds after which an idle on-phone server exits |
| `GWAGENT_FAST_START_TIMEOUT` | `30` | Seconds to wait for the on-phone server to start |
| `GWAGENT_FAST_BUSY_TIMEOUT` | `25` | Seconds of silence before the server is considered stuck and restarted |
| `GWAGENT_TRACE` | unset | `1` prints every adb call and server command with its duration on stderr |
| `GWAGENT_BUILD_SDK` | `~/.cache/gwagent-sdk` | Android SDK used by `dumpserver/build.sh` |

Test-only variables: `GWAGENT_LIVE=1` / `GWAGENT_OUTWARD=1` (same as `--live` / `--outward`),
`GWAGENT_REGEN_MANIFEST=1` and `GWAGENT_REGEN_OPS_DOC=1` (regenerate golden files),
`GWAGENT_PRIVACY_TERMS` (extra comma-separated terms for the fixture privacy scan).
`GWAGENT_DUMP_NONCE` is set by gwagent itself when it starts the on-phone server.

## CLI

Output is always one JSON object on stdout; the exit code is 0 on success and an error-specific
code otherwise (see [docs/OPERATIONS.md#error-codes](docs/OPERATIONS.md#error-codes)).

```sh
uv run gwagent list                       # every operation (name, summary, ui, confirm, params)
uv run gwagent list --app gmail --names   # just the names of one app's ops
uv run gwagent describe gmail.search      # one op's full description and JSON schema
uv run gwagent gmail.search --query "is:unread" --limit 5
uv run gwagent gmail search --query "is:unread"            # app and action as two words
uv run gwagent call gmail.read '{"index": 0, "expect_subject": "Weekly report"}'
uv run gwagent gmail.send --to alice@example.com --subject Hi --body "Ciao" --confirm
uv run gwagent calendar.events --period week --pretty
uv run gwagent mcp                        # MCP server on stdio
```

Flags: `--name value`, `--name=value`, bare `--flag` (true), `--no-flag` (false); a repeated flag
becomes a list (`--to a@example.com --to b@example.com`, or `--to a@example.com,b@example.com`, or
a JSON list). `--args-json '{...}'` merges a JSON object; `--pretty` indents the output.

Result envelope:

```json
{"ok": true, "op": "gmail.search", "result": {"rows": [...]}, "elapsed_s": 6.1}
{"ok": false, "op": "gmail.reply", "error": "not_found", "message": "...", "elapsed_s": 3.2}
```

## MCP server

`gwagent mcp` runs a stdio MCP server exposing every op as a tool; the tool name is the op
name with `.` replaced by `_` (`gmail.search` -> `gmail_search`). Tool results are the same
JSON envelope as the CLI.

Claude Code:

```sh
claude mcp add gwagent -- uv --directory /path/to/gwagent run gwagent mcp
```

Any MCP client (Claude Desktop `claude_desktop_config.json`, a project `.mcp.json`, ...):

```json
{
  "mcpServers": {
    "gwagent": {
      "command": "uv",
      "args": ["--directory", "/path/to/gwagent", "run", "gwagent", "mcp"],
      "env": {
        "GWAGENT_SERIAL": "emulator-5554",
        "GWAGENT_PIN_FILE": "/Users/you/.android-ufficio-pin"
      }
    }
  }
}
```

Give the agent [AGENT_GUIDE.md](AGENT_GUIDE.md) (e.g. reference it from the client's project
instructions). Ops can take tens of seconds; set the client's tool timeout to at least 5 minutes
(`MCP_TOOL_TIMEOUT=300000` for Claude Code), since a UI op may also wait for the UI lock.

## How it works

```
src/gwagent/
  adb.py, device.py   adb wrapper; Device: unlock, tap/swipe/type, dump -> Screen, wait_for, recover
  ui.py               Screen/Node/Selector: parsing and querying UI dumps
  fastdump.py         client of the on-phone server (assets/gwagent-dump.dex)
  lock.py             the inter-process UI lock
  registry.py         @op, Context, call() -> {"ok": ...} envelope, offline recovery
  cli.py, mcp_server.py   generated front ends
  ops/                gmail*.py, calendar.py (provider), calendar_ui.py (app), drive*.py, device.py
dumpserver/           Java source of the on-phone server; build.sh -> src/gwagent/assets/gwagent-dump.dex
```

### Fast path: the on-phone server

`Device.dump/tap/swipe/key/type_text` go through a persistent server on the phone that holds
one UiAutomation connection. It is a `.dex` pushed to `/data/local/tmp` and started with
`app_process` (shell user); **it is not an APK and nothing is installed**. A UI dump takes
~20-60 ms instead of 4-10 s, a tap a few ms instead of ~1 s, and `type_text` sets text through
accessibility, so any Unicode works in every field. If the server is unavailable, everything
falls back to `uiautomator dump` and `adb shell input` automatically. It exits after 45 s idle
and is restarted on demand (also after an emulator restart).

Transport and trust:

- The server listens on an **abstract unix socket** (not a TCP port) with a random name
  `gwagent-dump-<16 hex>` chosen at each start; the host reaches it through
  `adb forward tcp:<port> localabstract:<name>`.
- **Server checks the client**: only peers with uid 2000 (shell, which is what adbd forwards
  as) or 0 are served (`getPeerCredentials`); other apps on the phone are refused.
- **Client checks the server**: the abstract namespace has no permissions, so an app could bind
  a name and pose as the server. Each start gets a random **nonce** (passed to the server in its
  environment); the name and nonce are kept in `/data/local/tmp/gwagent-dump.token` (mode 600,
  shell only). The client accepts the server only if `version` answers `"<PROTO> <nonce>"`.
- The protocol is versioned (`fastdump.PROTO` = `DumpServer.PROTO`, currently **5**); a running
  server with another version is replaced. Run one gwagent version per phone.

## Safety model

- **confirm**: ops that send, delete, share or are otherwise outward-facing or irreversible are
  marked `requires_confirm`; without `confirm=true` they fail with `confirmation_required` and do
  nothing. Some ops need it only in some cases (`calendar.create` / `calendar.update` on events
  with guests, since Google e-mails them).
- **Guards against acting on the wrong item**: the last Gmail list (folder/query and each row's
  sender, subject, date) is remembered on the host. An op given a bare `index` re-opens that list
  and checks the row is still the same conversation; moved rows are found again, vanished ones
  fail with `not_found` before anything is touched. `expect_subject` (Gmail read/reply/forward/
  attachments/...) and `match` (bulk actions) add explicit checks. Drive ops resolve names in the
  remembered location, not in whatever the screen shows. `drive.empty_trash` requires
  `only_if_all_match` or `all=true`.
- **Recipients are verified** on the compose screen before Send: any unexpected or missing address
  keeps the mail as a draft and raises an error.
- **Send is never re-sent blind**: when Gmail's reaction to Send is unclear, the mail is looked up
  in Outbox / Sent; if that cannot tell, the op fails with `send_state_unknown` and the caller must
  check the Sent folder before any retry. Ops with `confirm` are never repeated automatically,
  not even after the phone went offline.
- **Calendar out of office** (`calendar.out_of_office`) only declines new invitations by default
  (`decline="only_new"`); meetings already accepted are declined (and their organizers notified)
  only with `decline="new_and_existing"`.
- **UI lock**: only one program drives the phone at a time (see below).
- **Live tests** only create objects whose name or subject contains `AGT-TEST`, send mail only to
  the phone's own account, add no guests to test events, and delete what they create. Tests that
  change account settings skip when the user has that setting in use and restore it afterwards.
- **Privacy**: the account is a real office account. No mail, calendar or Drive content, no
  screenshots or UI dumps of the phone, and no account address may be sent to cloud LLMs or
  committed; test fixtures are scrubbed (`tests/unit/test_fixture_privacy.py` scans them).
  The PIN is never printed, logged or committed. Run the agent that uses gwagent on a model the
  data owner has approved for this content.
- **No APKs**: nothing is installed on the phone. The only on-phone code is the dump server `.dex`
  run with `app_process`. (`device.install_adb_keyboard` exists for other phones only.)

### UI lock

Every `ui=true` op holds an exclusive `flock` on `/tmp/gwagent-ui.lock` (override
`GWAGENT_LOCK_FILE`) for its whole duration; it waits up to `GWAGENT_LOCK_TIMEOUT` seconds
(default 300), then fails with `busy` (exit 7). `ui=false` ops (calendar provider, file
transfers) do not take it. Other programs that touch the phone's UI must take the same lock;
the exact contract and a wrapper for shell commands are in
[docs/OPERATIONS.md#ui-lock-contract](docs/OPERATIONS.md#ui-lock-contract).

## Tests

| Level | Command | Needs |
|---|---|---|
| Unit | `GWAGENT_NO_LOCK=1 uv run pytest tests/unit -q` | nothing (FakeAdb + scrubbed fixtures), ~seconds |
| Live | `uv run pytest tests/live --live -q` | the phone; creates and deletes `AGT-TEST` objects |
| Live + outward | `uv run pytest tests/live --live --outward -q` | the phone; also sends mail to the account itself and changes settings (restored) |
| Quality | `uv run python scripts/quality_ratchet.py` | no new broad `except` in `src/` |

The whole suite (`uv run pytest --live --outward`) takes several hours. Never run two live
sessions at once: there is only one phone. One module at a time:
`uv run pytest tests/live/test_gmail_read_live.py --live --outward -q`.

Golden files checked by unit tests: `tests/fixtures/mcp_manifest.json` (the MCP tool list,
regenerate with `GWAGENT_REGEN_MANIFEST=1 uv run pytest tests/unit/test_mcp_manifest.py -q`)
and `docs/OPERATIONS.md` (regenerate with `uv run python scripts/gen_ops_doc.py`).

## Limitations

- **Speed**: ops drive the real UI; typical ops take ~5-35 s, long lists and uploads minutes.
  Host performance matters most: on macOS the emulator must run as an interactive process
  (see [PORTING.md](PORTING.md)).
- **English UI only**; one phone, one account, one gwagent version per phone.
- **Google Docs/Sheets/Slides** cannot be created or edited (their apps are not installed and
  installing apps is not allowed): `drive.create_doc` fails with `app_missing`. Native Google files
  can only be read; a Google Sheet is read through Drive's export, which is a PDF (table structure
  lost). The Drive app has no "Manage versions".
- `drive.read_text` with `method="viewer"` only sees the pages the viewer rendered (usually the
  first); the default download method needs `pdftotext` for PDFs.
- Drive search filters use search operators (`type:`, `owner:`, `after:`, `before:`,
  `is:starred`); the app's filter chips are not used. `drive.set_role` / `drive.unshare` could only
  be live-tested on their guards (single-account phone).
- Gmail dates are parsed from screen text (English formats, phone time zone): `date_iso` next to
  the raw value. Gmail bulk actions beyond one screen need `all=true` with a query or folder.
- `gmail.undo` recalls a sent mail only while Gmail still shows the Undo snackbar (seconds).
- Gemini in Gmail depends on the account's licence (`gemini_unavailable` otherwise).
- Calendar: working locations, Out of office and Focus time are not in the provider (they are set,
  read and deleted through the app). Avoid rapid successive edits of one event (an edit made while
  the previous one uploads can be overwritten by the server copy; check `pending_sync`). A
  recurring series that never synced has no instances until it syncs.
- Google Tasks are only reachable through the Calendar app UI.

## Attribution

Parts of the on-phone server and a few host helpers are adapted from
[google/artemis](https://github.com/google/artemis), Copyright 2026 Google LLC, licensed under the
Apache License 2.0. Adapted code carries a comment saying so; the list of adapted files is in
[NOTICE](NOTICE) and the license text in [LICENSES/Apache-2.0.txt](LICENSES/Apache-2.0.txt).
Artemis itself is not used or run.

## License

Apache License 2.0: see [LICENSE](LICENSE). Third-party attributions are in [NOTICE](NOTICE).