# Mobile Agent Guardian

A five-level safety and verification layer for **embodied mobile agents** running in [pi](https://pi.dev).

Mobile Agent Guardian turns raw Appium/MCP phone control into an auditable, verified protocol:

> **observe → classify → authorize → act → verify → record → anchor**

It was designed for the unusual same-device flow where pi runs in Termux and controls that same Android phone through wireless ADB and Appium. It also works when Appium targets another Android or iOS device.

## Architecture

```text
                     ┌──────────────────────────────────────┐
                     │           Planning Agent              │
                     │  (decomposes tasks, delegates work)   │
                     └────────┬────────────────┬────────────┘
                              │ delegation     │ delegation
                              │ token          │ token
                     ┌────────▼────────┐  ┌────▼───────────┐
                     │  Mobile Worker  │  │  Browser Worker │
                     │  (Appium MCP)   │  │  (Playwright)   │
                     └────────┬────────┘  └────┬───────────┘
                              │                │
                     ┌────────▼────────────────▼───────────┐
                     │         Mobile Agent Guardian        │
                     │                                      │
                     │  L1  Vision classifier               │
                     │  L2  Verify-after-act loop            │
                     │  L3  Playwright policy bridge         │
                     │  L4  Cryptographic anchoring          │
                     │  L5  Multi-agent delegation           │
                     │                                      │
                     │  Policy · Audit chain · Redaction     │
                     └──────────┬───────────────┬──────────┘
                                │               │
                     ┌──────────▼──┐  ┌─────────▼──────────┐
                     │ Appium MCP  │  │ Remote Playwright   │
                     │ UiAutomator │  │ (Modal / headless)  │
                     └──────┬──────┘  └─────────┬──────────┘
                            │                   │
                     ┌──────▼──────┐  ┌─────────▼──────────┐
                     │   Device    │  │  Cloud Chromium     │
                     │ (ADB/WiFi)  │  │  (DOM + JS + Net)   │
                     └─────────────┘  └────────────────────┘
```

## Five levels

### Level 1: Vision classifier (`src/vision.ts`)

Sends screenshots to an OpenAI-compatible vision endpoint for structured risk classification. Returns labels: `login`, `credentials`, `consent`, `payment`, `destructive`, `safe`. Falls back to text heuristics from accessibility output when no vision API is configured.

```bash
GUARDIAN_VISION_API_KEY=sk-... GUARDIAN_VISION_BASE_URL=https://api.openai.com/v1 GUARDIAN_VISION_MODEL=gpt-4o-mini
```

### Level 2: Verify-after-act loop (`src/verifier.ts`)

After every non-trivial mutation, captures a post-action fingerprint, diffs against the pre-action state, and classifies the outcome as `expected`, `unexpected`, `unchanged`, `failed`, or `timeout`. Two consecutive `unexpected` outcomes freeze the verifier—blocking all further mutations until the human observes the screen or runs `/mobile-guard unfreeze`.

### Level 3: Playwright bridge (`src/playwright-bridge.ts`)

Exposes `guardian_browse` for web DOM access through a remote Playwright MCP endpoint. All calls pass through Guardian's policy engine and are audited. Configure with:

```bash
GUARDIAN_PLAYWRIGHT_URL=https://your-workspace--pi-frontend-mcp-web.modal.run/mcp
```

### Level 4: Cryptographic anchoring (`src/anchor.ts`)

Periodically posts the latest audit hash + sequence number to a remote append-only endpoint, signed with HMAC-SHA256 using a device-local secret. Enables third-party verification that no events were removed from the chain.

```bash
GUARDIAN_ANCHOR_URL=https://your-anchor-endpoint/api/anchor
GUARDIAN_ANCHOR_INTERVAL=50
```

### Level 5: Multi-agent delegation (`src/delegation.ts`)

Authority tokens scope what sub-agents can do:
- Allowed tool patterns (glob syntax)
- Maximum risk ceiling (critical is never delegatable)
- TTL and usage limits
- Auto-revocation on policy violation (configurable violation threshold)

Tools: `guardian_delegate`, `guardian_revoke`, `guardian_check_delegation`.

## Install

```bash
pi install git:github.com/salus-ryan/mobile-agent-guardian
```

Or test a checkout:

```bash
npm install
pi -e ./src/index.ts
```

## Quick start

```bash
# Minimal: text heuristics only, no external dependencies
pi -e ./src/index.ts

# With vision classification
GUARDIAN_VISION_API_KEY=sk-... pi -e ./src/index.ts

# With remote Playwright
GUARDIAN_PLAYWRIGHT_URL=https://... pi -e ./src/index.ts

# Monitor mode (log without enforcing)
MOBILE_GUARD_MODE=monitor pi -e ./src/index.ts
```

## Behavior

Observation and navigation remain low-friction. When screen content contains authentication, OAuth, credential, permission, financial, or destructive indicators, Guardian changes state:

1. **Classify**: vision model or text heuristics label the screen.
2. **Gate**: sensitive mutation requires a fresh observation (< 30s) and human confirmation.
3. **Act**: the mutation executes.
4. **Verify**: post-action fingerprint is compared against pre-action state.
5. **Freeze** (if needed): two consecutive unexpected outcomes block further mutations.
6. **Record**: every step is hashed into the tamper-evident audit chain.
7. **Anchor**: periodically signed and posted to remote storage.

OAuth URLs trigger a **human handoff boundary**: the agent may open the provider, but subsequent taps or typing cross the guarded boundary.

### Commands and tools

| Name | Type | Purpose |
|------|------|---------|
| `/mobile-guard` | Command | Show state, verifier, audit integrity |
| `/mobile-guard unfreeze` | Command | Clear verifier freeze |
| `mobile_guard_status` | Tool | Model-callable status and audit verification |
| `guardian_browse` | Tool | Policy-gated remote Playwright navigation |
| `guardian_delegate` | Tool | Create scoped delegation token |
| `guardian_revoke` | Tool | Revoke a delegation token |
| `guardian_check_delegation` | Tool | Check token authorization |

## Audit evidence

Logs are written with mode `0600` to:

```text
~/.pi/agent/mobile-guardian/audit-YYYY-MM-DD.jsonl
```

Every record includes the previous record's SHA-256 hash. Editing, deleting, or reordering an interior event breaks verification. Raw page source and sensitive argument values are not stored; the log records hashes and byte counts.

## Development

```bash
npm install
npm test        # 27 tests
npm run typecheck
```

## Status

Reference implementation. Not a substitute for Android device policy, app sandboxing, or careful review of OAuth consent.

See [`THREAT_MODEL.md`](THREAT_MODEL.md) for security boundaries and limitations.

## License

Apache-2.0