# Fast GUI

Fast GUI is a local STDIO MCP server and Codex plugin for low-latency, structured browser control. It uses Playwright against a dedicated Microsoft Edge profile by default and intentionally exposes no arbitrary JavaScript, shell, cookie, tunneling, or remote browser-control tools.

## Current MVP

- `capabilities`, `list_targets`, `close_target`, `price_radar`, `external_model_plan`, `open_url`, `open_click_verify`, `extract_tables`, `observe`, `act`, `batch`, `wait_for`, `verify`, and `capture`.
- Official PriceAI public snapshot lookup with a validated persistent cache; no PriceAI HTML crawl or internal API dependency.
- One-round-trip allowlisted GET navigation, local wait, optional compact table extraction, and optional structured observation.
- Known-URL calls can auto-select a sole page or create an isolated `new_target`, avoiding an initial target-list round trip while preserving existing pages.
- One-call exact interaction path that can select/create a target, navigate, wait, match one main-document button/link/tab uniquely, click, and prove a required postcondition without returning a full page observation.
- DOM-backed element references bound to a target epoch and scope revision.
- Local condition waits and short deterministic batches.
- State caching and added/removed/updated diffs.
- Domain allowlist, sensitive-field redaction, high-risk action blocking, serialized browser action dispatch, and in-memory screenshot handles with TTL.
- Cache-preserving CDP document guards for owned pages; the broader Playwright route exists only during the exact-click safety window.
- Fast reconciliation: a stable net page change with a wrong postcondition returns bounded, redacted evidence instead of waiting the full deadline. Possible writes stay locked until an explicit observation or fresh navigation.
- Explicit owned-target cleanup through `close_target`; unresolved targets require an intentional discard flag and CDP-attached user pages are never closed.
- Dedicated Edge profile or an explicitly configured CDP endpoint.
- Lazy Playwright loading, single-flight browser initialization, fail-fast profile ownership, and STDIO EOF cleanup for lower cold-start and fewer stranded browser processes.
- Capacity-resilient direct orchestration: the current task calls one fused local tool instead of creating a second model task for clicks.
- Selective supporting delegation: larger requests may use Luna-style read-only evidence and Terra-style independent review agents while the current task remains the single default browser owner.
- Optional request/dispatch timestamps on fused tools produce one `timeline` from request start through MCP and browser completion.
- Cooperative profile handoff: a contender waits briefly, and the current task releases an empty dedicated browser after an idle grace period.
- Adaptive routing keeps an already-bound signed-in session or one hot native click on the native Browser surface.
- Optional OpenAI-compatible external model planner. It is disabled by default, returns one schema-validated proposal, and cannot execute the proposal or bypass browser policy.

Windows UIA, trusted approval UI, Lark API writes, OCR, and vision providers are deliberately not claimed by this browser MVP.

## Build and test

```powershell
npm ci
npm test
npm run benchmark
npm run benchmark:e2e
```

The integration tests and benchmark use a local synthetic page and headless Edge. They do not reuse personal browser data or move the physical mouse.

See `BENCHMARK.md` for the measured local baseline and its limits.

## Fast paths

- Model lane: the visible task calls the known Fast GUI MCP method directly for one fused action. It does not create a Luna/Sol worker solely for a click, so a second model queue cannot add latency or return a capacity error.
- Supporting agents: when a larger request has at least two independent bounded workstreams, Codex may proactively delegate read-only evidence or independent review. Supporting agents do not call Fast GUI; one selected agent owns the entire browser action sequence.
- Browser fallback: if Fast GUI's dedicated browser definitely fails before any click dispatch, the current task may make one exact attempt through the available Browser skill. `PROFILE_IN_USE`, unknown outcomes, or any possible dispatch never switch adapters.
- External planner: `external_model_plan` is an opt-in compatibility interface, not part of the normal direct path. It returns a proposal only; Codex must apply the same authorization and Fast GUI success gates before any separate browser call.
- Accuracy gate: a read needs `ok:true` with no material ambiguity/truncation/warning. An exact click needs `verified:true`, `certainty:"definite_success"`, and `matched_count:1`. Any anomaly leaves the fast lane for Sol analysis; `OUTCOME_UNKNOWN` permits observation/reconciliation, never a repeated possible write.
- Handoff: include the goal, last structured result, target/revisions, reconciliation state, and whether dispatch occurred. Stop with that evidence instead of starting another model task merely to click.
- PriceAI: call `price_radar` with an exact slug, then `open_url` directly to the chosen offer and verify the merchant page. The public snapshot is normally refreshed about every five minutes and must not be treated as final checkout truth.
- Known URL reads: call `open_url` with `wait`, `extract_tables`, and/or `observe` in one request. Use `new_target: true` to preserve existing pages; otherwise omitting `target_id` is allowed only with one open Fast GUI page.
- Large comparison pages: use `open_url.extract_tables` or `extract_tables` with a narrow query. This returns matching rows and links without serializing the full page.
- Known one-click tasks: use `open_click_verify` only when the user has authorized the exact role/name and a postcondition that is initially false is known. It replaces the usual `list_targets` + `open_url` + `observe` + `act(wait)` model round trips, scans the main document for duplicates, and shares `timeout_ms` across queueing and browser phases. It also accepts `new_target: true`. Cold browser startup and blocked-navigation recovery can add wall time.
- Target cleanup: call `close_target` for plugin-owned pages that are no longer needed. Observe an `OUTCOME_UNKNOWN` target first, or explicitly use `discard_reconciliation` when closing is itself the intended reconciliation.
- Visual preview: leave the headed browser on the requested page. Call `capture` only for an explicit screenshot request, canvas content, ambiguous structure, or material visual verification.

## Best-fit scenarios

- PriceAI Top 5 lookup and final merchant-page verification.
- Large comparison tables where only a few rows or links are relevant.
- Repeated deterministic DOM work with exact accessible names.
- A known one-click flow that must prove a newly satisfied URL or text condition.
- Read/act/verify flows where one fused call removes at least two model/tool boundaries.

Prefer native Browser for an already-connected authenticated session or one exact action that fits one hot native call. Prefer native Lark skills for Feishu whiteboards. Fast GUI deliberately stops on vague/duplicate controls, forms and payments, visual-only canvas state, risky writes, popups, downloads, unsupported iframes, and unknown outcomes.

## Configuration

All configuration is via environment variables. Important defaults:

| Variable | Default | Meaning |
|---|---|---|
| `FAST_GUI_BROWSER_CHANNEL` | `msedge` | Playwright browser channel. |
| `FAST_GUI_HEADLESS` | `false` | Use a visible dedicated browser for normal plugin use. |
| `FAST_GUI_PREWARM_PLAYWRIGHT` | `false` (bundled plugin sets `true`) | Background-load Playwright to reduce the first browser action; disabling it saves idle memory. |
| `FAST_GUI_PROFILE_ACQUIRE_WAIT_MS` | `2500` | Briefly wait for another task to release the dedicated profile before returning `PROFILE_IN_USE`. |
| `FAST_GUI_BROWSER_IDLE_RELEASE_MS` | `1500` | Release an empty owned browser/profile after this grace period; `0` disables idle release. |
| `FAST_GUI_PROFILE_DIR` | Local app-data Fast GUI profile | Dedicated persistent browser profile. |
| `FAST_GUI_CDP_ENDPOINT` | empty | Connect to an explicitly enabled CDP endpoint instead of launching a profile. |
| `FAST_GUI_ALLOWED_DOMAINS` | `localhost,127.0.0.1,feishu.cn,larksuite.com,priceai.cc,ldxp.cn,16688.com.cn,catfk.com,erp321.com` | Exact hosts or suffixes allowed for browser observation and action. |
| `FAST_GUI_MAX_NODES` | `200` | Maximum interactive nodes returned by one observation. |
| `FAST_GUI_SCREENSHOT_TTL_MS` | `30000` | In-memory screenshot lifetime. |
| `FAST_GUI_PRICE_RADAR_CACHE_PATH` | Local app-data Fast GUI cache | Validated cache of the public PriceAI snapshot. |
| `FAST_GUI_MODEL_PROVIDER` | `disabled` | Set to `openai-compatible` to enable proposal generation. |
| `FAST_GUI_MODEL_BASE_URL` | empty | API base ending before `chat/completions`; HTTPS required except loopback HTTP. |
| `FAST_GUI_MODEL_NAME` | empty | Provider model identifier. |
| `FAST_GUI_MODEL_API_KEY` | empty | Bearer key; required for non-loopback endpoints and never returned by tools. |
| `FAST_GUI_MODEL_TIMEOUT_MS` | `15000` | External planning timeout, clamped to 1–60 seconds. |
| `FAST_GUI_MODEL_ALLOW_PAGE_DATA` | `false` | Explicit opt-in before a caller may send a bounded page summary. |

`price_radar`, `open_url`, and `open_click_verify` accept an optional `trace` object for explicit latency tests. Supply a `trace_id` plus any known request, parent-dispatch, and worker-receipt Unix epoch milliseconds. The structured result returns a `timeline`; ordinary calls should omit it.

Domain suffix matching is boundary-aware: allowing `feishu.cn` permits `www.feishu.cn`, not `evilfeishu.cn`.

## Safety boundary

- High-risk actions are returned as `CONFIRMATION_REQUIRED` and are not executed because the MVP has no trusted human approval channel yet.
- Recognizable form submitters, activation keys on risky controls, Enter-key submission, downloads, declared non-self window targets, unsafe navigation, and sensitive controls are blocked pending trusted confirmation. In the dedicated profile, Service Workers are disabled and `open_click_verify` aborts non-GET/HEAD HTTP requests during preflight and the immediate click-verification window. It is disabled for CDP attach. The tool is intentionally non-read-only and non-idempotent because WebSocket traffic, delayed scripts, local state, and GET endpoints cannot be proven side-effect-free from DOM labels alone.
- Compact error samples are deduplicated and capped at eight entries with total/truncation metadata; popup counts remain explicit.
- If a dispatched action cannot be proved, the result is `OUTCOME_UNKNOWN` with bounded redacted evidence; the original page and any popup stay write-locked until reconciled. Never repeat the whole flow automatically.
- Dedicated pages continuously enforce the document allowlist through CDP Fetch without disabling HTTP cache. A short context route protects every non-navigation action window and a popup's initial request. Existing pages are isolated rather than closed; CDP-attached pages are never closed. Outside tool actions, a newly created popup's first request cannot be claimed protected until Playwright exposes that page.
- One process owns the dedicated persistent profile at a time through an OS-backed local IPC lease. Competing tasks receive the short `PROFILE_IN_USE` error instead of repeated raw browser-launch logs; Windows releases the lease automatically when its owner exits.
- Host-level delegation never creates multiple profile owners. Supporting agents remain read-only, and ownership cannot switch after a possible dispatch, `OUTCOME_UNKNOWN`, or `PROFILE_IN_USE`.
- A timeout after a possible write is reported as `OUTCOME_UNKNOWN`; callers must observe and reconcile instead of retrying through another adapter.
- Screenshot bytes remain in memory and are not written to disk.
- External model planning is disabled by default. When enabled, only the caller-supplied task and an explicitly opted-in bounded page summary leave the machine. The returned proposal is schema-validated and never executed by `external_model_plan`.
- The one-attempt Browser fallback remains subject to the Browser skill's browser-selection and interaction rules. It is allowed only after Fast GUI proves that no click was dispatched.
- Only PriceAI's public, non-authenticated snapshot data is persisted in the dedicated local cache; credentials, cookies, page HTML, and screenshots are not cached there.
- The server writes protocol messages only to stdout; diagnostic logs go to stderr and never include action values or page content.

See `SECURITY.md` for the threat model and reporting guidance.