Bosun
by The-Glassy
README.md
# Bosun
**The boatswain browser for agents.** A self-hosted browser service that gives AI agents a real browser — and
calls a human on deck when it needs one (CAPTCHA, cookie wall, 2FA).
> Status: **prototype (M0)**. Works end to end in one deployment; APIs will change. See [docs/DESIGN.md](docs/DESIGN.md)
> and [CHANGELOG.md](CHANGELOG.md).
> Security: Chromium runs **with its own sandbox** ([`CHROMIUM_SANDBOX`](#chromium-sandbox)), downloads are refused
> unless a tenant is given [download hosts](#downloading-files), and a tenant can be kept to a [host allowlist](#per-tenant-host-allowlist)
> (`BROWSER_TENANT_ALLOW`). A review of 0.7.0, and what is still risky, is in
> [docs/SECURITY-REVIEW-0.7.0.md](docs/SECURITY-REVIEW-0.7.0.md); a Linux deployment guide is in
> [docs/DEPLOY-SYSTEMD.md](docs/DEPLOY-SYSTEMD.md).
## What it does today
- Headful Chromium (Playwright) under Xvfb — fewer bot flags than headless.
- **Engines** (`ENGINE`, or `engine` on `session_open`): `chromium` (default), `patchright` (Chromium without the
CDP/automation leaks) and `camoufox` (anti-detect Firefox). See [docs/ENGINES.md](docs/ENGINES.md) for what each one
passes and what it costs.
- MCP server over **stdio** (Claude Code, Codex, …) or **WebSocket** (shared service).
- Tools: `session_open` / `session_close` / `sessions`, `navigate`, `read` (accessibility snapshot with element
refs), `click`, `type`, `select`, `press`, `back`, `wait_for`, `screenshot`, `request_human`, `handoff_wait`,
`keepalive_set` / `keepalive_list` / `keepalive_remove`, `secret_create`, `upload_create` / `upload` (unless `FEATURES_UPLOADS=0`),
`download_wait` / `downloads` / `download_get` (tenants with download hosts),
`macro_save` / `macro_list` / `macro_get` / `macro_delete` / `macro_run`,
and (unless `FEATURES_MEETINGS=0`) `meeting_join` /
`meeting_status` / `meeting_chat_read` / `meeting_chat_send` / `meeting_stop_recording` /
`meeting_leave` / `meeting_recording_delete` / `recording_get`.
- **Keepalive** for persistent profiles: scheduled visits keep sliding login sessions alive without a model, and a
logged-out profile raises an MCPL `bosun:needs-human` push event. A visit can also **watch** one value (say, a
"43 neue Nachrichten" counter) and push `bosun:watch-changed` when it goes up. See
[Keeping logins alive](#keeping-logins-alive) and [Watching a value](#watching-a-value).
- **Meetings (PoC):** joins a Google Meet (as a guest or signed in) or a Zoom meeting (signed in, in Zoom's web
client) as a notetaker, announces itself in the chat, records the audio to a file and leaves; a "please don't record" in the chat stops and deletes the recording. The agent fetches the file with
`recording_get` and transcribes it with any tool, or takes part with a live transcript and speech. See [Meetings (PoC)](#meetings-poc).
- **File uploads** into pages: the agent sends the file with one `curl -T` to a one-time URL on the MCP port
(streamed to disk, up to 300 MB), then `upload` puts it into a file input, answers the file chooser a button
opens, or drops it on a drop zone. See [Uploading files](#uploading-files).
- **Typing secrets** without them passing through the agent: the agent's shell pipes the password into a one-time
URL on the MCP port (`sops … | curl -T - …`), and `type { secret_id }` fills it in. It is kept in memory only,
typed once, and goes into password fields unless the call allows others. See [Typing secrets](#typing-secrets).
- **Macros:** a multi-step flow on a site saved once (`macro_save`) and rerun with one call (`macro_run`), with
parameters, file uploads, secrets by id, downloads and asserts; it runs only on the hosts it names, and steps that
pay, buy or delete an account need an explicit confirmation. See [Macros](#macros).
- Sessions are **ephemeral** (wiped on close) or use a named **persistent profile** of the tenant (stays logged in;
no RAM while idle). A session belongs to the connection that opened it: other connections can't see, use or close
it, and it is closed when that connection goes away or after `IDLE_TIMEOUT_SEC` without use.
- **Every call is answered within its deadline** (60 s by default, at most 120 s), with a clear error when it runs
out. Bosun checks the page: if it still answers, the session is usable as it is; if it doesn't, Bosun replaces it
with a new tab in the same session (cookies kept, at `about:blank`). See [Per-call deadline](#per-call-deadline).
- `navigate` and `keepalive_set` accept only `http://` and `https://` URLs (no `file:`, `data:`, `javascript:`,
`about:`, `chrome:`, `view-source:`, `blob:`, `ftp:` …).
- **Bot-wall detection** after navigation/click (Cloudflare challenge pages, CAPTCHA iframes, "access denied" texts).
A bot check that keeps reloading itself (IP reputation) is reported as such: `bot check keeps reloading (likely IP
reputation); try engine chromium + request_human, or another network`.
- **Human handoff:** `request_human` returns a link (15 min) to a **live view** of the tab, made for a phone:
**Trackpad** mode (cursor, drag to move, tap to click, two fingers scroll, hold and drag for sliders) or **Tap**
mode, a masked **paste bar** for passwords and codes, a key strip, a desktop mouse works as usual. The link can be
reopened (one viewer at a time) until **Done** resumes the agent. See [Handoff](#handoff) for what is and isn't
protected.
## Run
```sh
npm ci # add --omit=optional to skip patchright/camoufox
npx playwright install --with-deps chromium
npx patchright install chromium # only for ENGINE=patchright
npx camoufox-js fetch # only for ENGINE=camoufox (Node 22+)
npm test # headless, against a local test server; no internet needed
npm run test:engines # the same suite once per installed engine
# stdio (an MCP client spawns it)
xvfb-run -a node src/server.mjs
# WebSocket service + live view, reachable over a VPN interface (here 100.64.0.10; for a systemd service with
# both listeners on loopback behind a private-network proxy see docs/DEPLOY-SYSTEMD.md)
WS_HOST=100.64.0.10 WS_PORT=7360 BROWSER_TOKEN=<32+ random chars> \
LIVE_HOST=100.64.0.10 LIVE_PORT=7350 LIVE_BASE_URL=http://100.64.0.10:7350 \
xvfb-run -a -s "-screen 0 1366x900x24" node src/server.mjs
```
Clients authenticate with `Authorization: Bearer <token>` (the old `?token=` query parameter still works but puts
the secret in URLs and logs; prefer the header). The WebSocket listener **refuses to start** on a non-loopback
address without a token; on loopback an unauthenticated listener needs `ALLOW_NO_TOKEN=1` (local development only).
The two listeners are separate: `WS_HOST`/`WS_PORT` carry MCP (token-protected), `LIVE_HOST`/`LIVE_PORT` serve the
human's live view (protected by the handoff link). Both default to `127.0.0.1`; bind them to a VPN/mesh address or
put a TLS reverse proxy in front, and set `LIVE_BASE_URL` to the address the human's phone uses.
### Environment
| variable | default | |
|---|---|---|
| `BROWSER_TOKEN` | – | token for tenant `default` (16+ chars) |
| `BROWSER_TOKENS` | – | more tenants: `alice:<token>,bob:<token>` |
| `BROWSER_TOKEN_POLICIES` | – (all `full`) | per-tenant policy: `swarm=ephemeral` (see [Read-only tokens for subagents](#read-only-tokens-for-subagents)) |
| `BROWSER_MACRO_SOURCE` | – | `swarm=me`: the read-only tenant `swarm` lists, gets and runs the macros of tenant `me` (see [Macros](#macros)) |
| `BROWSER_EPHEMERAL_MAX_SESSIONS` | 2 | session cap of each `ephemeral` tenant (at most `MAX_SESSIONS_PER_TENANT`) |
| `BROWSER_TENANT_ALLOW` | – (unrestricted) | per-tenant host allowlist, enforced by a per-tenant forward proxy: `ops=*.example.org,127.0.0.1:4848; swarm=*.example.org` (see [Per-tenant host allowlist](#per-tenant-host-allowlist)) |
| `FEATURES_MEETINGS` | `1` | `0` = the meeting tools are not listed and their handlers not wired |
| `FEATURES_UPLOADS` | `1` | `0` = no upload tools and no PUT endpoint |
| `UPLOAD_MAX_BYTES` | 314572800 (300 MB) | largest file one upload may carry |
| `UPLOAD_TENANT_QUOTA_BYTES` | 1073741824 (1 GB) | uploaded bytes one tenant may hold at a time (at least `UPLOAD_MAX_BYTES`) |
| `UPLOAD_TTL_SEC` / `UPLOAD_URL_TTL_SEC` | 3600 / 900 | an uploaded file is deleted this long after it arrived; an upload URL is valid this long |
| `UPLOAD_BASE_URL` | – (the `Host` the client connected with, `http://`) | where tenants reach the MCP port, for the PUT URL; set it behind TLS or another name |
| `FEATURES_ENGINES` | all three | engines a session may ask for, comma list (e.g. `chromium`); `ENGINE` is always included |
| `CHROMIUM_SANDBOX` | `1` | `0` = run Chromium without its own sandbox (logged at every start); see [Chromium sandbox](#chromium-sandbox) |
| `BROWSER_TENANT_DOWNLOAD_ALLOW` | – (no downloads) | `glassy=web.plaud.ai,*.plaud.ai; ops=*`: which tenants keep downloads, and from which hosts (see [Downloading files](#downloading-files)) |
| `DOWNLOAD_MAX_BYTES` | 314572800 (300 MB) | largest file a download may be; past it the browser cancels it mid-stream |
| `DOWNLOAD_TENANT_QUOTA_BYTES` | 1073741824 (1 GB) | downloaded bytes one tenant may hold at a time, running and kept (at least `DOWNLOAD_MAX_BYTES`) |
| `DOWNLOAD_TTL_SEC` / `DOWNLOAD_URL_TTL_SEC` | 3600 / 900 | a download not fetched is deleted this long after it finished; a GET URL is valid this long |
| `DOWNLOADS` | `0` | before 0.11 the service-wide switch; `0` is still accepted, `1` stops the service with a hint (use `BROWSER_TENANT_DOWNLOAD_ALLOW`, `*` for any host) |
| `WS_HOST` / `WS_PORT` | `127.0.0.1` / – | WebSocket MCP listener (stdio when `WS_PORT` is unset) |
| `WS_ALLOWED_ORIGINS` | – | comma list of browser Origins allowed to connect (non-browser clients send none) |
| `ALLOW_NO_TOKEN` | – | `1` = no auth, loopback only |
| `MAX_CONNECTIONS` | 32 | concurrent WebSocket connections |
| `MAX_INFLIGHT` | 8 | concurrent tool calls per connection |
| `TOOL_TIMEOUT_SEC` | 60 | deadline of one tool call, 1 to 120 s (see [Per-call deadline](#per-call-deadline)) |
| `TOOL_TIMEOUTS` | – | per tool, overriding `TOOL_TIMEOUT_SEC`: `read=90,upload=120` |
| `LIVE_HOST` / `LIVE_PORT` / `LIVE_BASE_URL` | `127.0.0.1` / – / derived | live view listener and the URL put in links |
| `HANDOFF_TTL_SEC` | 900 | handoff link lifetime |
| `BROWSER_DATA` | `./data` | profiles (`profiles/<tenant>/<name>`) and saved screenshots |
| `HEADLESS` | – | `1` = headless browser (more detectable; use for tests) |
| `ENGINE` | `chromium` | default engine: `chromium`, `patchright`, `camoufox` (must be installed, or startup fails) |
| `CAMOUFOX_OS` | host OS | OS of the Camoufox fingerprint: `macos`, `windows`, `linux` |
| `MAX_SESSIONS` | 3 | sessions across all tenants |
| `MAX_SESSIONS_PER_TENANT` | `MAX_SESSIONS` | |
| `IDLE_TIMEOUT_SEC` | 900 | close sessions unused this long (`0` = never; a session in handoff is never idle) |
| `MAX_PAGES_PER_SESSION` | 5 | popups beyond this are closed |
| `KEEPALIVE` | `1` | `0` = don't run scheduled keepalive visits (schedules are kept) |
| `TZ` | `Europe/Vienna` | IANA time zone for keepalive `quiet_hours` |
| `RECORDING_RETENTION_DAYS` | 30 | meeting recordings are deleted this many days after they were written |
| `MEETING_OWNER` | – | name in the meeting announcement ("Hi, I'm Glassy, `<owner>`'s AI notetaker"); unset: "an AI notetaker" |
| `ELEVENLABS_API_KEY` | – | ElevenLabs key: the live meeting transcript (`meeting_join` `live_transcript`) and the `elevenlabs` voice of `meeting_speak`; unset: both refused. Kept in the service only (removed from its environment at start) |
| `ELEVENLABS_BASE_URL` | `https://api.elevenlabs.io` | another ElevenLabs API host (`https://*.elevenlabs.io`, e.g. `https://api.eu.residency.elevenlabs.io`); nothing else gets the key (http on loopback is accepted, for tests) |
| `ELEVENLABS_VOICE` | `EXAVITQu4vr4xnSDxMaL` ("Sarah") | fallback ElevenLabs voice id for `meeting_speak` with `backend: "elevenlabs"` (a tenant's speech setting wins) |
| `ELEVENLABS_TTS_MODEL` | `eleven_v4_turbo` | fallback ElevenLabs text-to-speech model for `meeting_speak` (`[a-z0-9_]`, 3–60 characters), e.g. `eleven_flash_v2_5` |
| `LIVE_TRANSCRIPT_DAILY_MIN` | 180 | minutes of live transcript per tenant and local day (`TZ`); at the cap the stream stops (`bosun:meeting-transcript-stopped`, `daily-cap`) |
| `LIVE_TRANSCRIPT_VAD_SILENCE_S` | 0.5 | fallback for the pause (seconds, 0.2–3) that ends a live transcript segment (`vad_silence_s`) |
| `TTS_URL` | – | the local text-to-speech service for `meeting_speak` (OpenAI-style `POST /v1/audio/speech`, WAV out; Supertonic), e.g. `http://10.78.0.11:5000`; unset: only uploaded audio or `backend: "elevenlabs"` |
| `TTS_VOICE` | – (the service's own) | fallback Supertonic voice: `supertonic-F1`..`F5`, `supertonic-M1`..`M5` |
| `TTS_DEFAULT_BACKEND` | `supertonic` | fallback `backend` of `meeting_speak` when none is passed: `supertonic` or `elevenlabs`; `elevenlabs` without `ELEVENLABS_API_KEY` falls back to `supertonic`, with a warning at start |
| `MAX_DOM_NODES` | 60000 | `read` (a11y, whole page) refuses larger pages; pass `selector` |
| `SAVE_SHOTS` / `SHOTS_PER_SESSION` | – / 20 | `1` = also keep screenshots on disk (0600, deleted when the session closes) |
The speech values above (`TTS_DEFAULT_BACKEND`, `ELEVENLABS_TTS_MODEL`, `ELEVENLABS_VOICE`, `TTS_VOICE`,
`LIVE_TRANSCRIPT_VAD_SILENCE_S`) are fallbacks only: each tenant sets its own with `speech_settings_set`, and a call's
own argument wins over both (see Live transcript and speaking).
All numbers are validated at startup; a bad value stops the service instead of silently disabling a limit.
### Per-call deadline
Every tool call is answered within its deadline: `TOOL_TIMEOUT_SEC` (60 s), or the tool's own value in
`TOOL_TIMEOUTS`, at most 120 s. Before 0.13 a page that stopped answering (a renderer stuck in a script, say) could
hold a `read` for as long as the agent was willing to wait, and every later call on that session waited behind it.
A page call is cut off 8 s before its deadline (a third of a deadline under 24 s). Then Bosun asks the page to
evaluate `1`:
- **It answers within 3 s.** The error says `the page still responds and the session is usable`. If what the call
started is still running (a click waiting for its element), the session's next call waits for it to stop, at most
20 s. After that the tab is replaced, as below, to stop it. Two calls never act on the page at once.
- **It doesn't.** Bosun opens a new tab in the same browser context and closes the stuck one; then the session uses
the new tab, at `about:blank`, so navigate again. The session keeps its id and its cookies. A site that keeps its
login in `sessionStorage` is logged out. The session is closed instead (`page-wedged`) when no new tab opens within
15 s, when the stuck tab doesn't close, when the new tab goes away first, or when the cut-off call still hasn't
stopped 3 s after its tab closed.
Firefox (camoufox) takes about 5 s to open a tab next to a stuck one, so there the answer may come while the tab
is still being replaced; it says so, and the next call waits for it.
A call still waiting behind others when too little of its time is left is answered at once and never runs. A
`session_open` that times out closes the session it was opening as soon as that open finishes, so no session is left
that the agent doesn't know about. `wait_for` (which waits up to 60 s by itself) gets at least 75 s unless
`TOOL_TIMEOUTS` names it. These tools keep their own limits and have no deadline: `download_wait`,
`request_human`, `handoff_wait` and the meeting tools. Keepalive visits and meetings are never cut off by the
deadline of the call that set them up.
### Engines
| engine | process model | live view | notes |
|---|---|---|---|
| `chromium` | one shared browser, a context per session | CDP screencast | Playwright's Chromium |
| `patchright` | a browser per session (persistent context; temp profile for ephemeral sessions) | CDP screencast | patchright's recommended setup: no custom UA, no automation-flag args, window-sized viewport |
| `camoufox` | a browser per session | Playwright screencast + `page.mouse`/`keyboard` | fingerprint (window, screen, fonts, UA) from Camoufox; `humanize` off (hangs clicks with Playwright 1.64) |
Only the engines in `FEATURES_ENGINES` (default: all three; `ENGINE` is always included) can be asked for;
`session_open`'s schema lists exactly those, and the session manager refuses the others. A session asking for an
engine that is enabled but isn't installed gets an error naming the install command; there is no silent
fallback to another engine. A persistent profile remembers its engine family (Chromium or Firefox) and refuses the
other. patchright and camoufox-js are `optionalDependencies`; their browsers are separate downloads.
Chromium spawns many threads: under systemd give the unit a generous `TasksMax` (e.g. 2000).
## Finding elements
`click`, `type`, `select`, `upload` and `screenshot` take the element as one of:
- `ref` from `read` (`"e12"`): that very element. Refs change with every `read`, so a macro can't use them.
- `role` (+ `name`), `selector` (CSS, `xpath=…`, `text=…`, any Playwright selector) or `text` (a substring of the
visible text). Bosun takes the **first visible** match: a hidden copy that comes earlier in the page (the closed
popover of a toolbar, the small-screen copy of a menu) is skipped. Finding it and waiting for it to be clickable
(or editable) share 15 s.
`select` and `upload` fall back to a hidden match when none is visible, since a hidden `<select>` or file input
still takes options or files.
For icon buttons with no name (the `button [ref=e149]` lines in a `read`), add one or more of these:
- `nth`: which visible match, `0` = first (default), `-1` = last. With `nth`, `after` or `before`, the element
that was counted is the one acted on, even if the page adds others meanwhile.
- `after: "<text>"`: only matches after the first visible element showing that text, in page order, the nearest
first. `{ role: "button", after: "Share" }` is the first button after Share; with `nth: 1`, the one after that.
- `before: "<text>"`: only matches before it, the nearest first.
Page order runs through open shadow roots. Neither `after` nor `before` counts an element that contains the anchor
or sits inside it. The anchor is the
**first** visible element with that text: if the text shows up twice (a sidebar entry and a toolbar button, say),
pick an anchor that shows up once. `read` shows what comes after what.
When nothing can be clicked, the error says why, instead of a bare timeout. It names elements by tag, id and
classes only, masked as `read` masks them:
```text
selector xpath=//*[normalize-space(text())='Share']/parent::*//button matched 2 elements, none visible within 15 s
(the first is <button>, inside a display: none <div>)
nothing matches selector #send within 15 s
no visible element shows the text "Download all" (after) within 15 s
role button: 1 visible match after "Share", so nth 3 is not there (4 more hidden)
the element did not become clickable within 15 s: it is covered by <div class="tooltip-mask">, which would get the click
the element did not become clickable within 15 s: it is disabled
```
`read { selector }`, `wait_for { selector | text }`, a macro's `assert` and a keepalive's `logged_in_check` also go
by the first visible match.
## Uploading files
Two calls and a `curl` between them:
```text
upload_create { filename: "report.pdf", size_bytes: 48213 }
-> { file_id: "f3a…", put_url: "http://127.0.0.1:8002/upload/f3a…?t=…",
curl: "curl -sS --fail-with-body -T 'report.pdf' 'http://127.0.0.1:8002/upload/f3a…?t=…'", valid_min: 15 }
$ curl -sS --fail-with-body -T report.pdf 'http://127.0.0.1:8002/upload/f3a…?t=…'
-> {"file_id":"f3a…","filename":"report.pdf","size":48213}
upload { session, selector: "input[type=file]", file_id: "f3a…" } # or ref / role+name / text
-> { method: "input", input_files: ["report.pdf"], url, title }
click { session, role: "button", name: "Send" }
```
- **Where the bytes go.** The PUT goes to the same host and port as MCP (the URL is built from the `Host` the MCP
client connected with, or `UPLOAD_BASE_URL`), so whatever reaches MCP reaches it. The body is streamed straight to
`BROWSER_DATA/uploads/<tenant>/<file_id>/<filename>` (0600): never held in memory, never base64. The browser
reads the file from there itself; no bytes cross the MCP connection.
- **The URL works once,** for `UPLOAD_URL_TTL_SEC` (15 min), and only for the tenant that asked for it. Its secret
(`?t=`) is the credential. Instead of it, `PUT /upload/<file_id>` with the tenant's own token in
`Authorization: Bearer` works too; another tenant's token is refused either way.
`size_bytes` is optional; when given, the PUT must carry exactly that many bytes.
- **Small files** (up to 1 MB): `upload_create { filename, content_base64 }` stores the file at once, no PUT.
- **Limits.** `UPLOAD_MAX_BYTES` per file (300 MB) and `UPLOAD_TENANT_QUOTA_BYTES` per tenant (1 GB) are checked
against `Content-Length` before any byte is read, and again while the body streams: past either, the answer is
`413` and nothing stays on disk. A tenant holds at most 50 uploads at a time (waiting, arriving or ready), and
streams at most 4 at once (16 across the service); past that a PUT gets `429` and its URL stays valid.
- **Files never cross tenants.** A `file_id` of another tenant answers like one that never existed.
- **Deletion.** A file is deleted when the session that used it closes (the last one, if several did),
`UPLOAD_TTL_SEC` (1 h) after it arrived, or when the service starts. Submit the form within the hour.
- **Read-only tokens** (policy `ephemeral`) can't upload: the tools are refused, and so is a PUT with such a token.
How `upload` puts the files in:
1. The element is an `<input type=file>`, hidden or not, or the `<label>` of one: the input gets the files
directly. Several files (`files: [id, id]`) only when it has `multiple`.
2. Anything else (an "Upload" button, a link) is clicked, and the file chooser it opens is answered.
3. If no chooser opens within 5 s, the files are **dropped** on the element: `dragenter`, `dragover`, `drop` with
the files in a `DataTransfer`. `accepted` says whether something on the page took the drop.
`method: input | chooser | drop` forces one of them.
Drag-and-drop is best effort:
- The events are synthetic (`isTrusted` is false). A drop zone that checks for real user events ignores them.
- On `camoufox`, Firefox hides files dropped from outside the page's own scripts, so Bosun dispatches the drop
from a script element in the page. A page whose Content-Security-Policy refuses inline scripts gets no files;
`note` says so. Use `chromium` for that page.
- An element inside a shadow root may not be found by the camoufox drop script; drop on its host, or use another
engine.
Prefer the page's file input or its "Upload" button where there is one: both are as good as a person picking a
file.
## Typing secrets
`type { value: "<password>" }` puts the password into the tool call, so it ends up in the agent's transcript and in
whatever logs the MCP client keeps. With a secret id the agent's shell sends the password to Bosun itself, and the
tool call carries only the id:
```text
secret_create { label: "github password" }
-> { secret_id: "s9c…", put_url: "http://127.0.0.1:8002/secret/s9c…?t=…",
curl: "curl -sS --fail-with-body -T - 'http://127.0.0.1:8002/secret/s9c…?t=…'", valid_min: 15, max_bytes: 4096 }
$ sops -d --extract '["github"]["password"]' secrets.yaml | curl -sS --fail-with-body -T - 'http://127.0.0.1:8002/secret/s9c…?t=…'
-> {"secret_id":"s9c…","label":"github password","bytes":24,"valid_min":10}
type { session, selector: "#password", secret_id: "s9c…", submit: true }
-> { url, title, secret: "typed and deleted" }
```
- **Pipe it, never print it.** The curl line reads the secret from stdin (`-T -`). Feed it from where the secret
lives (`sops --extract`, `pass show`, a file), not from `echo` with the secret on the command line. One trailing
newline is dropped, so `pass show x | curl …` works. Up to 4096 bytes of UTF-8; empty or invalid UTF-8 is `400`,
more is `413`.
- **The PUT goes where uploads go:** the MCP host and port, one request, within 15 min, for the tenant that asked.
`PUT /secret/<id>` with the tenant's own Bearer token works instead of `?t=`; another tenant's token is `403`.
- **Memory only.** Bosun keeps the secret in a buffer of its own: never on disk, never in a log line or an event.
Log lines name the id and the length in bytes. The buffer is zeroed when the secret is typed, 10 min after it
arrived, or when the service stops; a restart loses every secret. A PUT gets 60 s in all (30 s without a byte).
- **No debug logs.** Playwright's `DEBUG=pw:api` log prints what `fill` types, `pw:protocol` / `pw:channel` print
the messages that carry it, and Node's `NODE_DEBUG=stream` prints the buffers a PUT body arrives in. While `DEBUG`
turns any of those Playwright logs on, or `NODE_DEBUG` is set at all, `secret_create`, the PUT and `type` with
`secret_id` are refused, and the service warns at start. Node logs a body it parses before Bosun can refuse it,
so a PUT to a URL issued before a restart under `NODE_DEBUG` still lands in that log: don't set it where secrets
are sent.
- **Typed once.** `type` with `secret_id` takes the secret out of the store and fills the field. A second `type`
with that id gets "no secret …", the same answer as for an id that never existed or belongs to another tenant. If
the fill fails after the secret was taken, it is gone too: call `secret_create` again. Neither the result nor an
error carries the value. The result's URL and title have it replaced by `[masked]` (also URL-encoded, for a form
that submits with GET); later results (`read`, `navigate`, `sessions`, …) return the page's URL and title as they
are, so a page that puts the secret there shows it to the agent from then on.
- **One element.** The type check, the masking mark and the fill all go to the element found first; a page that
swaps the field for another one before the fill gets nothing typed. The fill types through the page's keyboard,
so a page can still move the focus while it runs. If the checked field didn't get the secret, the call fails and Bosun
leaves the page (`about:blank`), so nothing on it can show the secret to a later `read` or `screenshot`; if it
did, every field that holds it is masked.
- **Password fields only, unless you say otherwise.** The field must be an `<input type=password>`. For an OTP, an
e-mail address or a user name pass `allow_non_password: true`; any text `<input>` or `<textarea>` then takes it.
Checkboxes, buttons, file inputs and non-inputs are refused. A refusal before the fill leaves the secret unused.
- **Masked afterwards.** A field Bosun typed a secret into counts as a credential field from then on, whatever its
type or label: `read` shows `[masked]` for it (and for the value wherever else it appears in the page dump),
`screenshot` blacks it out, and it is cleared after a handoff that ends with Give up, like a password field.
- **Read-only tokens** (policy `ephemeral`) get no secrets: `secret_create` is refused, so is `type` with
`secret_id`, and so is a PUT with such a token.
- **stdio:** without `WS_PORT` there is no port to PUT to, and `secret_create` says so.
What this does not change: once the secret is in the page, the page has it. A page can copy it into ordinary text,
into its title or URL in a form Bosun doesn't recognise, or send it anywhere it can reach, and on engine `chromium` a
page can switch off the masking helpers (see [Security notes](#security-notes)). Type secrets only into pages you
would give them to by hand.
## Downloading files
Off for every tenant unless `BROWSER_TENANT_DOWNLOAD_ALLOW` names it. A tenant that is not named gets the browser
of 0.10 and earlier: its contexts refuse downloads and nothing a page fetches is written to disk.
```sh
BROWSER_TENANT_DOWNLOAD_ALLOW='glassy=web.plaud.ai,*.plaud.ai' # host, *.host, host:port, * — like BROWSER_TENANT_ALLOW
```
```text
click { session, selector: "button:has-text('Export')" }
-> { url, title, downloads: [ { download_id: "d4c…", state: "ready", filename: "Meeting.txt", size: 18342,
mime: "text/plain", source_host: "<bucket>.s3.amazonaws.com",
page_host: "web.plaud.ai", allowed_by: "page origin", expires_in_min: 60 } ] }
download_wait { session, timeoutMs? } # a download that takes longer than click waits (5 s), or one started otherwise
downloads { session? } # running, ready, refused and failed ones, with reasons
download_get { download_id }
-> { get_url: "http://127.0.0.1:8002/download/d4c…?t=…", curl: "curl -sS --fail-with-body -o 'Meeting.txt' '…'", valid_min: 15 }
download_get { download_id, inline: true } # UTF-8 text up to 256 KB comes back as content instead
```
- **Which downloads are kept.** One whose URL's host is on the tenant's list, **or** one started by a page whose
top-level origin is on it. So a Plaud export kept from `web.plaud.ai` may come from S3, `api.plaud.ai` or a
`blob:` the page made, without listing those. `blob:` and `data:` URLs have no host of their own and are judged
by the page. A popup with no page of its own (`about:blank`) is judged by its opener. Anything else is cancelled,
and the reason is in `click`, `download_wait`, `downloads` and the event, e.g. `example.com from page example.com:
neither is in the download allowlist of this token`.
- **Where the bytes go.**
- A session that may download gets a browser of its own. That browser writes downloads, streamed, into a
directory of that session on the data volume (`BROWSER_DATA/downloads/<tenant>/.in-…`): never into `/tmp`
(tmpfs, counted against `MemoryMax`), and never through Bosun's memory.
- The file's size is checked whenever the browser writes to it (the directory is watched), and every 250 ms. Past
`DOWNLOAD_MAX_BYTES` or the tenant's `DOWNLOAD_TENANT_QUOTA_BYTES`, the download is cancelled mid-stream and the
browser deletes what it wrote.
- At most 50 downloads per tenant, running or kept.
- A finished file moves to `BROWSER_DATA/downloads/<tenant>/<download_id>/<filename>` (0600). There it outlives
its session.
- **Fetching.**
- `download_get` gives a one-time GET URL on the MCP port, the same port and address as the upload PUT
(`UPLOAD_BASE_URL` applies), valid for `DOWNLOAD_URL_TTL_SEC`.
- Instead of `?t=`, `GET /download/<download_id>` with the tenant's own `Authorization: Bearer` token works too.
- Once the whole file has been sent, it is deleted. An interrupted GET leaves it, so call `download_get` again.
- At most 4 GETs per tenant at once (`429`). A GET whose client reads nothing for 60 s, or that takes over an hour,
is cut off.
- **Gone.** Once fetched, `DOWNLOAD_TTL_SEC` (1 h) after it finished, and all of them at every start and on
shutdown.
- **Tenants.** Another tenant's `download_id` answers like one that never existed, and a GET with another tenant's
token is refused. Read-only tokens (policy `ephemeral`) never download: the service refuses to start if one is
named.
- **Events.** MCPL feature set `bosun.download`, tag `bosun:download` (`bosun:state=ready|refused|failed`), when a
download finishes or is refused.
- **What starts a download.** Only sessions the agent opens with `session_open` (profile or not). Keepalive visits
and meetings never keep a download.
- **`navigate` to a download URL.** The page stays where it was, and the result carries `downloads`.
- **Limits.**
- The type (`mime`) comes from the file name's extension; the browser does not report the response's.
- The limits are enforced by watching and cancelling, not by a filesystem quota: a very fast download can pass one
by what arrives before the cancel lands. Give `BROWSER_DATA` a quota or a volume of its own for a hard bound.
- A tenant that is not named sees why in `click`: `downloads are off for this token`.
- A download a person starts during a handoff is kept for the agent like any other.
## Macros
An agent that keeps repeating the same flow on a site (Plaud: "Add audio" → "Import audio" → pick the file →
load the list again → "Generate" → "Generate now" → export → download) saves it once as a **macro** and reruns it
with one call. No model runs between the steps.
```text
macro_save { name: "plaud-import", site: ["web.plaud.ai", "*.plaud.ai"],
params: { audio: { type: "file_id" }, file_name: { type: "string" } },
steps: [ { tool: "navigate", url: "https://web.plaud.ai/" },
{ tool: "click", text: "Add audio" },
{ tool: "upload", text: "Import audio", file_id: "{{audio}}" }, … ] }
-> { saved: "plaud-import", version: 1, steps: 16, … }
macro_run { name: "plaud-import", session, args: { audio: "f3a…", file_name: "Standup" }, timeout_sec: 900 }
-> { ok: true, steps: [ { step: 1, tool: "navigate", status: "ok", ms: 1302 }, … ],
reads: [ { step: 12, as: "summary", text: "…" } ], downloads: [ { download_id: "d4c…", filename: "Standup.txt", … } ] }
```
The whole Plaud flow is in [examples/plaud-import.json](examples/plaud-import.json) (written from the UI, not run
against Plaud by the tests).
**A macro** is a per-tenant JSON document: `name` (lowercase letters, digits, `.` `_` `-`), `description`, `site`
(the hosts it may run on, patterns as in `BROWSER_TENANT_ALLOW`, `*` not allowed), `params`
(`{ name: { type: string | number | file_id | secret_id, required?, default?, description? } }`) and up to 100
`steps`. `macro_save` with a name that exists saves a new version; the last 20 are kept, and `macro_get` and
`macro_run` take `version`. `created_by` (default: the tenant) and `created_at` are stored with each version.
`macro_list` shows the latest version of each, `macro_delete` removes all of them.
**Steps** use the page tools' own vocabulary, without `session`: each `{ tool, …that tool's arguments, optional?,
note? }`.
| tool | arguments |
|---|---|
| `navigate` | `url`, `wait?` |
| `click` | `role` (+`name`), `selector` or `text`, optionally `nth`, `after`, `before` (see [Finding elements](#finding-elements)) — no `ref`: refs change with every `read` |
| `type` | the element, `value` or `secret_id: "{{param}}"`, `submit?`, `allow_non_password?` |
| `select` | the element, `value` |
| `press` | `key` |
| `wait_for` | `text` or `selector`, `timeoutMs?` |
| `upload` | the element, `file_id: "{{param}}"` or `files`, `method?` |
| `download_wait` | `timeoutMs?` |
| `read` | `mode?`, `selector?`, `maxChars?`, `as?` (a name for the text in `reads`) |
| `screenshot` | the element?, `fullPage?` |
| `assert` | `text`, `selector` and/or `url_contains`, `timeoutMs?` (default 5000): fails the step unless all hold |
- Any string may hold `{{param}}`. A step that names a param the run wasn't given (one that is not required) is
skipped, so an optional param switches its steps on and off.
- `optional: true`: the step's failure is noted (`failed_optional`) and the run goes on.
- Nothing else: no script, no `evaluate`, no code. Each step runs through the same handler as the tool of that
name, so everything the tool checks applies: the host allowlist, bot-check detection, masking, the downloads
allowlist, the upload and secret rules, and the token's policy.
**`macro_run { name, session, args, version?, dry_run?, timeout_sec? }`** runs the steps in order in the session.
- Each step has its tool's deadline (`TOOL_TIMEOUT_SEC` / `TOOL_TIMEOUTS`; `assert` that of `wait_for`), and the
whole run `timeout_sec` (default 300, at most 900), the failure report included: the steps stop 15 s (a third of
a shorter run) before it ends. A step's own wait (`wait_for`, `assert`, `download_wait`) is shortened to end
within its deadline. A run longer than your MCP client's request timeout (60 s in
the SDK) needs that timeout raised, or a `progressToken`: Bosun sends a progress notification after every step,
which a client with `resetTimeoutOnProgress` counts as a sign of life. A cancelled request stops the run before
its next step.
- The answer: `ok`, every step's `status` (`ok`, `failed_optional`, `skipped`, `failed`, `not_run`) and time,
`reads` (the `read` steps' text), `downloads` (fetch them with `download_get`) and the `screenshot` steps' images.
A `download_wait` step also takes a download that the `click` before it already reported.
- On failure (`isError`): `failure { step, tool, error, url, title, page (the start of the a11y snapshot, masked as
in read), screenshot (attached as an image, masked) }`, to fix the macro and save a new version.
- `dry_run: true` checks the args and returns the steps as they would run, without a session or the page.
**Safety.**
- **Site.** A `navigate` to a host outside `site` is refused before the first step runs (also when an arg made the
URL). The page is checked again in the session's queue right before each page operation of a step (another call
on the session may have moved it), and once more right before `click`, `type`, `select` or `upload` acts on its
element (and before `upload` fills a file chooser or an input of its own),
which must itself be in a document on the site (not in a frame from elsewhere). Any navigation of the tab outside
the site while a step runs, a redirect that comes back included, stops the run after that step, even an
`optional` one. `about:blank` counts as no page. The tenant's host allowlist still applies on top, enforced by its
proxy.
- **Secrets** go in only as a `{{param}}` of type `secret_id`, in `type`'s `secret_id`: a stored macro never holds
one, nor a `file_id` (both are used once). A `secret_id` or `file_id` param can't go into any other field. A
`type` with `value` into a password field (also through its `<label>`) is refused, whether the value was stored
or passed as an arg, and a field that turns into one while it is filled is cleared.
- **Risky steps.** A step whose text (any of its strings, after the args are filled in) says pay, buy, purchase,
submit or place order, checkout or delete account (also: kaufen, bezahlen, zahlungspflichtig, kostenpflichtig
bestellen, Konto löschen) is refused unless the macro was saved with `confirm_risky: true`. A `click` is also
checked against what the element it hits says (all its text, shadow roots included, and every `alt`,
`aria-label`, `aria-labelledby`, `title` and value in it, each in full), and what every button, link or label
around it says, so a selector can't point at a "Buy now" button (or the icon in it)
unnoticed (an `upload`, which clicks what is not a file input, too): once the element is ready to click, then that same element is clicked. If it can't
be looked at, it isn't clicked. `macro_list` and `macro_save` show `confirm_risky` and the risky steps.
- **Storage.** `BROWSER_DATA/macros/<tenant>/<name>.json` (0600), all kept versions in one file, written
atomically; it survives restarts and upgrades with the data volume. Per tenant at most 200 macros of 256 KB each.
- **Read-only tokens** (policy `ephemeral`) may `macro_list`, `macro_get` and `macro_run` but not save or delete,
and their steps hold to their policy (no `upload`, `download_wait` or `secret_id`). They have no macros of their
own: `BROWSER_MACRO_SOURCE=swarm=me` lets the read-only tenant `swarm` list, get and run those of tenant `me`, in
its own sessions, under its own allowlist.
## Handoff
<p><img src="docs/liveview/iphone-trackpad.png" width="240" alt="Live view on an iPhone, trackpad mode">
<img src="docs/liveview/iphone-tap.png" width="240" alt="Live view on an iPhone, tap mode with the key strip"></p>
- **One viewer at a time, reopenable until the end.** While the handoff runs, the link opens the live view whenever
nobody else is connected — closing a chat app's in-app browser and tapping the link again works. A second viewer
is refused (its page says the link is open on another device and retries) until the first disconnects or has been
silent for more than 5 s. Opening the page alone (e.g. a link preview) doesn't connect. Done, Give up, expiry,
`session_close` and disconnect of the agent end the handoff; from then on the link is dead (404).
- **Controls** (bottom bar): **Tap | Trackpad** (trackpad is the default on touch screens), **Keys** (Tab, Enter, ⌫,
Esc, arrows, and the phone keyboard), the **paste bar**, **Give up**, **Done**. Trackpad: one finger moves the
cursor (accelerated), tap clicks at the cursor, two fingers scroll, hold then drag holds the button down. The
service moves the page's mouse along an interpolated, slightly jittered path rather than jumping.
- **Paste bar:** paste (long-press → Paste on iOS) or type into the masked field, then **Insert** or Enter: the text
goes into the page's focused field (tap that field first) and the bar is cleared. The eye button shows what's in
it. The text is not logged, not kept by the service after it is typed, and never sent to the agent — but once it
is in the page, the page field rules below apply (a password field is masked; an ordinary field is not).
- A reconnecting viewer gets the last frame at once, and a page that doesn't repaint is re-captured every 2 s.
- While a handoff is active **every** page tool (`read`, `screenshot`, `navigate`, `click`, `type`, `select`,
`press`, `back`, `wait_for`) is refused, and `sessions`/`handoff_wait` report no URL or title. Tool calls that were
already running when `request_human` came in finish first; the human gets the tab after them.
- `handoff_wait` waits 50 s by default (below the MCP SDK's 60 s request timeout); call it again on `still_waiting`.
Longer `timeoutSec` needs a longer client-side request timeout.
- After the handoff the agent can read the page again. What the human typed into **password, PIN, OTP and card
security-code fields** (detected by input type, `autocomplete`, name/id and label, across frames and open shadow
roots, including show-password toggles) is masked as `[masked]` in `read` output and blacked out in screenshots.
After **Give up / timeout / cancel** those fields are emptied before the agent resumes; if that fails, the
session is closed instead. If a frame can't be inspected, `read`/`screenshot` fail rather than return unmasked
output. The handoff ends only after the human's last input has reached the page.
- A password field that is revealed with a show-password toggle stays masked: an event-driven tracker remembers every
input that ever was `type=password` (incl. shadow roots), and Bosun also notes them when a handoff starts and every
500 ms during it. On `chromium` Bosun's helpers run in the page's own JavaScript world, so a page can detect them
— and replace them, which switches masking off for fields that aren't sensitive by markup (probed, see
[docs/SECURITY-REVIEW-0.7.0.md](docs/SECURITY-REVIEW-0.7.0.md) finding H2); on `patchright` and `camoufox` they run
in an isolated world, which stops that and nothing else (details in [docs/ENGINES.md](docs/ENGINES.md)).
- Not protected: text the human types into ordinary fields, secrets the page itself echoes as normal text, custom
widgets that aren't `<input>`/`<textarea>` (e.g. per-digit OTP boxes built from `div`s), closed shadow roots.
Those are visible to the agent after Done. Ask the human to finish the login (submit it) before pressing Done.
## Keeping logins alive
A persistent profile stays logged in only as long as the site's session does. Many sites extend the session each
time it is used ("sliding" sessions) and end it after some idle time. Bosun can visit a profile on a schedule so
that idle time never runs out — **without a model**:
```jsonc
// once, after the human logged the profile in (session_open({profile:"shop"}) + request_human)
keepalive_set({ "profile": "shop", "url": "https://shop.example/account",
"every_hours": 12, "jitter_min": 30, // defaults; every_hours >= 1
"logged_in_check": { "text": "My account" } }) // or selector / url_not_contains
keepalive_list() // schedule, next run, last results: ok | logged_out | bot_check | error
keepalive_remove({ "profile": "shop" })
```
The interval is `every_hours` (default 12) or `every_minutes` (at least 30), each visit shifted by up to
± `jitter_min` (default 30, or a quarter of a shorter interval). `quiet_hours` (`"22-08"`, `"22:30-07:00"`, local
time in `TZ`, default `Europe/Vienna`) makes the interval **four times longer** while it lasts, so a 30-minute
watch visits about every two hours at night. An interval that starts before a quiet window is stretched only for
its part inside it. Quiet hours follow the wall clock, and on DST nights the time is counted as it really passes. Calling `keepalive_set` again replaces the whole configuration of that profile.
Each visit, at the interval ± `jitter_min` (first one an interval after `keepalive_set`):
- opens the profile with the **engine and mode it was last used with** (recorded in the profile; a profile used
headful is never opened headless, so the fingerprint stays the same);
- loads `url`, then for 20–60 s moves the mouse along human-looking paths (the live view's trackpad interpolation),
scrolls 1–3 times and sometimes rests over a link. It **never clicks, types or navigates**;
- checks `logged_in_check` (every given condition must hold: `text` visible, `selector` visible, URL doesn't
contain `url_not_contains`), reads the [watched value](#watching-a-value) if there is one, closes the profile and
records the result (time, status, duration; last 10 kept).
Visits run one at a time, a few minutes apart, and never take the last free session slot from agents. A profile an
agent has open is skipped (it's in use anyway); an agent opening a profile during a visit takes it over at once.
Schedules and results live in `BROWSER_DATA/keepalive.json` (mode 0600) and survive restarts. Nothing from the
page is stored or logged beyond the check result and the watched value.
When a visit finds the profile **logged out** or stuck at a **bot check**, Bosun pushes an MCPL event to the
tenant's agent (feature set `bosun.handoff`, tags `bosun:needs-human`, `bosun:source=keepalive`,
`bosun:reason=logged-out|bot-check`, `bosun:profile=<name>`; origin `{profile, url, reason}`), so the agent can open
the profile and call `request_human`. Events are only sent to connections whose host asked for MCPL at
initialize and granted `pushEvents` (`featureSets/update`); if none is connected, up to 20 per tenant wait for the
next one. There is no retry loop: the next attempt is the next scheduled visit.
**Limits — what this can't do:**
- It only helps sites with **sliding sessions**. A cookie with an absolute expiry (e.g. "remember me for 30 days")
ends on that date however often the profile is visited.
- **Device re-verification, periodic re-login and 2FA prompts** still need a human (that's what the event is for).
- Visits look like a person glancing at the page, but **too-frequent visits can look bot-like**, especially at
the same time every day — hence the 12 h default and the jitter. Don't go below a few hours unless the site's
idle timeout requires it.
- The visit runs from Bosun's IP with Bosun's browser; if the site ties the session to a different network or
device, keeping it alive here doesn't help.
### Watching a value
A visit can read one value from the page after the logged-in check and tell the agent when it changes. For
willhaben's message counter in the header:
```jsonc
keepalive_set({ "profile": "willhaben", "url": "https://www.willhaben.at/iad/myprofile/chat",
"every_minutes": 30, "quiet_hours": "22-08",
"logged_in_check": { "url_not_contains": "/login" },
"watch": { "text_regex": "(\\d+) neue Nachrichten", "kind": "count", "label": "willhaben messages" } })
```
- **What is read:** the `selector`'s `textContent` if given, else the page's visible text. `text_regex` picks the
value (its first capture group, else the whole match). The regex runs in the page with a 5 s limit, so a
pathological pattern can't stall the service.
- `kind: "count"`: the first whole number in the text. Thousands separators inside it are ignored, but the
regex has to capture them: use `([\d.]+)` rather than `(\d+)` if the site writes "1.234". No match counts
as **0**, since many badges disappear when nothing is new.
- `kind: "text"`: the text with whitespace collapsed, at most 200 characters. No match is `null`.
- **When it is reported:** the first successful read is the **baseline** and raises no event. After that, a count
that **went up** or a text that **differs** raises one `bosun:watch-changed` event. A count that went down (you
read the messages) is stored silently, and the next rise is measured from there.
- Nothing is read when the visit isn't logged in.
- A read that fails is recorded as `watch timeout`, `no number in the watched text` or `watch failed` (say, a
selector that isn't valid) and leaves the stored value alone. Only a selector or regex that matches nothing
counts as "no match". An agent taking the profile over mid-read aborts the visit, and nothing is recorded.
- Changing `selector`, `text_regex`, `kind` or the `url` starts a new baseline; a new `label` doesn't. A visit
still running for the old configuration records nothing.
- **What is stored:** only the value (a number, or at most 200 characters), when it was read and when it last
changed, shown under `watch` in `keepalive_list`, plus `baseline` / `unchanged` / `changed` / `lower` per visit
in `last`. Nothing else from the page.
The event uses its own MCPL feature set **`bosun.watch`**: tags `bosun:watch-changed`, `bosun:source=keepalive`,
`bosun:profile=<name>`; origin `{profile, label, old, new, url}`; text e.g. `Bosun watch "willhaben messages" on
profile "willhaben" changed: 43 -> 45 (https://…)`. Delivery and queueing work as for `bosun:needs-human`.
**Host recipe change:** a recipe without `enabledFeatureSets` for Bosun gets both feature sets. One that lists
`"bosun.handoff"` must add `"bosun.watch"`, or use `"bosun.*"`, otherwise watch events are not delivered (they wait
in the queue, up to 20 per tenant, until a connection grants the set):
```jsonc
"mcpServers": { "bosun": { "url": "wss://bosun.example/mcp", "transport": "websocket",
"enabledFeatureSets": ["bosun.handoff", "bosun.watch"] } }
```
Limits: a watch sees only what that one page shows (the visible text, or a selector's text, at visit time).
Between visits nothing is watched, so with `every_minutes: 30` a new message is reported about 15 minutes after it
arrived on average and up to about 40 minutes after (longer in quiet hours). Several changes between two visits make one event, carrying the latest value.
## Meetings (PoC)
> `FEATURES_MEETINGS=0` switches meetings off: the tools are not listed and their handlers are not wired, and nothing
> of the below runs — no recordings directory, no audio server. Read the consent notes before using them.
Bosun can attend a **Google Meet** or a **Zoom** meeting, record its audio and leave. There are two ways to use it.
**Mode 1, notetaker (record only).** `meeting_join { url }` and nothing else. No ElevenLabs key, no voice, and the
microphone is never unmuted. Bosun announces itself in the chat, records, and leaves when the meeting ends. Afterwards
the agent calls **`recording_get { meeting_id }`**, downloads the file with the `curl` line it returns, and transcribes
it with any tool (whisper, Plaud, ElevenLabs batch, …). This is the mode for a passive notetaker agent.
```jsonc
meeting_join({ "url": "https://meet.google.com/abc-defg-hij" }) // -> { "meeting_id": "m1a2b3c4d5e6", "state": "waiting", ... }
// … the call happens; bosun:meeting-ended arrives, or meeting_status says "ended" …
recording_get({ "meeting_id": "m1a2b3c4d5e6" })
// -> { "get_url": "http://<host>:<port>/recording/m1a2b3c4d5e6?t=…", "size": 10874213, "seconds": 905, "valid_min": 15,
// "curl": "curl -sS --fail-with-body -o 'm1a2b3c4d5e6.ogg' 'http://<host>:<port>/recording/m1a2b3c4d5e6?t=…'" }
```
`recording_get` gives the calling tenant's own recording only, as a **one-time GET URL** on the MCP port, valid
`DOWNLOAD_URL_TTL_SEC` (15 min). It is refused while the recording is still being written and once it is deleted. The
file is **not** deleted by the fetch (the retention sweep and `meeting_recording_delete` do that): call `recording_get`
again for a new URL. A read-only (`ephemeral`) token may call it. There is no inline mode; the audio is binary. The
recording's path in `meeting_status` and `bosun:meeting-ended` is a path inside the browser container: an agent host
cannot open it, so fetch it with `recording_get`.
**Mode 2, participant.** `meeting_join { url, live_transcript: true }` and the speaking tools: the agent hears the
call as text while people speak, and can speak, mute and unmute, raise a hand, answer on the meeting channel and speak
speculatively (see [Live transcript and speaking](#live-transcript-and-speaking)). It needs
`ELEVENLABS_API_KEY` for the live transcript (Scribe realtime); the Supertonic voice speaks without the key, only the
live transcript does not work without it. When to speak is always the agent's call: Bosun only provides the
mechanism. The recording is made in this mode too.
| Tool | Mode 1 (notetaker) | Mode 2 (participant) |
|---|---|---|
| `meeting_join` (`url`, `profile`, `announce`, …) | yes | yes, with `live_transcript: true` |
| `meeting_status`, `meeting_chat_read`, `meeting_chat_send` | yes | yes |
| `meeting_stop_recording`, `meeting_leave`, `meeting_recording_delete` | yes | yes |
| **`recording_get`** | **yes: how the agent gets the audio** | yes |
| `meeting_transcript`, the `bosun:meeting-transcript` events | no (needs a live transcript) | yes, needs `ELEVENLABS_API_KEY` |
| `meeting_speak`, `meeting_stop_speaking`, `meeting_mute`, `meeting_raise_hand` | not used (the microphone stays muted) | yes (`meeting_speak` with the Supertonic voice needs no key; the ElevenLabs voice does) |
| `speech_settings_get` / `speech_settings_set`, the meeting channel, speculative speech | not used | yes |
Platforms:
- **Meet:** as an **unauthenticated guest**, or with a Bosun profile **signed in to a Google account**.
- **Zoom:** in Zoom's web client, with a Bosun profile **signed in to a Zoom account**, or as a guest. Zoom may turn an
anonymous browser guest away (`sign-in-required` or `captcha`); it admitted guests in our tests, so try a guest join
first and use a profile when Zoom refuses. Teams is not supported yet. Each platform lives in one file under `src/meetings/platforms/`
(`meet.mjs`, `zoom.mjs`) behind a small interface, and the meeting flow around it is shared.
```jsonc
meeting_join({ "url": "https://meet.google.com/abc-defg-hij" }) // name, announce_text, announce ("chat" | "host" | "none"),
// max_wait_admit_min (10), profile, engine, leave_on_optout (true)
meeting_join({ "url": "https://us02web.zoom.us/j/81234567890?pwd=…", "profile": "zoom" })
// -> { "meeting_id": "m1a2b3c4d5e6", "platform": "zoom", "state": "waiting", "joined_as": "account", ... }
meeting_join({ "url": "https://meet.google.com/abc-defg-hij", "profile": "google", "dry_run": true })
// -> { "dry_run": true, "state": "lobby", "lobby": "account", "join_button": "Ask to join without microphone & camera",
// "microphone": "on (no device found)", "camera": "no device", "challenge": false, "page": "…/meet/abc-defg-hij", ... }
meeting_status({ "meeting_id": "m1a2b3c4d5e6" }) // joining | waiting | admitted | recording | ended, participants, recording_seconds,
// announce: { mode, sent_to: everyone | host | none | failed, reason, failures }
meeting_chat_read({ "meeting_id": "…", "since": 0 }) meeting_chat_send({ "meeting_id": "…", "text": "…" })
meeting_stop_recording({ "meeting_id": "…", "delete": true }) meeting_leave({ "meeting_id": "…" })
meeting_recording_delete({ "meeting_id": "…" })
meeting_join({ "url": "…", "live_transcript": true }) // transcript_partials (true), vad_silence_s
meeting_transcript({ "meeting_id": "…", "since": 0 }) // finals so far
meeting_speak({ "meeting_id": "…", "text": "…", "lang": "en" }) // voice, backend ("supertonic" | "elevenlabs"), model,
// audio_file_id, auto_mute (true), interrupt, wait
speech_settings_get({}) speech_settings_set({ "backend": "elevenlabs", "elevenlabs_voice": "…", "vad_silence_s": 0.5 })
meeting_stop_speaking({ "meeting_id": "…" }) meeting_mute({ "meeting_id": "…", "on": true }) meeting_raise_hand({ "meeting_id": "…", "on": true })
```
**How a meeting runs:**
1. **Browser:** a fresh browser of its own, or `profile` (Chromium or patchright only; camoufox is refused).
- Media permission prompts are accepted (`--use-fake-ui-for-media-stream`), but no fake devices are used: the
microphone is an audio source that is silent unless Bosun speaks (`meeting_speak`), and there is no camera.
- The browser asks for an **English UI** (Meet: `hl=en`; Zoom: its language cookie; both: Accept-Language
`en-US`). The account or region can still win, so Meet's texts and labels are also recognized in German,
Slovak, Ukrainian and Russian, and Zoom's in German and Ukrainian (some Russian).
- **Zoom links** (`zoom.us/j/<id>?pwd=…`, any `*.zoom.us` subdomain, webinar `/w/` and web client `/wc/`
links) are rewritten to the web client, `https://app.zoom.us/wc/<id>/join?pwd=…`, so Zoom's "open the app"
page doesn't come up. If it does anyway, Bosun clicks "Launch Meeting" and then "Join from your browser".
2. **Lobby:**
- Microphone and camera must be **verifiably off** before joining. Bosun switches a device off and checks it,
unless Meet shows the device doesn't exist ("Camera not found" / "Kamera nicht gefunden") and no control shows
it on; then it already counts as off. If neither can be confirmed, Bosun does not join.
- A guest types its name (default `Glassy (notetaker)`). A signed-in Meet profile has no name field and joins
under the account's name, so `name` is ignored. Zoom fills in a signed-in account's name, and Bosun keeps it; an
empty field (Zoom's web client shows one to signed-in accounts too) gets `name`.
- **Zoom:** "Do you want people to see you in the meeting?" and then "Do you want people to hear you in the
meeting?" are answered "Continue without …"; "Use microphone" is never clicked. Without permission, Zoom shows
"Mute" with a crossed microphone, and clicking it asks "hear you" again instead of muting. So on Zoom a device
also counts as off if the browser lists none, its permission is denied, or clicking its control made Zoom ask for
it (answered without it). A control that takes clicks and neither turns off nor asks is a live device: no join.
Bosun asks `zoom.us/profile` whether the profile is still signed in; a signed-out profile joins as a
guest, and `meeting_status` carries a `warning` saying how to sign it in again.
- Tips and dialogs over the lobby ("Do you want people to hear you in the meeting?", "Got it", cookie banners,
"This meeting is being recorded") are closed. Only buttons inside a dialog or banner are clicked.
- Then **Ask to join** / **Join now** / **Teilnahme erbitten** (Meet) or **Join** (Zoom). Meet names the button
after the devices it joins without ("Ask to join without microphone & camera"); its visible text is enough too.
A join button that
is still disabled ("Getting ready...") is waited for, up to 25 s. Bosun then waits up to
`max_wait_admit_min` for admission. `meeting_status` shows `joined_as`: `account` or `guest`.
- **Meet, account already in the call on another device:** Meet offers "Switch here", which would drop the
other device. Bosun picks **"Join here too"** (directly, or under "Other ways to join") and uses "Switch here"
only when Meet offers nothing else.
3. **Not getting in:**
- A denied request or no admission in time ends the meeting with a clear error.
- **Google turning the guest away** ("You can't join this video call" / "Sie können nicht an diesem Videoanruf
teilnehmen" after asking, with no waiting room seen) ends with reason `refused_by_google`.
- That refusal comes with a hint: join with a signed-in profile, or from a residential IP. It is not a captcha,
and `navigate` reports it as a refusal too.
- Zoom ends a join early with `invalid-meeting` (wrong meeting ID or passcode, or a link without `?pwd=` for a
meeting that has one, or "This meeting link is invalid (3,001)"; the link's `pwd` is an encrypted token and is
never typed into a passcode field), `sign-in-required` (the host admits only signed-in users) or `captcha`.
- **Captcha** means a challenge that is shown: a reCAPTCHA / hCaptcha / Turnstile frame that is visible, on screen
and at least 60×60 px, and not reCAPTCHA's invisible badge. Meet and Zoom's web client are "protected by
reCAPTCHA" on every join; that alone is not a captcha, for meetings or for `navigate`. Being removed from Zoom's
waiting room counts as `denied`; Meet's "No one responded to your request" too.
- **`dry_run: true`** loads the lobby exactly as a join would (profile, engine, devices) and reports what Bosun
sees there, without joining: lobby kind, the button it would click, microphone and camera, a shown challenge,
and for Zoom whether the profile is signed in. Use it to check a link or a profile before a meeting.
[docs/MEETING-BOT-REFERENCES.md](docs/MEETING-BOT-REFERENCES.md) compares the flows with other open-source bots.
4. Once admitted (Zoom: computer audio joined first, since without it nothing is heard, then microphone and
camera checked off again), Bosun opens the chat and **posts the announcement first**, then starts recording. Who
is told depends on `announce`:
- **`"chat"` (default): everyone.** On Zoom the chat is set to Everyone (or "Meeting Group Chat", a meeting with
continuous meeting chat), and the announcement has to show up in the chat, not marked as private. If the chat
can't reach everyone (Zoom allows chat with the host only, Meet's chat is turned off, the message doesn't show
up), Bosun tells the host instead (Zoom's "Host and co-hosts"; Meet has no chat to the host alone). If that
fails too, Bosun **records anyway**: `meeting_status` shows `announce.sent_to: "failed"`. Anything short of
everyone sends `bosun:meeting-announce` with `sent_to`, `reason` and every attempt's error.
- **`"host"`: the host only**, on Zoom.
- **`"none"`: nobody.** Recording starts without any message. Use it only when the meeting owner told you so: the
caller is then responsible for everyone knowing they are recorded.
In every mode the chat is watched and an objection stops and deletes the recording. `announce.reason` says why
the announcement didn't reach everyone: `chat-restricted` (the chat can't be addressed to everyone),
`chat-unavailable` (no chat), `sent-privately`, `not-sent` (still in the box), `not-confirmed` (it never showed
up as Bosun's), and for the host `host-unreachable`. Later messages (`meeting_chat_send`, the opt-out
confirmation) go where the announcement went. The default text is in English and German: *"Hi, I'm Glassy, an AI notetaker. I'm recording audio for notes. If you'd
rather not be recorded, just say so here in chat (e.g. 'please don't record') and I'll stop and delete the
recording."* With `MEETING_OWNER` set it says whose notetaker it is.
5. While in the call:
- Every chat message from someone else is pushed to the agent as `bosun:meeting-chat`, with `meeting_id`,
`sender` and `text`. `meeting_chat_read` returns them too.
- Bosun leaves by itself when the meeting ends, when it is removed, or when it is alone. That is **30 s** after
everyone else left (`alone_leave_sec`), or 2 minutes if nobody else has joined yet.
- The agent can leave with `meeting_leave`.
- On leaving, `bosun:meeting-ended` carries `meeting_id`, `reason`, `duration_s`, `bytes` and the `recording`:
path (inside the browser container), size and the hint `call recording_get { meeting_id }`. There is no `file://`
link: an agent host refuses it, and nothing outside the container can open it. `meeting_status` (ended) has the
same `path` and hint.
**Signed-in profile.** Google turns away anonymous automated guests far more often than signed-in users, and Zoom
may turn anonymous browser guests away (it admitted them in our tests, but a profile is the dependable way in).
- **Setup:** `session_open({ "profile": "google" })`, `navigate` to `https://accounts.google.com` (Zoom:
`https://zoom.us/signin`), `request_human` so a person signs in, `session_close`.
- **Joining:** `meeting_join({ "url": "…", "profile": "google" })` or `{ …, "profile": "zoom" }`.
- **Engine:** the profile is opened with the engine it was last used with. It can't be in use by an agent session at
the same time.
- **Keep it signed in.** Both sign-ins expire when nobody uses them. Give each profile a keepalive, for example
`keepalive_set({ "profile": "google", "url": "https://myaccount.google.com/", "every_hours": 24 })` and
`keepalive_set({ "profile": "zoom", "url": "https://zoom.us/profile", "every_hours": 24 })`, and look at
`keepalive_list` before an important meeting. A `logged_out` result means a person has to sign in again.
- **Keepalive and meetings share the profile.** A meeting that starts while a keepalive visit has the profile open
takes it over, and a visit that comes due during a meeting is skipped and runs at its next slot. With
`MAX_SESSIONS=2`, a meeting and one agent session fill the service, so a keepalive visit then waits too.
The meeting runs in a session of its own, so it outlives the agent's connection. It counts against `MAX_SESSIONS`.
Events use the MCPL feature set **`bosun.meeting`** (tags `bosun:meeting-chat`, `bosun:meeting-optout`,
`bosun:meeting-announce`, `bosun:meeting-ended`, and with 0.15 `bosun:meeting-transcript`,
`bosun:meeting-transcript-stopped`, `bosun:meeting-speech`); host recipes that list `enabledFeatureSets` need to add it (or `bosun.*`).
**Consent.** Recording starts only after the announcement was tried (or, with `announce: "none"`, without one). No slash commands are needed: a plain-language request
not to be recorded is enough, in English, German, Ukrainian, Russian or Slovak (`please don't record`, `stop
recording`, `bitte nicht aufnehmen`, `не записуй`, `не записывай`, `nenahrávaj`, …).
- **What happens:** Bosun **stops the recording and deletes the file at once**, posts "Recording stopped and
deleted. / Aufnahme gestoppt und gelöscht." in the chat, sends `bosun:meeting-optout` and leaves. With
`leave_on_optout: false` it stays but never records again in that meeting.
- **The matcher is conservative.** "don't stop recording", "keep recording" or "record the next steps" do not
trigger it, and neither do instructions about opting out ("if you'd rather not…", "to opt out, write 'stop
recording'"): quoted words in a longer message are a mention. Bosun's own messages never count. A message that
contains something Bosun posted is its own, however the chat panel labels it.
- **Because it can miss phrasings,** every chat message also reaches the agent, which can call
`meeting_stop_recording` itself.
**Recordings:** `BROWSER_DATA/recordings/<tenant>/<meeting_id>.ogg`, mode 0600, Ogg/Opus 16 kHz mono 24 kbps (about
11 MB per hour). They are deleted after `RECORDING_RETENTION_DAYS` (default 30), or with `meeting_recording_delete`.
The agent gets one with `recording_get` (see *Mode 1*), after the meeting ended or the recording was stopped without `delete`.
A deletion leaves an empty marker `<meeting_id>.deleted` beside it, for the same retention period.
Chat text is kept only in memory, for the last 50 meetings.
### Live transcript and speaking
**Live transcript** (`meeting_join { live_transcript: true }`, off by default; needs `ELEVENLABS_API_KEY`):
- **What it is:** the meeting audio Bosun records is also read by a second ffmpeg as raw PCM (16 kHz mono) and
streamed to **ElevenLabs Scribe v2 Realtime**: `wss://api.elevenlabs.io/v1/speech-to-text/realtime`, model
`scribe_v2_realtime`, `audio_format=pcm_16000`, `commit_strategy=vad` (a pause of `vad_silence_s` ends a segment:
`meeting_join`'s, else the tenant's speech setting, else `LIVE_TRANSCRIPT_VAD_SILENCE_S`, else 0.5 s), word
timestamps and language detection on, no language set (Scribe detects it per segment; a uk/ru/en/de/sk mix is fine).
- **The pause that ends a segment** (`vad_silence_s`, 0.2–3 s; `meeting_status` → `live_transcript.vad_silence_s`):
measured with real Scribe on English and Ukrainian speech (2026-10-28), 0.5 s brings a final 0.85–0.97 s after the
speaker stops, 1.0 s 1.44–1.49 s, 0.7 s 1.06–1.11 s. Fluent sentences stay whole at 0.5 s; a pause of about 0.5 s
or more inside a sentence (someone thinking) ends a segment there, so a final may be half a sentence (no words are
lost; the next final carries the rest). Before answering on a final, check that it reads as finished, or use 0.7 s.
- **Events:** `bosun:meeting-transcript` with `meeting_id`, `seq`, `start_s`, `end_s` (seconds since the transcript
started), `text`, `final`, `lang`, and `speaker` when the page showed who was speaking (see *Speaker labels*). While someone speaks, partials (`final: false`, at most two a second, each carrying the `seq` its final
will have); then the final, always. `transcript_partials: false` sends finals only. These events go to the
connections there at the time and are **never queued** (they would crowd out chat and opt-out events);
`meeting_transcript { since }` returns every final so far. With the meeting channel (below) taking the text, these
events are not sent too (speech setting `transcript_events`: `auto` by default, `always`, `never`).
- **Speaker labels** (best effort; a guest join is enough). Every 300 ms Bosun reads who the page shows speaking.
- Zoom: the speaker view's active tile, or a gallery tile marked active, with the name in its footer.
- Meet: a tile with a non-zero `data-audio-level` or Meet's speaking-animation class, with the name in
`span.notranslate`.
- Bosun's own tile never counts.
- A segment's `speaker` is whoever was shown speaking longest while it was said, from 0.3 s before it. `speakers`
lists them all when there were several.
- Limits:
- Zoom's gallery view without an active mark, and a screen shared full size, show nobody.
- Meet renames its classes from time to time, so the label then goes missing (never wrong names: the name is read
from the tile that carries the mark).
- Two people talking over each other get the one shown longer.
- The page's highlight lags the voice by a few hundred ms.
- `meeting_status.speaker_labels` says whether a speaker has been seen at all (`seen` / `none-seen`).
- **Consent:** it starts and stops **with the recording**: only after the announcement, which then says *"This call
is recorded and transcribed live for notes …"* (German too). An opt-out in the chat stops both **at once**, deletes
the recording and forgets the transcript's text, and posts *"Recording and live transcript stopped and deleted."*
`meeting_stop_recording` stops it too (`delete: true` forgets its text), and so does `meeting_recording_delete`.
- **Retention at ElevenLabs:** Bosun always asks for zero retention (`enable_logging=false`). ElevenLabs applies it
only on enterprise and trial plans; otherwise it answers with a warning and logs the session. `meeting_status`
shows `live_transcript.zero_retention` (`false` on a Pro plan) and the warning.
- **Cost and cap:** $0.39 per hour of audio (about $0.0065 a minute). `LIVE_TRANSCRIPT_DAILY_MIN` (180) per tenant and
day: a join is refused once it is used up, and a running stream stops with `bosun:meeting-transcript-stopped`
(`reason: daily-cap`) while the meeting and the recording go on. The count survives restarts
(`BROWSER_DATA/live-transcript-usage.json`).
- **`NODE_DEBUG`:** while it is set, the key is not sent (Node's diagnostics print request headers): `live_transcript`
and the `elevenlabs` voice are refused, as secrets are.
- **Drops:** a dropped connection is opened again (with growing pauses, 8 tries in a row at most); up to 20 s of
audio meanwhile is kept and sent once it's back, with the last words as context. A wrong key, a used-up quota or
unaccepted terms stop it (`reason: error`).
- `meeting_status` → `live_transcript { state, zero_retention, segments, streamed_min, today_min, daily_cap_min,
final_lag_s }`.
**Speaking** (`meeting_speak`, once admitted and after the announcement):
- **What is played:** `text`, synthesized by `backend` **`supertonic`** (the local TTS service at `TTS_URL`,
voices `supertonic-F1`..`F5` / `M1`..`M5`, pass `lang`) or **`elevenlabs`** (`voice` = an ElevenLabs voice id,
`model` = an ElevenLabs model id, default `eleven_v4_turbo`; zero retention asked for). Or an uploaded file
(`audio_file_id` from `upload_create`, any format ffmpeg reads).
- **Speech settings** (`speech_settings_get`, `speech_settings_set`; per tenant, kept in
`BROWSER_DATA/speech-settings/<tenant>.json`): `backend`, `elevenlabs_model`, `elevenlabs_voice`,
`supertonic_voice`, `vad_silence_s`, checked like the env values; `null` removes one. What a call leaves out comes
from the tenant's setting, then the env fallback, then Bosun's default; `speech_settings_get` shows the effective
values and where each comes from (`tenant`, `env`, `default`). `backend` `elevenlabs` without `ELEVENLABS_API_KEY`
falls back to `supertonic` (with a `warning`). A read-only (ephemeral) token may read them, not set them.
- **How:** ffmpeg plays it into the meeting's `bosun_mic` sink, whose monitor is the browser's microphone. The join
still starts muted; `meeting_speak` unmutes for the utterance and mutes again 0.4 s after it ends (`auto_mute`,
default true; a microphone that was already on stays on). It returns once the audio plays, with `duration_s`,
`prepare_ms` (synthesis, with the unmute alongside), `start_ms` and, for text in several pieces, `gap_ms` (the
longest pause between pieces because synthesis fell behind; 0 when each piece was there before the one before it had
played); `wait: true` returns once it has played.
`bosun:meeting-speech` says when an utterance ended (`done`, `interrupted`, `failed`).
- **Muting again:** each click on the microphone control first moves the mouse over the toolbar (Zoom and Meet fade it
out while the mouse is still; a faded "Mute" still counts as shown, and a click on it lands behind it). Three tries,
an open menu closed in between; the utterance's end waits at most 20 s for them. A microphone still on then is
reported (`mute_error` in the result, the event and `meeting_status`, plus `mic_open: true` in `meeting_status`) and
tried again every 5 s (6 times) while nobody speaks; the next utterance or `meeting_mute` tries too.
- **Muting again on real Zoom (0.17.0).** On 2026-10-28 production said "no microphone control found in Zoom" after
a 2.5 s utterance, and the microphone was left on. When the toolbar's button can't be worked, Bosun now:
1. reads the control from the page itself, including a hidden one or a `<div>`, but only in the toolbar, and clicks
it there;
2. then sends Alt+A (first in the page, in the same step that reads the state; then for real, right after reading
it again), but only while the page shows the microphone still on.
Every step is checked by reading the state back. If nothing works: `mic_open: true` and a `mute_error`, a
`bosun:meeting-speech` event starting "WARNING: Bosun's microphone may still be ON", and a queued
`bosun:meeting-mic-open` event (again with `gave_up` when the retries are over). `meeting_mute { on: true }` or
`meeting_leave` if it must be off. The microphone is `bosun_mic_src`, which carries only what Bosun plays, so a
microphone left on sends silence, never the room.
- **Barge-in:** `meeting_stop_speaking` stops it at once (and mutes again). A second `meeting_speak` while one plays
is refused unless `interrupt: true`, which replaces it.
- **`meeting_mute { on }`** and **`meeting_raise_hand { on }`** on Meet (the toolbar's microphone and hand buttons,
whose names say their state) and Zoom (Mute/Unmute; the hand in the Reactions menu). On
Zoom a profile that never gave Zoom the microphone can't unmute: `meeting_speak` then says so.
- **Latency:** from the call to the first sound in the call about 0.35 s plus the synthesis of the first sentence
(Supertonic: about 1–1.3 s for a short one on a busy CPU; ElevenLabs `eleven_v4_turbo`: 0.3–0.6 s, and the next
pieces arrive long before the one before has played, so `gap_ms` is 0). Start with a short sentence for a quick
answer.
- **Engines:** chromium and patchright (meetings refuse camoufox).
**The meeting as an MCPL channel (0.17.0).** With a host that grants MCPL channels (connectome-host grants what a
server advertises unless the recipe disables it), each meeting Bosun is in becomes a channel `bosun:meeting:<id>`
once the announcement is out.
- **Lifecycle:** registered with `channels/register` (bidirectional, `initiallyOpen`) on every connection of the
tenant, and removed with `channels/changed` when the meeting ends. The speech setting `meeting_channel: false` turns
it off.
- **Incoming:** transcript finals arrive as `channels/incoming`:
- `messageId` `<meeting>:<seq>`, `threadId` the meeting.
- `author` is the speaker when known, else "Meeting".
- `metadata` `{meeting_id, platform, seq, start_s, end_s, final, lang, speaker, speakers}`, and on finals
`turn_seq` and `addressed` (whether the text names the agent, see [Group calls](#group-calls)), tags
`bosun:meeting-transcript` and `bosun:meeting=<id>`.
- With `agent_partials: true`, partial text comes too, tagged `bosun:partial`, at most every
`agent_partials_interval_s` (1.5 s), so the agent can prepare.
- **Speaking:** text the host publishes to the channel (`channels/publish`) is acknowledged at once
(`{delivered: true, messageId}`) and spoken right away, sentence by sentence (the first one cut at a comma when it
is long).
- Text published while it plays is added to the same utterance, and the microphone stays on.
- Text published while `meeting_speak` plays follows it.
- `bosun:meeting-speech` reports the end, with `kind: "stream"`.
- In connectome-host an agent publishes with the `channel_publish` tool. Short pieces (a first sentence, then the
rest) make the first audio come soonest.
- **Limits:** a publish of more than 2000 characters, or with more than 6000 characters (or 120 s of audio) from the
channel still waiting, is refused before it is acknowledged.
- **Stopping:** `meeting_stop_speaking` also drops channel text still waiting. An opt-out with
`leave_on_optout: false` closes the channel, so no transcript and no automatic speech; `meeting_speak` still works.
- **Chunks:** `channels/outgoing/chunk` / `complete` are accepted but never spoken (MCPL §14.5: only a publish
delivers). With `channel_prepare_chunks: true`, each sentence is synthesized as soon as the stream completes it,
and the publish that delivers it plays that audio without waiting for synthesis. This helps only with a host that
streams before it publishes: connectome-host sends chunks after the publish, and Bosun doesn't synthesize those
again.
- **Security:** only channels registered on that connection, for that tenant, under the grant as it is now. Anything
else gets `-32023` (unknown channel) or `-32002` (capability denied).
- **Acknowledgement** (`ack_on_final`, off by default). When a final ends someone's turn (at least `ack_min_words`
words, ending a sentence), Bosun plays a short clip after `ack_delay_ms` (100):
- The clip is one of `ack_phrases`, synthesized once per voice when Bosun is let in, in `ack_lang`.
- Nothing is played while something is being said, if the agent starts speaking first, or within
`ack_cooldown_s` (8) of the last one.
- The agent's reply waits for a clip that is playing (it lasts about half a second) instead of cutting it, and the
microphone stays on in between.
- Measured locally (stand-ins): the clip's audio starts about 0.4 s after the final reached the host, most of it the
unmute click.
**Turns, barge-in and stale answers (0.17.1).** At a 0.5 s pause, Scribe's VAD ends a segment in the middle of
a sentence. In 0.17.0 the agent answered that fragment, and what the speaker added came in while it spoke.
- **Turn hold:** a final is held `turn_hold_ms` (default 900) for more speech. When the speaker goes on, their
partials keep the turn open until their next final, and the agent gets one merged final: `metadata.merged_seqs`
lists the segments, `seq` is the last one. `held_ms` says how long the turn waited after its last final.
- A question, or a complete sentence (three or more words, a full stop, not trailing off with "…" or a word such as
"and", "the", "що", "und"), is held only `turn_hold_complete_ms` (default 300).
- Someone else's final ends the turn held before it. A turn goes at the latest at the first final 30 s after it
began.
- `turn_hold_ms: 0` turns the hold off, as in 0.17.0.
- The acknowledgement (`ack_on_final`) plays once, at the real end. `meeting_transcript` still has every segment.
- **Barge-in** (`barge_in`, default true): someone else speaking while Bosun's audio plays stops it, as
`meeting_stop_speaking` would.
- What counts: at least `barge_in_min_words` (2) new words, where agreeing words ("mhm", "yeah", "ok", "так", "ja")
don't count; or one word while the page shows someone speaking for `barge_in_min_s` (0.6) since Bosun began.
- What happens: the microphone is muted again, and text from the channel still waiting is dropped.
- An uploaded audio file (`audio_file_id`) is never cut off this way, because there is no text to recognise its
echo by.
- What the agent hears: `bosun:meeting-speech` comes with `state: "interrupted"`, `barge_in: true`, `said` (the
text played so far) and `interrupted_by`. The channel gets a message tagged `bosun:meeting-interrupted`, author
`bosun`, `(interrupted at "...")`, with `said` and `unsaid` in its metadata. The interrupter's turn follows as
usual.
- **Bosun's own voice never interrupts it.**
- Text that is mostly words Bosun is saying, or said in the last 8 s, is its own voice coming back through
someone's speakers. It never barges in, never extends a turn, and its final is marked `echo_suspected`.
- Where the page names speakers, someone other than Bosun must be shown speaking while Bosun's audio plays.
- **Stale answers:** the acknowledgement of each `channels/publish` carries `superseded_by_new_speech`. It is true
when someone has spoken since the last turn the agent was given (a turn still held, or still being said). The text
is said all the same, and the agent decides whether to stop (`meeting_stop_speaking`) or go on. `last_turn_seq` is
the seq of that last turn, so an agent can tell when it is answering an older one.
**Still thinking (0.17.1).** When the speaker goes on after their turn went to the agent but before the agent answered
it (no publish and no `meeting_speak` for it yet), within `turn_extend_s` (default 20 s; 0 turns this off):
- **At once:** the channel gets a message tagged `bosun:meeting-turn-extended`, author `bosun`, metadata `{turn_seq,
turn_text, text_so_far, speaker}`. Without the channel it comes as the live event `bosun:meeting-turn-extended`.
- **The turn is reopened:** it is held again with what the speaker adds. When they are done it comes again as one
merged final: `extends_turn` names the old turn, and `merged_seqs` covers both parts.
- **Publishes for the old turn:** with `superseded_publish: "drop"` (default) they are not said. The acknowledgement
is `delivered: false, dropped: true, reason: "superseded_by_new_speech"`, so a host counts it as not delivered.
With `"speak"` they are said, with `superseded_by_new_speech: true`.
- **Which turn a publish answers:** the one current when it arrived (`turn_seq` in the acknowledgement). Finals carry
`turn_seq`, and a publish may pass `turn_seq` (in its params or `metadata`) to name the turn it answers.
- **Acknowledgement:** an acknowledgement still waiting for the old turn (`ack_delay_ms`) is not played, and the
merged turn gets its own. A clip that already played can't be taken back, so the merged turn's clip then waits out
`ack_cooldown_s` as usual.
- **Not reopened:** a turn the agent has answered (its answer may be playing, and barge-in handles that), a turn of
someone else, and a turn whose transcript was deleted.
**Drafts: removed in 0.19.0.** `meeting_draft` and the settings `draft_enabled` / `draft_max_new_words` (0.18.0)
are gone: in a live test the draft fired once, too early, and gave nothing speculative speech doesn't. Setting
either one (`speech_settings_set`, `meeting_join`) is an error that says so; stored settings that still have them
load without them.
**Speculative speech (0.18.0, `speculative_speech`).** The meeting channel declares `streaming: {speculative: true}`.
A host that supports it streams the `content` of the agent's `channel_publish` while the model writes it, and Bosun
speaks each sentence as soon as it is complete. `complete` with `cancelled` stops it at once, and the publish that
follows is not said twice. The contract is in [docs/STREAMING.md](docs/STREAMING.md). Measured locally: the first
audio starts about 0.4 s after the chunk that completes the first sentence.
**Audio setup (Linux).** Each meeting gets a private PulseAudio server: its own `XDG_RUNTIME_DIR`, socket and
config, with two null sinks:
- **`bosun_out`:** the browser plays the meeting into it; ffmpeg records its monitor, and a second one reads it for
the live transcript.
- **`bosun_mic`:** its monitor is the browser's microphone, silent unless Bosun speaks.
Meetings never hear each other or the host's sound card. The image needs:
```sh
apt-get install -y pulseaudio pulseaudio-utils ffmpeg
```
Each meeting starts its own `pulseaudio` process, so PipeWire alone (`pipewire-pulse`) isn't enough. Both can be
installed side by side: the private servers don't touch the system one.
Meetings need a **headful** browser (headless Chromium plays no audio), so run the service under Xvfb without
`HEADLESS=1`. On macOS or without these packages, `meeting_join` refuses with the reason.
`docker/meeting-test.Dockerfile` builds an image that runs the audio test:
`docker run --rm --init -v "$PWD":/app -w /app bosun-meeting-test xvfb-run -a node --test test/meeting-audio.test.mjs`.
**Limits — what this can't do (yet):**
- **Guests need a host.** Someone in the meeting has to admit the bot. Meetings that don't allow guests, or that
only admit people from the organisation, can't be joined.
- **Google reviews bot-like guests.** Join requests from automated browsers can land in a review queue or be refused
outright ("You can't join this video call", reason `refused_by_google`). Datacenter IPs make this much more
likely. A signed-in profile or a residential connection works best.
- **Meet's and Zoom's markup changes without notice.** Labels and texts are matched in several languages (English
is requested), and the chat sender name is a best-effort guess. A change on Google's or Zoom's side breaks the
join or the chat watch until `platforms/meet.mjs` or `platforms/zoom.mjs` is updated.
- **Tested against stand-ins.** The tests run against local stand-ins (`test/fakemeet.mjs`, `test/fakezoom.mjs`)
built from how the real pages look. The signed-in Meet lobby (dialogs, "Getting ready...", "Join here too") and
all of Zoom are new in 0.9.0 and have not had a live run yet. The Zoom chat watch depends most on Zoom's markup
and is the part most likely to need a fix after the first live meeting.
- **Zoom may connect computer audio with the microphone on.** Bosun mutes it as soon as the control appears. The
browser's microphone is a silent source, so only silence can go out in that moment, and the server has no camera.
- **An objection in the last second before Zoom leaves the page can be missed.** Bosun reads the chat once more
when a meeting ends, but if Zoom has already navigated to its end page, messages it hadn't read yet are gone.
The recording is then kept; the agent can still delete it with `meeting_recording_delete`.
- **Meet's chat in a calendar meeting of another organisation was not tested live.** The group call of 2026-11-02
failed to post with "the Meet chat panel did not open" and could not be reproduced with Glassy's account, whose
plan has no continuous chat. 0.20.0 looks for the chat box in frames too, clicks the chat button only while the
panel is closed (it toggles), retries with the toolbar shown and Meet's Ctrl+Alt+C, reports a chat the host or
their organisation keeps closed as `chat-restricted`, confirms that each message shows up as a new message, and
watches the chat in frames too, so an objection there is heard. When it still
fails, the error says what the page showed (chat button, side panel heading, text boxes, frames), so the next
failure can be fixed from `meeting_status` alone.
- **One mixed audio track.** There is no speaker separation, also not in the live transcript.
- **Bosun doesn't hear itself.** The meeting doesn't play Bosun's own voice back, so the live transcript has only
the others; what Bosun said is what the agent passed to `meeting_speak`.
- **Consent is the operator's responsibility.** The announcement and the opt-out are a courtesy, not a legal basis.
Check what applies where the participants are; some places require everyone's explicit consent before recording.
### Group calls
This is guidance, not a mode. Bosun has no group setting or group tool. An agent in a call with several people uses
the settings below and decides for itself. The advice comes from the first group call (Google Meet, four people and
Glassy, 2026-11-02): acknowledgements on every turn annoyed the group, answers took 1 to 2 s, and the agent talked
too much.
**Turn acknowledgements off.** `speech_settings_set({ "ack_on_final": false })` (the default). In a group every
finished sentence is a turn, so "Mhm" after each one is noise.
**Shorter hold after a complete sentence.** `speech_settings_set({ "turn_hold_complete_ms": 300 })` (the default; the
group call ran with 800). A turn that ends with a complete sentence reaches the agent that much later, when nothing
merges into it: measured with the stand-ins
(`node scripts/measure-turn-hold.mjs`, results in `docs/measurements/turn-hold.json`), the first final reached the
agent 0, 150, 300, 500 and 800 ms after Scribe ended it for those settings, within 11 ms. The rest of the wait is
Scribe itself. In the Zoom call of 2026-11-02 a final reached Bosun 0.64 to 1.54 s after the speaker stopped (median
0.82 s, `vad_silence_s` 0.5 included).
- What a shorter hold costs: a speaker who pauses after a full sentence and goes on is more often split into two
turns. In that call, after 8 of 12 complete sentences the same speaker went on after a pause of 0.68 to 1.06 s.
Taking Scribe's first partial as 0.3 to 0.5 s into new speech, 800 ms merged all 8, 500 ms merged 4 to 7, 300 ms 1
to 4, and 0 ms none or one.
- Why 300 anyway: in a group the agent answers few turns (see below), so a split mostly costs nothing. When it does
hit a turn the agent is answering, `turn_extend_s` (default 20) reopens the turn and drops the stale answer. Such
a turn reaches the agent 0.5 s sooner than at 800. The agent's own thinking and speech come on top and were not
part of this measurement. Go lower than 300 only if the agent waits for `addressed` before it answers anything.
- `turn_hold_ms` (900) stays as it is. It applies after a fragment ("so I was thinking about the"), where the speaker
almost always goes on. It also caps `turn_hold_complete_ms`.
**Clarifications go to the chat.** A short factual note ("The deadline in the doc is 14 Nov.", a link, a number)
goes to `meeting_chat_send`, not to the voice. It doesn't interrupt anyone, and people read it when they want to.
Keep it to one or two lines. The chat works on Meet and Zoom with a signed-in profile (0.20.0 checks that each
message shows up in the chat; see `meeting_status.announce` and `chat_error` when it doesn't).
**Speak only when addressed.** Every final carries `metadata.addressed: true | false` (also in
`bosun:meeting-transcript` events). It is true when the text names the agent: the join `name` (a bracketed note such
as "(notetaker)" is left out, and the first word of a longer name counts on its own), the account name the call
shows (Meet: the name on Bosun's own tile, read once Bosun is in with a signed-in profile), or one of the speech
setting `name_aliases` (up to 20). Case, accents, spelling ("Glassi") and Ukrainian or
Russian forms ("Гласі", "Глассі", "Гласси" for "Glassy") don't matter. Add the nicknames people actually use:
`speech_settings_set({ "name_aliases": ["Гласька", "Glassy AI"] })`. Bosun only marks finals and never answers by
itself.
- Speak when `addressed` is true and the turn asks something of the agent.
- Stay silent otherwise, or put a clarification in the chat.
- A question to "everyone" or "the notetaker" without the name is not marked, so add such words to `name_aliases`
only if the group uses them for the agent. A word that sounds like the name ("glassy eyes") is marked too, so the
agent still reads the turn.
**Announce and privacy.** The announcement goes to the chat before recording starts, as in every call. In a group,
check `meeting_status.announce.sent_to`: anything but `everyone` means some people may not have seen it. Then say it
once by voice (`meeting_speak`) or leave. Everyone can still object in the chat ("please don't record"), and the
agent should take a spoken objection the same way and call `meeting_stop_recording`.
**Example settings for a group call:**
```jsonc
speech_settings_set({ "ack_on_final": false, "turn_hold_complete_ms": 300, "name_aliases": ["Гласька"] })
meeting_join({ "url": "https://meet.google.com/abc-defg-hij", "profile": "google", "name": "Glassy", "live_transcript": true })
```
## Read-only tokens for subagents
Give helper agents (for example `claude -p` subprocesses) a browser without your logins, keepalive, meetings or
handoff: a tenant of their own with policy `ephemeral`.
```sh
BROWSER_TOKENS=me:<token>,swarm:<other token>
BROWSER_TOKEN_POLICIES=swarm=ephemeral # tenants not named stay "full"
BROWSER_EPHEMERAL_MAX_SESSIONS=2 # default
```
With policy `ephemeral` the server enforces, not just hides:
- `tools/list` shows only `session_open`, `session_close`, `navigate`, `read`, `click`, `type`, `select`, `press`,
`back`, `wait_for`, `screenshot`, `sessions`, `macro_list`, `macro_get` and `macro_run` (a macro's steps hold
to the same set; its macros are those of `BROWSER_MACRO_SOURCE`, see [Macros](#macros)). Any other tool answers `not allowed for this token`, whatever its
arguments; its handler is never wired up for the connection.
- `session_open` refuses `profile` ("profiles are not allowed for this token") and any engine but the default
(`ENGINE`, which startup checked, so a session never installs a browser). The session manager enforces both for
the tenant as well, and refuses meeting browsers.
- The tenant has `BROWSER_EPHEMERAL_MAX_SESSIONS` sessions at most, across all its connections. Sessions belong to
the connection and close when it closes (or after `IDLE_TIMEOUT_SEC`).
- Nothing is written to `BROWSER_DATA`: screenshots are returned but never saved, even with `SAVE_SHOTS=1`. Ephemeral
sessions use temporary browser profiles that are deleted on close, and their browsers refuse downloads (a read-only
tenant can't be given download hosts). `upload_create` and `upload` are refused, and so is an upload PUT
that carries this token.
- No MCPL push events.
A policy only narrows what the token can do; the tenant is still separate from yours, so it never sees your profiles
anyway. Egress is not narrowed: block private networks around the service (see [Security notes](#security-notes)).
### stdio bridge
MCP clients that only start stdio servers (such as `claude --mcp-config`) use `bosun-mcp-stdio` (this package's
second bin). It reads `BOSUN_URL` (`ws://` or `wss://`) and `BOSUN_TOKEN`, sends the token as
`Authorization: Bearer` (never in the URL, never printed), and pipes newline-delimited JSON-RPC between
stdin/stdout and the WebSocket, one message per frame.
```json
{ "mcpServers": { "bosun": { "command": "bosun-mcp-stdio",
"env": { "BOSUN_URL": "ws://127.0.0.1:7360", "BOSUN_TOKEN": "<swarm token>" } } } }
```
Records are split on LF only (CRLF is accepted); a record that is not JSON or is over 2 MiB is dropped with a line
on stderr. The opening handshake times out after 15 s.
The bridge ends when its client is gone, also when its stdin stays open because another process holds it:
- stdin ends or closes, or stdout is closed (nobody reads the answers);
- its parent process exits (checked every second; Node has no `PR_SET_PDEATHSIG`);
- `BOSUN_IDLE_EXIT_SEC` (3600; `0` = never) pass with no message either way, no request waiting for its answer (the
client's, such as a long `handoff_wait`, or the server's) and no answer still being written to stdout.
Exit codes: `0` for those, `1` connection failed or lost, `2` bad configuration (missing variable, a non-ws URL,
credentials in the URL, a bad `BOSUN_IDLE_EXIT_SEC`), `3` token refused. The bridge needs only Node and the `ws`
package.
## Chromium sandbox
The browser parses untrusted web content, so it runs with **Chromium's own sandbox** (`chromiumSandbox: true`,
which is what `CHROMIUM_SANDBOX=1`, the default, means). Playwright's default is `--no-sandbox`; Bosun does not use
it (0.7.0 and earlier did).
The sandbox needs something from the host, and startup checks it: Bosun starts the browser once, closes it again,
and **refuses to run** if that fails, printing what to do. On Linux, one of these has to be true:
1. unprivileged user namespaces are allowed —
`echo 'kernel.apparmor_restrict_unprivileged_userns=0' | sudo tee /etc/sysctl.d/60-apparmor-userns.conf && sudo sysctl --system`
(Ubuntu 23.10+ restricts them through AppArmor), or
2. an AppArmor profile grants `userns` to the browser binary, or
3. Chromium's setuid helper next to the browser is root-owned and mode 4755, with `CHROME_DEVEL_SANDBOX` pointing at
it — and then the service must **not** run with `NoNewPrivileges`.
[docs/DEPLOY-SYSTEMD.md](docs/DEPLOY-SYSTEMD.md) has the commands for each. A container or CI box usually allows
none of them; there, `CHROMIUM_SANDBOX=0` is the explicit way out, and the service says so on every start:
```
bosun: WARNING — CHROMIUM_SANDBOX=0: the browser runs without its own sandbox, …
```
Check what is actually running, not just the setting: `ps -eo args | grep -- --type=renderer` must show no
`--no-sandbox`.
## Per-tenant host allowlist
Keep a tenant to the hosts it has business with — so a page that says "now open this other URL" gets nowhere, and
neither does a redirect, an iframe, a `fetch`, a WebSocket or anything a worker starts:
```sh
BROWSER_TENANT_ALLOW='ops=*.example.org,example.org,127.0.0.1:4848; swarm=*.example.org'
```
Entries are separated by `;` (or newlines), hosts within an entry by `,`. A host pattern is:
| pattern | matches |
|---|---|
| `example.org` | exactly that host, on any port |
| `*.example.org` | any subdomain, on any port — **not** the apex (list it separately) |
| `example.org:8443`, `127.0.0.1:4848`, `[::1]:4848` | that host on that port only |
| `*` | every host (the escape hatch) |
### How it is enforced
A tenant that is named gets **its own forward proxy inside the service** (`src/proxy.mjs`, on loopback, with a
random password of its own), and **each of its sessions runs in a browser of its own** (a temporary profile,
removed with the session), launched with that proxy as its only way out
(`--proxy-bypass-list=<-loopback>`, so not even loopback is exempt; QUIC off and WebRTC's non-proxied UDP off, so
nothing can go around it; Chromium does not fall back to a direct connection when a proxy refuses). Full Chromium
(headful, `patchright`) ignores the WebRTC flag, so the browser's profile also gets the `webrtc.ip_handling_policy`
preference before every proxied launch (a persistent profile loses it again on an unproxied launch, if Bosun set
it).
`camoufox` gets the same as Firefox preferences: loopback through the proxy too
(`network.proxy.allow_hijacking_localhost`), WebRTC through the proxy only, no HTTP/3, no DNS prefetching. Those
settings belong to that browser alone — tenants without an allowlist share a browser launched exactly the way upstream
launches it, so one tenant's allowlist never changes another tenant's networking. Every
connection the browser wants — a navigation, each hop of a redirect, a subresource, XHR, a WebSocket from the page
**or from a worker**, a service-worker fetch, a prefetch — arrives at the proxy as a request or a `CONNECT` and is
checked against the allowlist before a byte leaves the host:
- A denied connection is never made. The browser gets `403` from the proxy, so a denied navigation leaves the tab
on that URL with the proxy's one-line reason and `navigate` reports `status: 403` and `refused: "host allowlist"`.
An `https://` hop is refused at `CONNECT`, which fails the navigation in the browser; `navigate` then answers
"navigation refused by the host allowlist of this token: <host>:<port>".
- Denials are logged as `bosun: tenant <t>: blocked <host>:<port> — <reason>`: host and port, never a path or a
query. One line per target per tenant (a repeat is counted, not logged again).
- **DNS is resolved by the proxy, once**, and the connection goes to that address, so the address a decision was
made about is the address that gets used — a name cannot change where it points between the check and the
connection. A name that resolves into loopback or a private range is only followed if the allowlist names that
address as well, so `127.0.0.1:4848` has to be in the list for a name that resolves there.
- **Nothing is intercepted.** The proxy tunnels TLS (`CONNECT`) and streams plain HTTP; it never terminates TLS,
rewrites a response or buffers a body. The browser's own rules therefore all still hold: a redirect to another
allowed host loads **as that host**, with its own origin, storage and cookies, and the tab shows the URL it
really ended up on.
- `navigate` and `keepalive_set` also check the allowlist up front, so the agent gets a clear refusal instead of a
403 page, and `navigate`'s tool description names the hosts it may reach.
A tenant that is not named is unrestricted: no proxy, nothing intercepted, and a browser with upstream's own flags
— exactly upstream behaviour, whatever another tenant's allowlist says.
### What it does not do
- The allowlist is host and port, and a `CONNECT` tunnel is not inspected, so an entry without a port allows a
tunnel to **any** port on that host. Prefer `host:port` entries.
- It is not a URL filter: a path or a query is never looked at (and never logged).
- Hosts you allow can talk to each other with the browser's normal rules; the allowlist decides reachability, not
mutual trust. Keep it to the hosts a tenant needs.
- Plain `http://` goes through the proxy as HTTP/1.1 (no h2c, no QUIC); `https://` is end-to-end inside the tunnel.
- A firewall is still the outer boundary: the proxy is a per-tenant rule inside the service, not a replacement for
network egress policy (see [Security notes](#security-notes) and
[docs/SECURITY-REVIEW-0.7.0.md](docs/SECURITY-REVIEW-0.7.0.md)).
## Security notes
- One service is one trust domain per **tenant** (token): a tenant's connections share its persistent profiles;
sessions are private to the connection. For hard isolation run one process (and data dir) per tenant.
- Web content is untrusted (prompt injection). Run the service where it **can't reach private networks**
(block 10/8, 172.16/12, 192.168/16, 100.64/10, 169.254/16 egress) — otherwise a page can steer it at your LAN.
The `http(s)`-only check on `navigate` does not replace that boundary. Run it as a user that can't read secrets.
- Live-view links are **bearer tokens for the whole handoff** (up to `HANDOFF_TTL_SEC`): anyone holding the link can
open the live view whenever nobody else is connected, or 5 s after the connected viewer went silent. Serve them on
a private network (VPN/mesh) or behind TLS, send them over a private channel, and press Done or Give up when
finished — that revokes the link for good (no replay afterwards). A connected, responsive viewer can't be pushed
out by a second one.
- No rate limiting yet beyond the connection, in-flight and session caps.
- A named tenant can be kept to a list of hosts, enforced by a per-tenant forward proxy that is the browser's only
way out (see [Per-tenant host allowlist](#per-tenant-host-allowlist)). That narrows what the agent, the page, its
workers and its service workers can reach; it does not replace the egress boundary above.
- The browser runs with its own sandbox (see [Chromium sandbox](#chromium-sandbox)). Without it (0.7.0 and earlier,
or `CHROMIUM_SANDBOX=0`) Chromium runs with `--no-sandbox`: then a renderer compromise is directly on the service user,
which is where the profiles and the service's environment (tokens included) are.
- Downloads are off unless a tenant is named in `BROWSER_TENANT_DOWNLOAD_ALLOW`, and then kept only from its hosts
(or pages on them), streamed to the data volume and cancelled past a per-file cap and a per-tenant quota, so a page
cannot fill the volume or RAM. Kept files leave once fetched, after an hour and at every start.
- Uploads are written to `BROWSER_DATA/uploads` (on the data volume, not a RAM-backed `/tmp`) under a per-file cap
and a per-tenant quota, and deleted after use, after an hour and at every start. Once a file is in a page, the page
can send it anywhere it can reach: upload only what you would hand that site.
- Secrets sent for `type { secret_id }` stay in the service's memory (at most 4 KB each, 20 per tenant, 10 min),
never on disk or in its log, and are zeroed once typed. They keep the password out of the agent's transcript, not
out of the page: the page that receives one can do with it what it likes.
- Credential masking around a handoff covers credential **fields** — their value, and values Bosun saw in them. A
page that copies what the human typed into ordinary text, or into something it reads later, is not masked on
**any** engine, and on `chromium` a page can also neutralize the masking helpers outright (they run in its own
JavaScript world; `patchright` uses an isolated one, which stops that technique and nothing more). Treat the page
where a credential is typed as trusted, or don't type it there. Details, probes and the standing test:
[docs/SECURITY-REVIEW-0.7.0.md](docs/SECURITY-REVIEW-0.7.0.md) (finding H2).
## Roadmap
M1 meetings beyond the PoC (Teams, speaker separation, streaming TTS for lower speak latency), tenant quotas/audit log, mobile viewport for handoff, more push events (handoff done/timeout) · M2 vault (`fill_credential`), downloads as content
references, watching pages outside keepalive visits · M3 recording macros from an agent's own calls (`macro_record_start/stop`).
## License
Apache License 2.0 — see [LICENSE](LICENSE) and [NOTICE](NOTICE).
**Status: early preview (0.x).** APIs, tool names and config may change without notice.
This server cannot be deployed
Maintenance
ActivityNo data
ResponsivenessOutdated data