Bosun

The boatswain browser for agents. A self-hosted browser service that gives AI agents a real browser — and calls a human on deck when it needs one (CAPTCHA, cookie wall, 2FA).

Status: prototype (M0). Works end to end in one deployment; APIs will change. See docs/DESIGN.md and CHANGELOG.md.

Security: Chromium runs with its own sandbox (CHROMIUM_SANDBOX), downloads are refused unless a tenant is given download hosts, and a tenant can be kept to a host allowlist (BROWSER_TENANT_ALLOW). A review of 0.7.0, and what is still risky, is in docs/SECURITY-REVIEW-0.7.0.md; a Linux deployment guide is in docs/DEPLOY-SYSTEMD.md.

What it does today

• Headful Chromium (Playwright) under Xvfb — fewer bot flags than headless.

• Engines (ENGINE, or engine on session_open): chromium (default), patchright (Chromium without the CDP/automation leaks) and camoufox (anti-detect Firefox). See docs/ENGINES.md for what each one passes and what it costs.

• MCP server over stdio (Claude Code, Codex, …) or WebSocket (shared service).

• Tools: session_open / session_close / sessions, navigate, read (accessibility snapshot with element refs), click, type, select, press, back, wait_for, screenshot, request_human, handoff_wait, keepalive_set / keepalive_list / keepalive_remove, secret_create, upload_create / upload (unless FEATURES_UPLOADS=0), download_wait / downloads / download_get (tenants with download hosts), macro_save / macro_list / macro_get / macro_delete / macro_run, and (unless FEATURES_MEETINGS=0) meeting_join / meeting_status / meeting_chat_read / meeting_chat_send / meeting_stop_recording / meeting_leave / meeting_recording_delete / recording_get.

• Keepalive for persistent profiles: scheduled visits keep sliding login sessions alive without a model, and a logged-out profile raises an MCPL bosun:needs-human push event. A visit can also watch one value (say, a "43 neue Nachrichten" counter) and push bosun:watch-changed when it goes up. See Keeping logins alive and Watching a value.

• Meetings (PoC): joins a Google Meet (as a guest or signed in) or a Zoom meeting (signed in, in Zoom's web client) as a notetaker, announces itself in the chat, records the audio to a file and leaves; a "please don't record" in the chat stops and deletes the recording. The agent fetches the file with recording_get and transcribes it with any tool, or takes part with a live transcript and speech. See Meetings (PoC).

• File uploads into pages: the agent sends the file with one curl -T to a one-time URL on the MCP port (streamed to disk, up to 300 MB), then upload puts it into a file input, answers the file chooser a button opens, or drops it on a drop zone. See Uploading files.

• Typing secrets without them passing through the agent: the agent's shell pipes the password into a one-time URL on the MCP port (sops … | curl -T - …), and type { secret_id } fills it in. It is kept in memory only, typed once, and goes into password fields unless the call allows others. See Typing secrets.

• Macros: a multi-step flow on a site saved once (macro_save) and rerun with one call (macro_run), with parameters, file uploads, secrets by id, downloads and asserts; it runs only on the hosts it names, and steps that pay, buy or delete an account need an explicit confirmation. See Macros.

• Sessions are ephemeral (wiped on close) or use a named persistent profile of the tenant (stays logged in; no RAM while idle). A session belongs to the connection that opened it: other connections can't see, use or close it, and it is closed when that connection goes away or after IDLE_TIMEOUT_SEC without use.

• Every call is answered within its deadline (60 s by default, at most 120 s), with a clear error when it runs out. Bosun checks the page: if it still answers, the session is usable as it is; if it doesn't, Bosun replaces it with a new tab in the same session (cookies kept, at about:blank). See Per-call deadline.

• navigate and keepalive_set accept only http:// and https:// URLs (no file:, data:, javascript:, about:, chrome:, view-source:, blob:, ftp: …).

• Bot-wall detection after navigation/click (Cloudflare challenge pages, CAPTCHA iframes, "access denied" texts). A bot check that keeps reloading itself (IP reputation) is reported as such: bot check keeps reloading (likely IP reputation); try engine chromium + request_human, or another network.

• Human handoff: request_human returns a link (15 min) to a live view of the tab, made for a phone: Trackpad mode (cursor, drag to move, tap to click, two fingers scroll, hold and drag for sliders) or Tap mode, a masked paste bar for passwords and codes, a key strip, a desktop mouse works as usual. The link can be reopened (one viewer at a time) until Done resumes the agent. See Handoff for what is and isn't protected.

Related MCP server: AIHawk

Run

npm ci                                   # add --omit=optional to skip patchright/camoufox
npx playwright install --with-deps chromium
npx patchright install chromium          # only for ENGINE=patchright
npx camoufox-js fetch                    # only for ENGINE=camoufox (Node 22+)
npm test            # headless, against a local test server; no internet needed
npm run test:engines  # the same suite once per installed engine

# stdio (an MCP client spawns it)
xvfb-run -a node src/server.mjs

# WebSocket service + live view, reachable over a VPN interface (here 100.64.0.10; for a systemd service with
# both listeners on loopback behind a private-network proxy see docs/DEPLOY-SYSTEMD.md)
WS_HOST=100.64.0.10 WS_PORT=7360 BROWSER_TOKEN=<32+ random chars> \
LIVE_HOST=100.64.0.10 LIVE_PORT=7350 LIVE_BASE_URL=http://100.64.0.10:7350 \
  xvfb-run -a -s "-screen 0 1366x900x24" node src/server.mjs

Clients authenticate with Authorization: Bearer <token> (the old ?token= query parameter still works but puts the secret in URLs and logs; prefer the header). The WebSocket listener refuses to start on a non-loopback address without a token; on loopback an unauthenticated listener needs ALLOW_NO_TOKEN=1 (local development only). The two listeners are separate: WS_HOST/WS_PORT carry MCP (token-protected), LIVE_HOST/LIVE_PORT serve the human's live view (protected by the handoff link). Both default to 127.0.0.1; bind them to a VPN/mesh address or put a TLS reverse proxy in front, and set LIVE_BASE_URL to the address the human's phone uses.

Environment

The speech values above (TTS_DEFAULT_BACKEND, ELEVENLABS_TTS_MODEL, ELEVENLABS_VOICE, TTS_VOICE, LIVE_TRANSCRIPT_VAD_SILENCE_S) are fallbacks only: each tenant sets its own with speech_settings_set, and a call's own argument wins over both (see Live transcript and speaking).

All numbers are validated at startup; a bad value stops the service instead of silently disabling a limit.

Per-call deadline

Every tool call is answered within its deadline: TOOL_TIMEOUT_SEC (60 s), or the tool's own value in TOOL_TIMEOUTS, at most 120 s. Before 0.13 a page that stopped answering (a renderer stuck in a script, say) could hold a read for as long as the agent was willing to wait, and every later call on that session waited behind it.

A page call is cut off 8 s before its deadline (a third of a deadline under 24 s). Then Bosun asks the page to evaluate 1:

• It answers within 3 s. The error says the page still responds and the session is usable. If what the call started is still running (a click waiting for its element), the session's next call waits for it to stop, at most 20 s. After that the tab is replaced, as below, to stop it. Two calls never act on the page at once.

• It doesn't. Bosun opens a new tab in the same browser context and closes the stuck one; then the session uses the new tab, at about:blank, so navigate again. The session keeps its id and its cookies. A site that keeps its login in sessionStorage is logged out. The session is closed instead (page-wedged) when no new tab opens within 15 s, when the stuck tab doesn't close, when the new tab goes away first, or when the cut-off call still hasn't stopped 3 s after its tab closed. Firefox (camoufox) takes about 5 s to open a tab next to a stuck one, so there the answer may come while the tab is still being replaced; it says so, and the next call waits for it.

A call still waiting behind others when too little of its time is left is answered at once and never runs. A session_open that times out closes the session it was opening as soon as that open finishes, so no session is left that the agent doesn't know about. wait_for (which waits up to 60 s by itself) gets at least 75 s unless TOOL_TIMEOUTS names it. These tools keep their own limits and have no deadline: download_wait, request_human, handoff_wait and the meeting tools. Keepalive visits and meetings are never cut off by the deadline of the call that set them up.

Engines

Only the engines in FEATURES_ENGINES (default: all three; ENGINE is always included) can be asked for; session_open's schema lists exactly those, and the session manager refuses the others. A session asking for an engine that is enabled but isn't installed gets an error naming the install command; there is no silent fallback to another engine. A persistent profile remembers its engine family (Chromium or Firefox) and refuses the other. patchright and camoufox-js are optionalDependencies; their browsers are separate downloads.

Chromium spawns many threads: under systemd give the unit a generous TasksMax (e.g. 2000).

Finding elements

click, type, select, upload and screenshot take the element as one of:

• ref from read ("e12"): that very element. Refs change with every read, so a macro can't use them.

• role (+ name), selector (CSS, xpath=…, text=…, any Playwright selector) or text (a substring of the visible text). Bosun takes the first visible match: a hidden copy that comes earlier in the page (the closed popover of a toolbar, the small-screen copy of a menu) is skipped. Finding it and waiting for it to be clickable (or editable) share 15 s. select and upload fall back to a hidden match when none is visible, since a hidden <select> or file input still takes options or files.

For icon buttons with no name (the button [ref=e149] lines in a read), add one or more of these:

• nth: which visible match, 0 = first (default), -1 = last. With nth, after or before, the element that was counted is the one acted on, even if the page adds others meanwhile.

• after: "<text>": only matches after the first visible element showing that text, in page order, the nearest first. { role: "button", after: "Share" } is the first button after Share; with nth: 1, the one after that.

• before: "<text>": only matches before it, the nearest first.

Page order runs through open shadow roots. Neither after nor before counts an element that contains the anchor or sits inside it. The anchor is the first visible element with that text: if the text shows up twice (a sidebar entry and a toolbar button, say), pick an anchor that shows up once. read shows what comes after what.

When nothing can be clicked, the error says why, instead of a bare timeout. It names elements by tag, id and classes only, masked as read masks them:

selector xpath=//*[normalize-space(text())='Share']/parent::*//button matched 2 elements, none visible within 15 s
  (the first is <button>, inside a display: none <div>)
nothing matches selector #send within 15 s
no visible element shows the text "Download all" (after) within 15 s
role button: 1 visible match after "Share", so nth 3 is not there (4 more hidden)
the element did not become clickable within 15 s: it is covered by <div class="tooltip-mask">, which would get the click
the element did not become clickable within 15 s: it is disabled

read { selector }, wait_for { selector | text }, a macro's assert and a keepalive's logged_in_check also go by the first visible match.

Uploading files

Two calls and a curl between them:

upload_create { filename: "report.pdf", size_bytes: 48213 }
  -> { file_id: "f3a…", put_url: "http://127.0.0.1:8002/upload/f3a…?t=…",
       curl: "curl -sS --fail-with-body -T 'report.pdf' 'http://127.0.0.1:8002/upload/f3a…?t=…'", valid_min: 15 }
$ curl -sS --fail-with-body -T report.pdf 'http://127.0.0.1:8002/upload/f3a…?t=…'
  -> {"file_id":"f3a…","filename":"report.pdf","size":48213}
upload { session, selector: "input[type=file]", file_id: "f3a…" }      # or ref / role+name / text
  -> { method: "input", input_files: ["report.pdf"], url, title }
click { session, role: "button", name: "Send" }

• Where the bytes go. The PUT goes to the same host and port as MCP (the URL is built from the Host the MCP client connected with, or UPLOAD_BASE_URL), so whatever reaches MCP reaches it. The body is streamed straight to BROWSER_DATA/uploads/<tenant>/<file_id>/<filename> (0600): never held in memory, never base64. The browser reads the file from there itself; no bytes cross the MCP connection.

• The URL works once, for UPLOAD_URL_TTL_SEC (15 min), and only for the tenant that asked for it. Its secret (?t=) is the credential. Instead of it, PUT /upload/<file_id> with the tenant's own token in Authorization: Bearer works too; another tenant's token is refused either way. size_bytes is optional; when given, the PUT must carry exactly that many bytes.

• Small files (up to 1 MB): upload_create { filename, content_base64 } stores the file at once, no PUT.

• Limits. UPLOAD_MAX_BYTES per file (300 MB) and UPLOAD_TENANT_QUOTA_BYTES per tenant (1 GB) are checked against Content-Length before any byte is read, and again while the body streams: past either, the answer is 413 and nothing stays on disk. A tenant holds at most 50 uploads at a time (waiting, arriving or ready), and streams at most 4 at once (16 across the service); past that a PUT gets 429 and its URL stays valid.

• Files never cross tenants. A file_id of another tenant answers like one that never existed.

• Deletion. A file is deleted when the session that used it closes (the last one, if several did), UPLOAD_TTL_SEC (1 h) after it arrived, or when the service starts. Submit the form within the hour.

• Read-only tokens (policy ephemeral) can't upload: the tools are refused, and so is a PUT with such a token.

How upload puts the files in:

1. The element is an <input type=file>, hidden or not, or the <label> of one: the input gets the files directly. Several files (files: [id, id]) only when it has multiple.

2. Anything else (an "Upload" button, a link) is clicked, and the file chooser it opens is answered.

3. If no chooser opens within 5 s, the files are dropped on the element: dragenter, dragover, drop with the files in a DataTransfer. accepted says whether something on the page took the drop.

method: input | chooser | drop forces one of them.

Drag-and-drop is best effort:

• The events are synthetic (isTrusted is false). A drop zone that checks for real user events ignores them.

• On camoufox, Firefox hides files dropped from outside the page's own scripts, so Bosun dispatches the drop from a script element in the page. A page whose Content-Security-Policy refuses inline scripts gets no files; note says so. Use chromium for that page.

• An element inside a shadow root may not be found by the camoufox drop script; drop on its host, or use another engine.

Prefer the page's file input or its "Upload" button where there is one: both are as good as a person picking a file.

Typing secrets

type { value: "<password>" } puts the password into the tool call, so it ends up in the agent's transcript and in whatever logs the MCP client keeps. With a secret id the agent's shell sends the password to Bosun itself, and the tool call carries only the id:

secret_create { label: "github password" }
  -> { secret_id: "s9c…", put_url: "http://127.0.0.1:8002/secret/s9c…?t=…",
       curl: "curl -sS --fail-with-body -T - 'http://127.0.0.1:8002/secret/s9c…?t=…'", valid_min: 15, max_bytes: 4096 }
$ sops -d --extract '["github"]["password"]' secrets.yaml | curl -sS --fail-with-body -T - 'http://127.0.0.1:8002/secret/s9c…?t=…'
  -> {"secret_id":"s9c…","label":"github password","bytes":24,"valid_min":10}
type { session, selector: "#password", secret_id: "s9c…", submit: true }
  -> { url, title, secret: "typed and deleted" }

• Pipe it, never print it. The curl line reads the secret from stdin (-T -). Feed it from where the secret lives (sops --extract, pass show, a file), not from echo with the secret on the command line. One trailing newline is dropped, so pass show x | curl … works. Up to 4096 bytes of UTF-8; empty or invalid UTF-8 is 400, more is 413.

• The PUT goes where uploads go: the MCP host and port, one request, within 15 min, for the tenant that asked. PUT /secret/<id> with the tenant's own Bearer token works instead of ?t=; another tenant's token is 403.

• Memory only. Bosun keeps the secret in a buffer of its own: never on disk, never in a log line or an event. Log lines name the id and the length in bytes. The buffer is zeroed when the secret is typed, 10 min after it arrived, or when the service stops; a restart loses every secret. A PUT gets 60 s in all (30 s without a byte).

• No debug logs. Playwright's DEBUG=pw:api log prints what fill types, pw:protocol / pw:channel print the messages that carry it, and Node's NODE_DEBUG=stream prints the buffers a PUT body arrives in. While DEBUG turns any of those Playwright logs on, or NODE_DEBUG is set at all, secret_create, the PUT and type with secret_id are refused, and the service warns at start. Node logs a body it parses before Bosun can refuse it, so a PUT to a URL issued before a restart under NODE_DEBUG still lands in that log: don't set it where secrets are sent.

• Typed once. type with secret_id takes the secret out of the store and fills the field. A second type with that id gets "no secret …", the same answer as for an id that never existed or belongs to another tenant. If the fill fails after the secret was taken, it is gone too: call secret_create again. Neither the result nor an error carries the value. The result's URL and title have it replaced by [masked] (also URL-encoded, for a form that submits with GET); later results (read, navigate, sessions, …) return the page's URL and title as they are, so a page that puts the secret there shows it to the agent from then on.

• One element. The type check, the masking mark and the fill all go to the element found first; a page that swaps the field for another one before the fill gets nothing typed. The fill types through the page's keyboard, so a page can still move the focus while it runs. If the checked field didn't get the secret, the call fails and Bosun leaves the page (about:blank), so nothing on it can show the secret to a later read or screenshot; if it did, every field that holds it is masked.

• Password fields only, unless you say otherwise. The field must be an <input type=password>. For an OTP, an e-mail address or a user name pass allow_non_password: true; any text <input> or <textarea> then takes it. Checkboxes, buttons, file inputs and non-inputs are refused. A refusal before the fill leaves the secret unused.

• Masked afterwards. A field Bosun typed a secret into counts as a credential field from then on, whatever its type or label: read shows [masked] for it (and for the value wherever else it appears in the page dump), screenshot blacks it out, and it is cleared after a handoff that ends with Give up, like a password field.

• Read-only tokens (policy ephemeral) get no secrets: secret_create is refused, so is type with secret_id, and so is a PUT with such a token.

• stdio: without WS_PORT there is no port to PUT to, and secret_create says so.

What this does not change: once the secret is in the page, the page has it. A page can copy it into ordinary text, into its title or URL in a form Bosun doesn't recognise, or send it anywhere it can reach, and on engine chromium a page can switch off the masking helpers (see Security notes). Type secrets only into pages you would give them to by hand.

Downloading files

Off for every tenant unless BROWSER_TENANT_DOWNLOAD_ALLOW names it. A tenant that is not named gets the browser of 0.10 and earlier: its contexts refuse downloads and nothing a page fetches is written to disk.

BROWSER_TENANT_DOWNLOAD_ALLOW='glassy=web.plaud.ai,*.plaud.ai'   # host, *.host, host:port, * — like BROWSER_TENANT_ALLOW

click { session, selector: "button:has-text('Export')" }
  -> { url, title, downloads: [ { download_id: "d4c…", state: "ready", filename: "Meeting.txt", size: 18342,
                                  mime: "text/plain", source_host: "<bucket>.s3.amazonaws.com",
                                  page_host: "web.plaud.ai", allowed_by: "page origin", expires_in_min: 60 } ] }
download_wait { session, timeoutMs? }        # a download that takes longer than click waits (5 s), or one started otherwise
downloads { session? }                       # running, ready, refused and failed ones, with reasons
download_get { download_id }
  -> { get_url: "http://127.0.0.1:8002/download/d4c…?t=…", curl: "curl -sS --fail-with-body -o 'Meeting.txt' '…'", valid_min: 15 }
download_get { download_id, inline: true }   # UTF-8 text up to 256 KB comes back as content instead

• Which downloads are kept. One whose URL's host is on the tenant's list, or one started by a page whose top-level origin is on it. So a Plaud export kept from web.plaud.ai may come from S3, api.plaud.ai or a blob: the page made, without listing those. blob: and data: URLs have no host of their own and are judged by the page. A popup with no page of its own (about:blank) is judged by its opener. Anything else is cancelled, and the reason is in click, download_wait, downloads and the event, e.g. example.com from page example.com: neither is in the download allowlist of this token.

• Where the bytes go.

  • A session that may download gets a browser of its own. That browser writes downloads, streamed, into a directory of that session on the data volume (BROWSER_DATA/downloads/<tenant>/.in-…): never into /tmp (tmpfs, counted against MemoryMax), and never through Bosun's memory.

  • The file's size is checked whenever the browser writes to it (the directory is watched), and every 250 ms. Past DOWNLOAD_MAX_BYTES or the tenant's DOWNLOAD_TENANT_QUOTA_BYTES, the download is cancelled mid-stream and the browser deletes what it wrote.

  • At most 50 downloads per tenant, running or kept.

  • A finished file moves to BROWSER_DATA/downloads/<tenant>/<download_id>/<filename> (0600). There it outlives its session.

• Fetching.

  • download_get gives a one-time GET URL on the MCP port, the same port and address as the upload PUT (UPLOAD_BASE_URL applies), valid for DOWNLOAD_URL_TTL_SEC.

  • Instead of ?t=, GET /download/<download_id> with the tenant's own Authorization: Bearer token works too.

  • Once the whole file has been sent, it is deleted. An interrupted GET leaves it, so call download_get again.

  • At most 4 GETs per tenant at once (429). A GET whose client reads nothing for 60 s, or that takes over an hour, is cut off.

• Gone. Once fetched, DOWNLOAD_TTL_SEC (1 h) after it finished, and all of them at every start and on shutdown.

• Tenants. Another tenant's download_id answers like one that never existed, and a GET with another tenant's token is refused. Read-only tokens (policy ephemeral) never download: the service refuses to start if one is named.

• Events. MCPL feature set bosun.download, tag bosun:download (bosun:state=ready|refused|failed), when a download finishes or is refused.

• What starts a download. Only sessions the agent opens with session_open (profile or not). Keepalive visits and meetings never keep a download.

• navigate to a download URL. The page stays where it was, and the result carries downloads.

• Limits.

  • The type (mime) comes from the file name's extension; the browser does not report the response's.

  • The limits are enforced by watching and cancelling, not by a filesystem quota: a very fast download can pass one by what arrives before the cancel lands. Give BROWSER_DATA a quota or a volume of its own for a hard bound.

  • A tenant that is not named sees why in click: downloads are off for this token.

  • A download a person starts during a handoff is kept for the agent like any other.

Macros

An agent that keeps repeating the same flow on a site (Plaud: "Add audio" → "Import audio" → pick the file → load the list again → "Generate" → "Generate now" → export → download) saves it once as a macro and reruns it with one call. No model runs between the steps.

macro_save { name: "plaud-import", site: ["web.plaud.ai", "*.plaud.ai"],
             params: { audio: { type: "file_id" }, file_name: { type: "string" } },
             steps: [ { tool: "navigate", url: "https://web.plaud.ai/" },
                      { tool: "click", text: "Add audio" },
                      { tool: "upload", text: "Import audio", file_id: "{{audio}}" }, … ] }
  -> { saved: "plaud-import", version: 1, steps: 16, … }
macro_run { name: "plaud-import", session, args: { audio: "f3a…", file_name: "Standup" }, timeout_sec: 900 }
  -> { ok: true, steps: [ { step: 1, tool: "navigate", status: "ok", ms: 1302 }, … ],
       reads: [ { step: 12, as: "summary", text: "…" } ], downloads: [ { download_id: "d4c…", filename: "Standup.txt", … } ] }

The whole Plaud flow is in examples/plaud-import.json (written from the UI, not run against Plaud by the tests).

A macro is a per-tenant JSON document: name (lowercase letters, digits, . _ -), description, site (the hosts it may run on, patterns as in BROWSER_TENANT_ALLOW, * not allowed), params ({ name: { type: string | number | file_id | secret_id, required?, default?, description? } }) and up to 100 steps. macro_save with a name that exists saves a new version; the last 20 are kept, and macro_get and macro_run take version. created_by (default: the tenant) and created_at are stored with each version. macro_list shows the latest version of each, macro_delete removes all of them.

Steps use the page tools' own vocabulary, without session: each { tool, …that tool's arguments, optional?, note? }.

• Any string may hold {{param}}. A step that names a param the run wasn't given (one that is not required) is skipped, so an optional param switches its steps on and off.

• optional: true: the step's failure is noted (failed_optional) and the run goes on.

• Nothing else: no script, no evaluate, no code. Each step runs through the same handler as the tool of that name, so everything the tool checks applies: the host allowlist, bot-check detection, masking, the downloads allowlist, the upload and secret rules, and the token's policy.

macro_run { name, session, args, version?, dry_run?, timeout_sec? } runs the steps in order in the session.

• Each step has its tool's deadline (TOOL_TIMEOUT_SEC / TOOL_TIMEOUTS; assert that of wait_for), and the whole run timeout_sec (default 300, at most 900), the failure report included: the steps stop 15 s (a third of a shorter run) before it ends. A step's own wait (wait_for, assert, download_wait) is shortened to end within its deadline. A run longer than your MCP client's request timeout (60 s in the SDK) needs that timeout raised, or a progressToken: Bosun sends a progress notification after every step, which a client with resetTimeoutOnProgress counts as a sign of life. A cancelled request stops the run before its next step.

• The answer: ok, every step's status (ok, failed_optional, skipped, failed, not_run) and time, reads (the read steps' text), downloads (fetch them with download_get) and the screenshot steps' images. A download_wait step also takes a download that the click before it already reported.

• On failure (isError): failure { step, tool, error, url, title, page (the start of the a11y snapshot, masked as in read), screenshot (attached as an image, masked) }, to fix the macro and save a new version.

• dry_run: true checks the args and returns the steps as they would run, without a session or the page.

Safety.

• Site. A navigate to a host outside site is refused before the first step runs (also when an arg made the URL). The page is checked again in the session's queue right before each page operation of a step (another call on the session may have moved it), and once more right before click, type, select or upload acts on its element (and before upload fills a file chooser or an input of its own), which must itself be in a document on the site (not in a frame from elsewhere). Any navigation of the tab outside the site while a step runs, a redirect that comes back included, stops the run after that step, even an optional one. about:blank counts as no page. The tenant's host allowlist still applies on top, enforced by its proxy.

• Secrets go in only as a {{param}} of type secret_id, in type's secret_id: a stored macro never holds one, nor a file_id (both are used once). A secret_id or file_id param can't go into any other field. A type with value into a password field (also through its <label>) is refused, whether the value was stored or passed as an arg, and a field that turns into one while it is filled is cleared.

• Risky steps. A step whose text (any of its strings, after the args are filled in) says pay, buy, purchase, submit or place order, checkout or delete account (also: kaufen, bezahlen, zahlungspflichtig, kostenpflichtig bestellen, Konto löschen) is refused unless the macro was saved with confirm_risky: true. A click is also checked against what the element it hits says (all its text, shadow roots included, and every alt, aria-label, aria-labelledby, title and value in it, each in full), and what every button, link or label around it says, so a selector can't point at a "Buy now" button (or the icon in it) unnoticed (an upload, which clicks what is not a file input, too): once the element is ready to click, then that same element is clicked. If it can't be looked at, it isn't clicked. macro_list and macro_save show confirm_risky and the risky steps.

• Storage. BROWSER_DATA/macros/<tenant>/<name>.json (0600), all kept versions in one file, written atomically; it survives restarts and upgrades with the data volume. Per tenant at most 200 macros of 256 KB each.

• Read-only tokens (policy ephemeral) may macro_list, macro_get and macro_run but not save or delete, and their steps hold to their policy (no upload, download_wait or secret_id). They have no macros of their own: BROWSER_MACRO_SOURCE=swarm=me lets the read-only tenant swarm list, get and run those of tenant me, in its own sessions, under its own allowlist.

Handoff

• One viewer at a time, reopenable until the end. While the handoff runs, the link opens the live view whenever nobody else is connected — closing a chat app's in-app browser and tapping the link again works. A second viewer is refused (its page says the link is open on another device and retries) until the first disconnects or has been silent for more than 5 s. Opening the page alone (e.g. a link preview) doesn't connect. Done, Give up, expiry, session_close and disconnect of the agent end the handoff; from then on the link is dead (404).

• Controls (bottom bar): Tap | Trackpad (trackpad is the default on touch screens), Keys (Tab, Enter, ⌫, Esc, arrows, and the phone keyboard), the paste bar, Give up, Done. Trackpad: one finger moves the cursor (accelerated), tap clicks at the cursor, two fingers scroll, hold then drag holds the button down. The service moves the page's mouse along an interpolated, slightly jittered path rather than jumping.

• Paste bar: paste (long-press → Paste on iOS) or type into the masked field, then Insert or Enter: the text goes into the page's focused field (tap that field first) and the bar is cleared. The eye button shows what's in it. The text is not logged, not kept by the service after it is typed, and never sent to the agent — but once it is in the page, the page field rules below apply (a password field is masked; an ordinary field is not).

• A reconnecting viewer gets the last frame at once, and a page that doesn't repaint is re-captured every 2 s.

• While a handoff is active every page tool (read, screenshot, navigate, click, type, select, press, back, wait_for) is refused, and sessions/handoff_wait report no URL or title. Tool calls that were already running when request_human came in finish first; the human gets the tab after them.

• handoff_wait waits 50 s by default (below the MCP SDK's 60 s request timeout); call it again on still_waiting. Longer timeoutSec needs a longer client-side request timeout.

• After the handoff the agent can read the page again. What the human typed into password, PIN, OTP and card security-code fields (detected by input type, autocomplete, name/id and label, across frames and open shadow roots, including show-password toggles) is masked as [masked] in read output and blacked out in screenshots. After Give up / timeout / cancel those fields are emptied before the agent resumes; if that fails, the session is closed instead. If a frame can't be inspected, read/screenshot fail rather than return unmasked output. The handoff ends only after the human's last input has reached the page.

• A password field that is revealed with a show-password toggle stays masked: an event-driven tracker remembers every input that ever was type=password (incl. shadow roots), and Bosun also notes them when a handoff starts and every 500 ms during it. On chromium Bosun's helpers run in the page's own JavaScript world, so a page can detect them — and replace them, which switches masking off for fields that aren't sensitive by markup (probed, see docs/SECURITY-REVIEW-0.7.0.md finding H2); on patchright and camoufox they run in an isolated world, which stops that and nothing else (details in docs/ENGINES.md).

• Not protected: text the human types into ordinary fields, secrets the page itself echoes as normal text, custom widgets that aren't <input>/<textarea> (e.g. per-digit OTP boxes built from divs), closed shadow roots. Those are visible to the agent after Done. Ask the human to finish the login (submit it) before pressing Done.

Keeping logins alive

A persistent profile stays logged in only as long as the site's session does. Many sites extend the session each time it is used ("sliding" sessions) and end it after some idle time. Bosun can visit a profile on a schedule so that idle time never runs out — without a model:

// once, after the human logged the profile in (session_open({profile:"shop"}) + request_human)
keepalive_set({ "profile": "shop", "url": "https://shop.example/account",
                "every_hours": 12, "jitter_min": 30,                     // defaults; every_hours >= 1
                "logged_in_check": { "text": "My account" } })          // or selector / url_not_contains
keepalive_list()     // schedule, next run, last results: ok | logged_out | bot_check | error
keepalive_remove({ "profile": "shop" })

The interval is every_hours (default 12) or every_minutes (at least 30), each visit shifted by up to ± jitter_min (default 30, or a quarter of a shorter interval). quiet_hours ("22-08", "22:30-07:00", local time in TZ, default Europe/Vienna) makes the interval four times longer while it lasts, so a 30-minute watch visits about every two hours at night. An interval that starts before a quiet window is stretched only for its part inside it. Quiet hours follow the wall clock, and on DST nights the time is counted as it really passes. Calling keepalive_set again replaces the whole configuration of that profile.

Each visit, at the interval ± jitter_min (first one an interval after keepalive_set):

• opens the profile with the engine and mode it was last used with (recorded in the profile; a profile used headful is never opened headless, so the fingerprint stays the same);

• loads url, then for 20–60 s moves the mouse along human-looking paths (the live view's trackpad interpolation), scrolls 1–3 times and sometimes rests over a link. It never clicks, types or navigates;

• checks logged_in_check (every given condition must hold: text visible, selector visible, URL doesn't contain url_not_contains), reads the watched value if there is one, closes the profile and records the result (time, status, duration; last 10 kept).

Visits run one at a time, a few minutes apart, and never take the last free session slot from agents. A profile an agent has open is skipped (it's in use anyway); an agent opening a profile during a visit takes it over at once. Schedules and results live in BROWSER_DATA/keepalive.json (mode 0600) and survive restarts. Nothing from the page is stored or logged beyond the check result and the watched value.

When a visit finds the profile logged out or stuck at a bot check, Bosun pushes an MCPL event to the tenant's agent (feature set bosun.handoff, tags bosun:needs-human, bosun:source=keepalive, bosun:reason=logged-out|bot-check, bosun:profile=<name>; origin {profile, url, reason}), so the agent can open the profile and call request_human. Events are only sent to connections whose host asked for MCPL at initialize and granted pushEvents (featureSets/update); if none is connected, up to 20 per tenant wait for the next one. There is no retry loop: the next attempt is the next scheduled visit.

Limits — what this can't do:

• It only helps sites with sliding sessions. A cookie with an absolute expiry (e.g. "remember me for 30 days") ends on that date however often the profile is visited.

• Device re-verification, periodic re-login and 2FA prompts still need a human (that's what the event is for).

• Visits look like a person glancing at the page, but too-frequent visits can look bot-like, especially at the same time every day — hence the 12 h default and the jitter. Don't go below a few hours unless the site's idle timeout requires it.

• The visit runs from Bosun's IP with Bosun's browser; if the site ties the session to a different network or device, keeping it alive here doesn't help.

Watching a value

A visit can read one value from the page after the logged-in check and tell the agent when it changes. For willhaben's message counter in the header:

keepalive_set({ "profile": "willhaben", "url": "https://www.willhaben.at/iad/myprofile/chat",
                "every_minutes": 30, "quiet_hours": "22-08",
                "logged_in_check": { "url_not_contains": "/login" },
                "watch": { "text_regex": "(\\d+) neue Nachrichten", "kind": "count", "label": "willhaben messages" } })

• What is read: the selector's textContent if given, else the page's visible text. text_regex picks the value (its first capture group, else the whole match). The regex runs in the page with a 5 s limit, so a pathological pattern can't stall the service.

  • kind: "count": the first whole number in the text. Thousands separators inside it are ignored, but the regex has to capture them: use ([\d.]+) rather than (\d+) if the site writes "1.234". No match counts as 0, since many badges disappear when nothing is new.

  • kind: "text": the text with whitespace collapsed, at most 200 characters. No match is null.

• When it is reported: the first successful read is the baseline and raises no event. After that, a count that went up or a text that differs raises one bosun:watch-changed event. A count that went down (you read the messages) is stored silently, and the next rise is measured from there.

  • Nothing is read when the visit isn't logged in.

  • A read that fails is recorded as watch timeout, no number in the watched text or watch failed (say, a selector that isn't valid) and leaves the stored value alone. Only a selector or regex that matches nothing counts as "no match". An agent taking the profile over mid-read aborts the visit, and nothing is recorded.

  • Changing selector, text_regex, kind or the url starts a new baseline; a new label doesn't. A visit still running for the old configuration records nothing.

• What is stored: only the value (a number, or at most 200 characters), when it was read and when it last changed, shown under watch in keepalive_list, plus baseline / unchanged / changed / lower per visit in last. Nothing else from the page.

The event uses its own MCPL feature set bosun.watch: tags bosun:watch-changed, bosun:source=keepalive, bosun:profile=<name>; origin {profile, label, old, new, url}; text e.g. Bosun watch "willhaben messages" on profile "willhaben" changed: 43 -> 45 (https://…). Delivery and queueing work as for bosun:needs-human.

Host recipe change: a recipe without enabledFeatureSets for Bosun gets both feature sets. One that lists "bosun.handoff" must add "bosun.watch", or use "bosun.*", otherwise watch events are not delivered (they wait in the queue, up to 20 per tenant, until a connection grants the set):

"mcpServers": { "bosun": { "url": "wss://bosun.example/mcp", "transport": "websocket",
                           "enabledFeatureSets": ["bosun.handoff", "bosun.watch"] } }

Limits: a watch sees only what that one page shows (the visible text, or a selector's text, at visit time). Between visits nothing is watched, so with every_minutes: 30 a new message is reported about 15 minutes after it arrived on average and up to about 40 minutes after (longer in quiet hours). Several changes between two visits make one event, carrying the latest value.

Meetings (PoC)

FEATURES_MEETINGS=0 switches meetings off: the tools are not listed and their handlers are not wired, and nothing of the below runs — no recordings directory, no audio server. Read the consent notes before using them.

Bosun can attend a Google Meet or a Zoom meeting, record its audio and leave. There are two ways to use it.

Mode 1, notetaker (record only). meeting_join { url } and nothing else. No ElevenLabs key, no voice, and the microphone is never unmuted. Bosun announces itself in the chat, records, and leaves when the meeting ends. Afterwards the agent calls recording_get { meeting_id }, downloads the file with the curl line it returns, and transcribes it with any tool (whisper, Plaud, ElevenLabs batch, …). This is the mode for a passive notetaker agent.

meeting_join({ "url": "https://meet.google.com/abc-defg-hij" })      // -> { "meeting_id": "m1a2b3c4d5e6", "state": "waiting", ... }
// … the call happens; bosun:meeting-ended arrives, or meeting_status says "ended" …
recording_get({ "meeting_id": "m1a2b3c4d5e6" })
//  -> { "get_url": "http://<host>:<port>/recording/m1a2b3c4d5e6?t=…", "size": 10874213, "seconds": 905, "valid_min": 15,
//       "curl": "curl -sS --fail-with-body -o 'm1a2b3c4d5e6.ogg' 'http://<host>:<port>/recording/m1a2b3c4d5e6?t=…'" }

recording_get gives the calling tenant's own recording only, as a one-time GET URL on the MCP port, valid DOWNLOAD_URL_TTL_SEC (15 min). It is refused while the recording is still being written and once it is deleted. The file is not deleted by the fetch (the retention sweep and meeting_recording_delete do that): call recording_get again for a new URL. A read-only (ephemeral) token may call it. There is no inline mode; the audio is binary. The recording's path in meeting_status and bosun:meeting-ended is a path inside the browser container: an agent host cannot open it, so fetch it with recording_get.

Mode 2, participant. meeting_join { url, live_transcript: true } and the speaking tools: the agent hears the call as text while people speak, and can speak, mute and unmute, raise a hand, answer on the meeting channel, prepare a draft and speak speculatively (see Live transcript and speaking). It needs ELEVENLABS_API_KEY for the live transcript (Scribe realtime); the Supertonic voice speaks without the key, only the live transcript does not work without it. When to speak is always the agent's call: Bosun only provides the mechanism. The recording is made in this mode too.

Platforms:

• Meet: as an unauthenticated guest, or with a Bosun profile signed in to a Google account.

• Zoom: in Zoom's web client, with a Bosun profile signed in to a Zoom account, or as a guest. Zoom may turn an anonymous browser guest away (sign-in-required or captcha); it admitted guests in our tests, so try a guest join first and use a profile when Zoom refuses. Teams is not supported yet. Each platform lives in one file under src/meetings/platforms/ (meet.mjs, zoom.mjs) behind a small interface, and the meeting flow around it is shared.

meeting_join({ "url": "https://meet.google.com/abc-defg-hij" })   // name, announce_text, announce ("chat" | "host" | "none"),
                                                                   // max_wait_admit_min (10), profile, engine, leave_on_optout (true)
meeting_join({ "url": "https://us02web.zoom.us/j/81234567890?pwd=…", "profile": "zoom" })
//  -> { "meeting_id": "m1a2b3c4d5e6", "platform": "zoom", "state": "waiting", "joined_as": "account", ... }
meeting_join({ "url": "https://meet.google.com/abc-defg-hij", "profile": "google", "dry_run": true })
//  -> { "dry_run": true, "state": "lobby", "lobby": "account", "join_button": "Ask to join without microphone & camera",
//       "microphone": "on (no device found)", "camera": "no device", "challenge": false, "page": "…/meet/abc-defg-hij", ... }
meeting_status({ "meeting_id": "m1a2b3c4d5e6" })   // joining | waiting | admitted | recording | ended, participants, recording_seconds,
                                                   // announce: { mode, sent_to: everyone | host | none | failed, reason, failures }
meeting_chat_read({ "meeting_id": "…", "since": 0 })      meeting_chat_send({ "meeting_id": "…", "text": "…" })
meeting_stop_recording({ "meeting_id": "…", "delete": true })   meeting_leave({ "meeting_id": "…" })
meeting_recording_delete({ "meeting_id": "…" })
meeting_join({ "url": "…", "live_transcript": true })            // transcript_partials (true), vad_silence_s
meeting_transcript({ "meeting_id": "…", "since": 0 })             // finals so far
meeting_speak({ "meeting_id": "…", "text": "…", "lang": "en" })    // voice, backend ("supertonic" | "elevenlabs"), model,
                                                                   // audio_file_id, auto_mute (true), interrupt, wait
speech_settings_get({})   speech_settings_set({ "backend": "elevenlabs", "elevenlabs_voice": "…", "vad_silence_s": 0.5 })
meeting_stop_speaking({ "meeting_id": "…" })   meeting_mute({ "meeting_id": "…", "on": true })   meeting_raise_hand({ "meeting_id": "…", "on": true })
meeting_draft({ "meeting_id": "…", "text": "…", "ready": true, "for_turn_seq": 7 })   // draft_enabled; text "" clears

How a meeting runs:

1. Browser: a fresh browser of its own, or profile (Chromium or patchright only; camoufox is refused).

   • Media permission prompts are accepted (--use-fake-ui-for-media-stream), but no fake devices are used: the microphone is an audio source that is silent unless Bosun speaks (meeting_speak), and there is no camera.

   • The browser asks for an English UI (Meet: hl=en; Zoom: its language cookie; both: Accept-Language en-US). The account or region can still win, so Meet's texts and labels are also recognized in German, Slovak, Ukrainian and Russian, and Zoom's in German and Ukrainian (some Russian).

   • Zoom links (zoom.us/j/<id>?pwd=…, any *.zoom.us subdomain, webinar /w/ and web client /wc/ links) are rewritten to the web client, https://app.zoom.us/wc/<id>/join?pwd=…, so Zoom's "open the app" page doesn't come up. If it does anyway, Bosun clicks "Launch Meeting" and then "Join from your browser".

2. Lobby:

   • Microphone and camera must be verifiably off before joining. Bosun switches a device off and checks it, unless Meet shows the device doesn't exist ("Camera not found" / "Kamera nicht gefunden") and no control shows it on; then it already counts as off. If neither can be confirmed, Bosun does not join.

   • A guest types its name (default Glassy (notetaker)). A signed-in Meet profile has no name field and joins under the account's name, so name is ignored. Zoom fills in a signed-in account's name, and Bosun keeps it; an empty field (Zoom's web client shows one to signed-in accounts too) gets name.

   • Zoom: "Do you want people to see you in the meeting?" and then "Do you want people to hear you in the meeting?" are answered "Continue without …"; "Use microphone" is never clicked. Without permission, Zoom shows "Mute" with a crossed microphone, and clicking it asks "hear you" again instead of muting. So on Zoom a device also counts as off if the browser lists none, its permission is denied, or clicking its control made Zoom ask for it (answered without it). A control that takes clicks and neither turns off nor asks is a live device: no join. Bosun asks zoom.us/profile whether the profile is still signed in; a signed-out profile joins as a guest, and meeting_status carries a warning saying how to sign it in again.

   • Tips and dialogs over the lobby ("Do you want people to hear you in the meeting?", "Got it", cookie banners, "This meeting is being recorded") are closed. Only buttons inside a dialog or banner are clicked.

   • Then Ask to join / Join now / Teilnahme erbitten (Meet) or Join (Zoom). Meet names the button after the devices it joins without ("Ask to join without microphone & camera"); its visible text is enough too. A join button that is still disabled ("Getting ready...") is waited for, up to 25 s. Bosun then waits up to max_wait_admit_min for admission. meeting_status shows joined_as: account or guest.

   • Meet, account already in the call on another device: Meet offers "Switch here", which would drop the other device. Bosun picks "Join here too" (directly, or under "Other ways to join") and uses "Switch here" only when Meet offers nothing else.

3. Not getting in:

   • A denied request or no admission in time ends the meeting with a clear error.

   • Google turning the guest away ("You can't join this video call" / "Sie können nicht an diesem Videoanruf teilnehmen" after asking, with no waiting room seen) ends with reason refused_by_google.

   • That refusal comes with a hint: join with a signed-in profile, or from a residential IP. It is not a captcha, and navigate reports it as a refusal too.

   • Zoom ends a join early with invalid-meeting (wrong meeting ID or passcode, or a link without ?pwd= for a meeting that has one, or "This meeting link is invalid (3,001)"; the link's pwd is an encrypted token and is never typed into a passcode field), sign-in-required (the host admits only signed-in users) or captcha.

   • Captcha means a challenge that is shown: a reCAPTCHA / hCaptcha / Turnstile frame that is visible, on screen and at least 60×60 px, and not reCAPTCHA's invisible badge. Meet and Zoom's web client are "protected by reCAPTCHA" on every join; that alone is not a captcha, for meetings or for navigate. Being removed from Zoom's waiting room counts as denied; Meet's "No one responded to your request" too.

   • dry_run: true loads the lobby exactly as a join would (profile, engine, devices) and reports what Bosun sees there, without joining: lobby kind, the button it would click, microphone and camera, a shown challenge, and for Zoom whether the profile is signed in. Use it to check a link or a profile before a meeting. docs/MEETING-BOT-REFERENCES.md compares the flows with other open-source bots.

4. Once admitted (Zoom: computer audio joined first, since without it nothing is heard, then microphone and camera checked off again), Bosun opens the chat and posts the announcement first, then starts recording. Who is told depends on announce:

   • "chat" (default): everyone. On Zoom the chat is set to Everyone (or "Meeting Group Chat", a meeting with continuous meeting chat), and the announcement has to show up in the chat, not marked as private. If the chat can't reach everyone (Zoom allows chat with the host only, Meet's chat is turned off, the message doesn't show up), Bosun tells the host instead (Zoom's "Host and co-hosts"; Meet has no chat to the host alone). If that fails too, Bosun records anyway: meeting_status shows announce.sent_to: "failed". Anything short of everyone sends bosun:meeting-announce with sent_to, reason and every attempt's error.

   • "host": the host only, on Zoom.

   • "none": nobody. Recording starts without any message. Use it only when the meeting owner told you so: the caller is then responsible for everyone knowing they are recorded.

   In every mode the chat is watched and an objection stops and deletes the recording. announce.reason says why the announcement didn't reach everyone: chat-restricted (the chat can't be addressed to everyone), chat-unavailable (no chat), sent-privately, not-sent (still in the box), not-confirmed (it never showed up as Bosun's), and for the host host-unreachable. Later messages (meeting_chat_send, the opt-out confirmation) go where the announcement went. The default text is in English and German: "Hi, I'm Glassy, an AI notetaker. I'm recording audio for notes. If you'd rather not be recorded, just say so here in chat (e.g. 'please don't record') and I'll stop and delete the recording." With MEETING_OWNER set it says whose notetaker it is.

5. While in the call:

   • Every chat message from someone else is pushed to the agent as bosun:meeting-chat, with meeting_id, sender and text. meeting_chat_read returns them too.

   • Bosun leaves by itself when the meeting ends, when it is removed, or when it is alone. That is 30 s after everyone else left (alone_leave_sec), or 2 minutes if nobody else has joined yet.

   • The agent can leave with meeting_leave.

   • On leaving, bosun:meeting-ended carries meeting_id, reason, duration_s, bytes and the recording: path (inside the browser container), size and the hint call recording_get { meeting_id }. There is no file:// link: an agent host refuses it, and nothing outside the container can open it. meeting_status (ended) has the same path and hint.

Signed-in profile. Google turns away anonymous automated guests far more often than signed-in users, and Zoom may turn anonymous browser guests away (it admitted them in our tests, but a profile is the dependable way in).

• Setup: session_open({ "profile": "google" }), navigate to https://accounts.google.com (Zoom: https://zoom.us/signin), request_human so a person signs in, session_close.

• Joining: meeting_join({ "url": "…", "profile": "google" }) or { …, "profile": "zoom" }.

• Engine: the profile is opened with the engine it was last used with. It can't be in use by an agent session at the same time.

• Keep it signed in. Both sign-ins expire when nobody uses them. Give each profile a keepalive, for example keepalive_set({ "profile": "google", "url": "https://myaccount.google.com/", "every_hours": 24 }) and keepalive_set({ "profile": "zoom", "url": "https://zoom.us/profile", "every_hours": 24 }), and look at keepalive_list before an important meeting. A logged_out result means a person has to sign in again.

• Keepalive and meetings share the profile. A meeting that starts while a keepalive visit has the profile open takes it over, and a visit that comes due during a meeting is skipped and runs at its next slot. With MAX_SESSIONS=2, a meeting and one agent session fill the service, so a keepalive visit then waits too.

The meeting runs in a session of its own, so it outlives the agent's connection. It counts against MAX_SESSIONS. Events use the MCPL feature set bosun.meeting (tags bosun:meeting-chat, bosun:meeting-optout, bosun:meeting-announce, bosun:meeting-ended, and with 0.15 bosun:meeting-transcript, bosun:meeting-transcript-stopped, bosun:meeting-speech); host recipes that list enabledFeatureSets need to add it (or bosun.*).

Consent. Recording starts only after the announcement was tried (or, with announce: "none", without one). No slash commands are needed: a plain-language request not to be recorded is enough, in English, German, Ukrainian, Russian or Slovak (please don't record, stop recording, bitte nicht aufnehmen, не записуй, не записывай, nenahrávaj, …).

• What happens: Bosun stops the recording and deletes the file at once, posts "Recording stopped and deleted. / Aufnahme gestoppt und gelöscht." in the chat, sends bosun:meeting-optout and leaves. With leave_on_optout: false it stays but never records again in that meeting.

• The matcher is conservative. "don't stop recording", "keep recording" or "record the next steps" do not trigger it, and neither do instructions about opting out ("if you'd rather not…", "to opt out, write 'stop recording'"): quoted words in a longer message are a mention. Bosun's own messages never count. A message that contains something Bosun posted is its own, however the chat panel labels it.

• Because it can miss phrasings, every chat message also reaches the agent, which can call meeting_stop_recording itself.

Recordings: BROWSER_DATA/recordings/<tenant>/<meeting_id>.ogg, mode 0600, Ogg/Opus 16 kHz mono 24 kbps (about 11 MB per hour). They are deleted after RECORDING_RETENTION_DAYS (default 30), or with meeting_recording_delete. The agent gets one with recording_get (see Mode 1), after the meeting ended or the recording was stopped without delete. A deletion leaves an empty marker <meeting_id>.deleted beside it, for the same retention period. Chat text is kept only in memory, for the last 50 meetings.

Live transcript and speaking

Live transcript (meeting_join { live_transcript: true }, off by default; needs ELEVENLABS_API_KEY):

• What it is: the meeting audio Bosun records is also read by a second ffmpeg as raw PCM (16 kHz mono) and streamed to ElevenLabs Scribe v2 Realtime: wss://api.elevenlabs.io/v1/speech-to-text/realtime, model scribe_v2_realtime, audio_format=pcm_16000, commit_strategy=vad (a pause of vad_silence_s ends a segment: meeting_join's, else the tenant's speech setting, else LIVE_TRANSCRIPT_VAD_SILENCE_S, else 0.5 s), word timestamps and language detection on, no language set (Scribe detects it per segment; a uk/ru/en/de/sk mix is fine).

• The pause that ends a segment (vad_silence_s, 0.2–3 s; meeting_status → live_transcript.vad_silence_s): measured with real Scribe on English and Ukrainian speech (2026-10-28), 0.5 s brings a final 0.85–0.97 s after the speaker stops, 1.0 s 1.44–1.49 s, 0.7 s 1.06–1.11 s. Fluent sentences stay whole at 0.5 s; a pause of about 0.5 s or more inside a sentence (someone thinking) ends a segment there, so a final may be half a sentence (no words are lost; the next final carries the rest). Before answering on a final, check that it reads as finished, or use 0.7 s.

• Events: bosun:meeting-transcript with meeting_id, seq, start_s, end_s (seconds since the transcript started), text, final, lang, and speaker when the page showed who was speaking (see Speaker labels). While someone speaks, partials (final: false, at most two a second, each carrying the seq its final will have); then the final, always. transcript_partials: false sends finals only. These events go to the connections there at the time and are never queued (they would crowd out chat and opt-out events); meeting_transcript { since } returns every final so far. With the meeting channel (below) taking the text, these events are not sent too (speech setting transcript_events: auto by default, always, never).

• Speaker labels (best effort; a guest join is enough). Every 300 ms Bosun reads who the page shows speaking.

  • Zoom: the speaker view's active tile, or a gallery tile marked active, with the name in its footer.

  • Meet: a tile with a non-zero data-audio-level or Meet's speaking-animation class, with the name in span.notranslate.

  • Bosun's own tile never counts.

  • A segment's speaker is whoever was shown speaking longest while it was said, from 0.3 s before it. speakers lists them all when there were several.

  • Limits:

    • Zoom's gallery view without an active mark, and a screen shared full size, show nobody.

    • Meet renames its classes from time to time, so the label then goes missing (never wrong names: the name is read from the tile that carries the mark).

    • Two people talking over each other get the one shown longer.

    • The page's highlight lags the voice by a few hundred ms.

  • meeting_status.speaker_labels says whether a speaker has been seen at all (seen / none-seen).

• Consent: it starts and stops with the recording: only after the announcement, which then says "This call is recorded and transcribed live for notes …" (German too). An opt-out in the chat stops both at once, deletes the recording and forgets the transcript's text, and posts "Recording and live transcript stopped and deleted." meeting_stop_recording stops it too (delete: true forgets its text), and so does meeting_recording_delete.

• Retention at ElevenLabs: Bosun always asks for zero retention (enable_logging=false). ElevenLabs applies it only on enterprise and trial plans; otherwise it answers with a warning and logs the session. meeting_status shows live_transcript.zero_retention (false on a Pro plan) and the warning.

• Cost and cap: $0.39 per hour of audio (about $0.0065 a minute). LIVE_TRANSCRIPT_DAILY_MIN (180) per tenant and day: a join is refused once it is used up, and a running stream stops with bosun:meeting-transcript-stopped (reason: daily-cap) while the meeting and the recording go on. The count survives restarts (BROWSER_DATA/live-transcript-usage.json).

• NODE_DEBUG: while it is set, the key is not sent (Node's diagnostics print request headers): live_transcript and the elevenlabs voice are refused, as secrets are.

• Drops: a dropped connection is opened again (with growing pauses, 8 tries in a row at most); up to 20 s of audio meanwhile is kept and sent once it's back, with the last words as context. A wrong key, a used-up quota or unaccepted terms stop it (reason: error).

• meeting_status → live_transcript { state, zero_retention, segments, streamed_min, today_min, daily_cap_min, final_lag_s }.

Speaking (meeting_speak, once admitted and after the announcement):

• What is played: text, synthesized by backend supertonic (the local TTS service at TTS_URL, voices supertonic-F1..F5 / M1..M5, pass lang) or elevenlabs (voice = an ElevenLabs voice id, model = an ElevenLabs model id, default eleven_v4_turbo; zero retention asked for). Or an uploaded file (audio_file_id from upload_create, any format ffmpeg reads).

• Speech settings (speech_settings_get, speech_settings_set; per tenant, kept in BROWSER_DATA/speech-settings/<tenant>.json): backend, elevenlabs_model, elevenlabs_voice, supertonic_voice, vad_silence_s, checked like the env values; null removes one. What a call leaves out comes from the tenant's setting, then the env fallback, then Bosun's default; speech_settings_get shows the effective values and where each comes from (tenant, env, default). backend elevenlabs without ELEVENLABS_API_KEY falls back to supertonic (with a warning). A read-only (ephemeral) token may read them, not set them.

• How: ffmpeg plays it into the meeting's bosun_mic sink, whose monitor is the browser's microphone. The join still starts muted; meeting_speak unmutes for the utterance and mutes again 0.4 s after it ends (auto_mute, default true; a microphone that was already on stays on). It returns once the audio plays, with duration_s, prepare_ms (synthesis, with the unmute alongside), start_ms and, for text in several pieces, gap_ms (the longest pause between pieces because synthesis fell behind; 0 when each piece was there before the one before it had played); wait: true returns once it has played. bosun:meeting-speech says when an utterance ended (done, interrupted, failed).

• Muting again: each click on the microphone control first moves the mouse over the toolbar (Zoom and Meet fade it out while the mouse is still; a faded "Mute" still counts as shown, and a click on it lands behind it). Three tries, an open menu closed in between; the utterance's end waits at most 20 s for them. A microphone still on then is reported (mute_error in the result, the event and meeting_status, plus mic_open: true in meeting_status) and tried again every 5 s (6 times) while nobody speaks; the next utterance or meeting_mute tries too.

• Muting again on real Zoom (0.17.0). On 2026-10-28 production said "no microphone control found in Zoom" after a 2.5 s utterance, and the microphone was left on. When the toolbar's button can't be worked, Bosun now:

  1. reads the control from the page itself, including a hidden one or a <div>, but only in the toolbar, and clicks it there;

  2. then sends Alt+A (first in the page, in the same step that reads the state; then for real, right after reading it again), but only while the page shows the microphone still on.

  Every step is checked by reading the state back. If nothing works: mic_open: true and a mute_error, a bosun:meeting-speech event starting "WARNING: Bosun's microphone may still be ON", and a queued bosun:meeting-mic-open event (again with gave_up when the retries are over). meeting_mute { on: true } or meeting_leave if it must be off. The microphone is bosun_mic_src, which carries only what Bosun plays, so a microphone left on sends silence, never the room.

• Barge-in: meeting_stop_speaking stops it at once (and mutes again). A second meeting_speak while one plays is refused unless interrupt: true, which replaces it.

• meeting_mute { on } and meeting_raise_hand { on } on Meet (the toolbar's microphone and hand buttons, whose names say their state) and Zoom (Mute/Unmute; the hand in the Reactions menu). On Zoom a profile that never gave Zoom the microphone can't unmute: meeting_speak then says so.

• Latency: from the call to the first sound in the call about 0.35 s plus the synthesis of the first sentence (Supertonic: about 1–1.3 s for a short one on a busy CPU; ElevenLabs eleven_v4_turbo: 0.3–0.6 s, and the next pieces arrive long before the one before has played, so gap_ms is 0). Start with a short sentence for a quick answer.

• Engines: chromium and patchright (meetings refuse camoufox).

The meeting as an MCPL channel (0.17.0). With a host that grants MCPL channels (connectome-host grants what a server advertises unless the recipe disables it), each meeting Bosun is in becomes a channel bosun:meeting:<id> once the announcement is out.

• Lifecycle: registered with channels/register (bidirectional, initiallyOpen) on every connection of the tenant, and removed with channels/changed when the meeting ends. The speech setting meeting_channel: false turns it off.

• Incoming: transcript finals arrive as channels/incoming:

  • messageId <meeting>:<seq>, threadId the meeting.

  • author is the speaker when known, else "Meeting".

  • metadata {meeting_id, platform, seq, start_s, end_s, final, lang, speaker, speakers}, tags bosun:meeting-transcript and bosun:meeting=<id>.

  • With agent_partials: true, partial text comes too, tagged bosun:partial, at most every agent_partials_interval_s (1.5 s), so the agent can prepare.

• Speaking: text the host publishes to the channel (channels/publish) is acknowledged at once ({delivered: true, messageId}) and spoken right away, sentence by sentence (the first one cut at a comma when it is long).

  • Text published while it plays is added to the same utterance, and the microphone stays on.

  • Text published while meeting_speak plays follows it.

  • bosun:meeting-speech reports the end, with kind: "stream".

  • In connectome-host an agent publishes with the channel_publish tool. Short pieces (a first sentence, then the rest) make the first audio come soonest.

• Limits: a publish of more than 2000 characters, or with more than 6000 characters (or 120 s of audio) from the channel still waiting, is refused before it is acknowledged.

• Stopping: meeting_stop_speaking also drops channel text still waiting. An opt-out with leave_on_optout: false closes the channel, so no transcript and no automatic speech; meeting_speak still works.

• Chunks: channels/outgoing/chunk / complete are accepted but never spoken (MCPL §14.5: only a publish delivers). With channel_prepare_chunks: true, each sentence is synthesized as soon as the stream completes it, and the publish that delivers it plays that audio without waiting for synthesis. This helps only with a host that streams before it publishes: connectome-host sends chunks after the publish, and Bosun doesn't synthesize those again.

• Security: only channels registered on that connection, for that tenant, under the grant as it is now. Anything else gets -32023 (unknown channel) or -32002 (capability denied).

• Acknowledgement (ack_on_final, off by default). When a final ends someone's turn (at least ack_min_words words, ending a sentence), Bosun plays a short clip after ack_delay_ms (100):

  • The clip is one of ack_phrases, synthesized once per voice when Bosun is let in, in ack_lang.

  • Nothing is played while something is being said, if the agent starts speaking first, or within ack_cooldown_s (8) of the last one.

  • The agent's reply waits for a clip that is playing (it lasts about half a second) instead of cutting it, and the microphone stays on in between.

  • Measured locally (stand-ins): the clip's audio starts about 0.4 s after the final reached the host, most of it the unmute click.

Turns, barge-in and stale answers (0.17.1). At a 0.5 s pause, Scribe's VAD ends a segment in the middle of a sentence. In 0.17.0 the agent answered that fragment, and what the speaker added came in while it spoke.

• Turn hold: a final is held turn_hold_ms (default 900) for more speech. When the speaker goes on, their partials keep the turn open until their next final, and the agent gets one merged final: metadata.merged_seqs lists the segments, seq is the last one. held_ms says how long the turn waited after its last final.

  • A question, or a complete sentence (three or more words, a full stop, not trailing off with "…" or a word such as "and", "the", "що", "und"), is held only turn_hold_complete_ms (default 300).

  • Someone else's final ends the turn held before it. A turn goes at the latest at the first final 30 s after it began.

  • turn_hold_ms: 0 turns the hold off, as in 0.17.0.

  • The acknowledgement (ack_on_final) plays once, at the real end. meeting_transcript still has every segment.

• Barge-in (barge_in, default true): someone else speaking while Bosun's audio plays stops it, as meeting_stop_speaking would.

  • What counts: at least barge_in_min_words (2) new words, where agreeing words ("mhm", "yeah", "ok", "так", "ja") don't count; or one word while the page shows someone speaking for barge_in_min_s (0.6) since Bosun began.

  • What happens: the microphone is muted again, and text from the channel still waiting is dropped.

  • An uploaded audio file (audio_file_id) is never cut off this way, because there is no text to recognise its echo by.

  • What the agent hears: bosun:meeting-speech comes with state: "interrupted", barge_in: true, said (the text played so far) and interrupted_by. The channel gets a message tagged bosun:meeting-interrupted, author bosun, (interrupted at "..."), with said and unsaid in its metadata. The interrupter's turn follows as usual.

• Bosun's own voice never interrupts it.

  • Text that is mostly words Bosun is saying, or said in the last 8 s, is its own voice coming back through someone's speakers. It never barges in, never extends a turn, and its final is marked echo_suspected.

  • Where the page names speakers, someone other than Bosun must be shown speaking while Bosun's audio plays.

• Stale answers: the acknowledgement of each channels/publish carries superseded_by_new_speech. It is true when someone has spoken since the last turn the agent was given (a turn still held, or still being said). The text is said all the same, and the agent decides whether to stop (meeting_stop_speaking) or go on. last_turn_seq is the seq of that last turn, so an agent can tell when it is answering an older one.

Still thinking (0.17.1). When the speaker goes on after their turn went to the agent but before the agent answered it (no publish and no meeting_speak for it yet), within turn_extend_s (default 20 s; 0 turns this off):

• At once: the channel gets a message tagged bosun:meeting-turn-extended, author bosun, metadata {turn_seq, turn_text, text_so_far, speaker}. Without the channel it comes as the live event bosun:meeting-turn-extended.

• The turn is reopened: it is held again with what the speaker adds. When they are done it comes again as one merged final: extends_turn names the old turn, and merged_seqs covers both parts.

• Publishes for the old turn: with superseded_publish: "drop" (default) they are not said. The acknowledgement is delivered: false, dropped: true, reason: "superseded_by_new_speech", so a host counts it as not delivered. With "speak" they are said, with superseded_by_new_speech: true.

• Which turn a publish answers: the one current when it arrived (turn_seq in the acknowledgement). Finals carry turn_seq, and a publish may pass turn_seq (in its params or metadata) to name the turn it answers.

• Acknowledgement: an acknowledgement still waiting for the old turn (ack_delay_ms) is not played, and the merged turn gets its own. A clip that already played can't be taken back, so the merged turn's clip then waits out ack_cooldown_s as usual.

• Not reopened: a turn the agent has answered (its answer may be playing, and barge-in handles that), a turn of someone else, and a turn whose transcript was deleted.

Drafts (0.18.0, draft_enabled). The agent can prepare its answer while the speaker still talks (with agent_partials it sees the turn as it is said). meeting_draft {meeting_id, text, ready, for_turn_seq} sets the draft, and text: "" clears it.

• Ready: the first sentence is synthesized now. When the turn reaches the agent (after the turn hold), Bosun says it at once, so no synthesis time is spent then.

• Dropped when the final added more than draft_max_new_words (4) words to what was heard when the draft was marked ready, or a "not"/"but" among them, or the draft was for another turn, not ready, or something else is being said. A barge-in or a reopened turn drops the draft too.

• The final says what happened: metadata.draft is spoken (with draft_text), dropped (with draft_reason) or none, and a second text block says it in words.

• The publish: one for that turn that starts with the sentence skips it (draft_prefix_skipped: true). One that starts otherwise is said in full.

• Measured locally (stand-ins): the sentence's audio starts about 0.4 s after the final reached the host, nearly all of it the unmute click.

Speculative speech (0.18.0, speculative_speech). The meeting channel declares streaming: {speculative: true}. A host that supports it streams the content of the agent's channel_publish while the model writes it, and Bosun speaks each sentence as soon as it is complete. complete with cancelled stops it at once, and the publish that follows is not said twice. The contract is in docs/STREAMING.md. Measured locally: the first audio starts about 0.4 s after the chunk that completes the first sentence.

Audio setup (Linux). Each meeting gets a private PulseAudio server: its own XDG_RUNTIME_DIR, socket and config, with two null sinks:

• bosun_out: the browser plays the meeting into it; ffmpeg records its monitor, and a second one reads it for the live transcript.

• bosun_mic: its monitor is the browser's microphone, silent unless Bosun speaks.

Meetings never hear each other or the host's sound card. The image needs:

apt-get install -y pulseaudio pulseaudio-utils ffmpeg

Each meeting starts its own pulseaudio process, so PipeWire alone (pipewire-pulse) isn't enough. Both can be installed side by side: the private servers don't touch the system one.

Meetings need a headful browser (headless Chromium plays no audio), so run the service under Xvfb without HEADLESS=1. On macOS or without these packages, meeting_join refuses with the reason. docker/meeting-test.Dockerfile builds an image that runs the audio test: docker run --rm --init -v "$PWD":/app -w /app bosun-meeting-test xvfb-run -a node --test test/meeting-audio.test.mjs.

Limits — what this can't do (yet):

• Guests need a host. Someone in the meeting has to admit the bot. Meetings that don't allow guests, or that only admit people from the organisation, can't be joined.

• Google reviews bot-like guests. Join requests from automated browsers can land in a review queue or be refused outright ("You can't join this video call", reason refused_by_google). Datacenter IPs make this much more likely. A signed-in profile or a residential connection works best.

• Meet's and Zoom's markup changes without notice. Labels and texts are matched in several languages (English is requested), and the chat sender name is a best-effort guess. A change on Google's or Zoom's side breaks the join or the chat watch until platforms/meet.mjs or platforms/zoom.mjs is updated.

• Tested against stand-ins. The tests run against local stand-ins (test/fakemeet.mjs, test/fakezoom.mjs) built from how the real pages look. The signed-in Meet lobby (dialogs, "Getting ready...", "Join here too") and all of Zoom are new in 0.9.0 and have not had a live run yet. The Zoom chat watch depends most on Zoom's markup and is the part most likely to need a fix after the first live meeting.

• Zoom may connect computer audio with the microphone on. Bosun mutes it as soon as the control appears. The browser's microphone is a silent source, so only silence can go out in that moment, and the server has no camera.

• An objection in the last second before Zoom leaves the page can be missed. Bosun reads the chat once more when a meeting ends, but if Zoom has already navigated to its end page, messages it hadn't read yet are gone. The recording is then kept; the agent can still delete it with meeting_recording_delete.

• One mixed audio track. There is no speaker separation, also not in the live transcript.

• Bosun doesn't hear itself. The meeting doesn't play Bosun's own voice back, so the live transcript has only the others; what Bosun said is what the agent passed to meeting_speak.

• Consent is the operator's responsibility. The announcement and the opt-out are a courtesy, not a legal basis. Check what applies where the participants are; some places require everyone's explicit consent before recording.

Read-only tokens for subagents

Give helper agents (for example claude -p subprocesses) a browser without your logins, keepalive, meetings or handoff: a tenant of their own with policy ephemeral.

BROWSER_TOKENS=me:<token>,swarm:<other token>
BROWSER_TOKEN_POLICIES=swarm=ephemeral       # tenants not named stay "full"
BROWSER_EPHEMERAL_MAX_SESSIONS=2             # default

With policy ephemeral the server enforces, not just hides:

• tools/list shows only session_open, session_close, navigate, read, click, type, select, press, back, wait_for, screenshot, sessions, macro_list, macro_get and macro_run (a macro's steps hold to the same set; its macros are those of BROWSER_MACRO_SOURCE, see Macros). Any other tool answers not allowed for this token, whatever its arguments; its handler is never wired up for the connection.

• session_open refuses profile ("profiles are not allowed for this token") and any engine but the default (ENGINE, which startup checked, so a session never installs a browser). The session manager enforces both for the tenant as well, and refuses meeting browsers.

• The tenant has BROWSER_EPHEMERAL_MAX_SESSIONS sessions at most, across all its connections. Sessions belong to the connection and close when it closes (or after IDLE_TIMEOUT_SEC).

• Nothing is written to BROWSER_DATA: screenshots are returned but never saved, even with SAVE_SHOTS=1. Ephemeral sessions use temporary browser profiles that are deleted on close, and their browsers refuse downloads (a read-only tenant can't be given download hosts). upload_create and upload are refused, and so is an upload PUT that carries this token.

• No MCPL push events.

A policy only narrows what the token can do; the tenant is still separate from yours, so it never sees your profiles anyway. Egress is not narrowed: block private networks around the service (see Security notes).

stdio bridge

MCP clients that only start stdio servers (such as claude --mcp-config) use bosun-mcp-stdio (this package's second bin). It reads BOSUN_URL (ws:// or wss://) and BOSUN_TOKEN, sends the token as Authorization: Bearer (never in the URL, never printed), and pipes newline-delimited JSON-RPC between stdin/stdout and the WebSocket, one message per frame.

{ "mcpServers": { "bosun": { "command": "bosun-mcp-stdio",
  "env": { "BOSUN_URL": "ws://127.0.0.1:7360", "BOSUN_TOKEN": "<swarm token>" } } } }

Records are split on LF only (CRLF is accepted); a record that is not JSON or is over 2 MiB is dropped with a line on stderr. The opening handshake times out after 15 s.

The bridge ends when its client is gone, also when its stdin stays open because another process holds it:

• stdin ends or closes, or stdout is closed (nobody reads the answers);

• its parent process exits (checked every second; Node has no PR_SET_PDEATHSIG);

• BOSUN_IDLE_EXIT_SEC (3600; 0 = never) pass with no message either way, no request waiting for its answer (the client's, such as a long handoff_wait, or the server's) and no answer still being written to stdout.

Exit codes: 0 for those, 1 connection failed or lost, 2 bad configuration (missing variable, a non-ws URL, credentials in the URL, a bad BOSUN_IDLE_EXIT_SEC), 3 token refused. The bridge needs only Node and the ws package.

Chromium sandbox

The browser parses untrusted web content, so it runs with Chromium's own sandbox (chromiumSandbox: true, which is what CHROMIUM_SANDBOX=1, the default, means). Playwright's default is --no-sandbox; Bosun does not use it (0.7.0 and earlier did).

The sandbox needs something from the host, and startup checks it: Bosun starts the browser once, closes it again, and refuses to run if that fails, printing what to do. On Linux, one of these has to be true:

1. unprivileged user namespaces are allowed — echo 'kernel.apparmor_restrict_unprivileged_userns=0' | sudo tee /etc/sysctl.d/60-apparmor-userns.conf && sudo sysctl --system (Ubuntu 23.10+ restricts them through AppArmor), or

2. an AppArmor profile grants userns to the browser binary, or

3. Chromium's setuid helper next to the browser is root-owned and mode 4755, with CHROME_DEVEL_SANDBOX pointing at it — and then the service must not run with NoNewPrivileges.

docs/DEPLOY-SYSTEMD.md has the commands for each. A container or CI box usually allows none of them; there, CHROMIUM_SANDBOX=0 is the explicit way out, and the service says so on every start:

bosun: WARNING — CHROMIUM_SANDBOX=0: the browser runs without its own sandbox, …

Check what is actually running, not just the setting: ps -eo args | grep -- --type=renderer must show no --no-sandbox.

Per-tenant host allowlist

Keep a tenant to the hosts it has business with — so a page that says "now open this other URL" gets nowhere, and neither does a redirect, an iframe, a fetch, a WebSocket or anything a worker starts:

BROWSER_TENANT_ALLOW='ops=*.example.org,example.org,127.0.0.1:4848; swarm=*.example.org'

Entries are separated by ; (or newlines), hosts within an entry by ,. A host pattern is:

How it is enforced

A tenant that is named gets its own forward proxy inside the service (src/proxy.mjs, on loopback, with a random password of its own), and each of its sessions runs in a browser of its own (a temporary profile, removed with the session), launched with that proxy as its only way out (--proxy-bypass-list=<-loopback>, so not even loopback is exempt; QUIC off and WebRTC's non-proxied UDP off, so nothing can go around it; Chromium does not fall back to a direct connection when a proxy refuses). Full Chromium (headful, patchright) ignores the WebRTC flag, so the browser's profile also gets the webrtc.ip_handling_policy preference before every proxied launch (a persistent profile loses it again on an unproxied launch, if Bosun set it). camoufox gets the same as Firefox preferences: loopback through the proxy too (network.proxy.allow_hijacking_localhost), WebRTC through the proxy only, no HTTP/3, no DNS prefetching. Those settings belong to that browser alone — tenants without an allowlist share a browser launched exactly the way upstream launches it, so one tenant's allowlist never changes another tenant's networking. Every connection the browser wants — a navigation, each hop of a redirect, a subresource, XHR, a WebSocket from the page or from a worker, a service-worker fetch, a prefetch — arrives at the proxy as a request or a CONNECT and is checked against the allowlist before a byte leaves the host:

• A denied connection is never made. The browser gets 403 from the proxy, so a denied navigation leaves the tab on that URL with the proxy's one-line reason and navigate reports status: 403 and refused: "host allowlist". An https:// hop is refused at CONNECT, which fails the navigation in the browser; navigate then answers "navigation refused by the host allowlist of this token: :".

• Denials are logged as bosun: tenant <t>: blocked <host>:<port> — <reason>: host and port, never a path or a query. One line per target per tenant (a repeat is counted, not logged again).

• DNS is resolved by the proxy, once, and the connection goes to that address, so the address a decision was made about is the address that gets used — a name cannot change where it points between the check and the connection. A name that resolves into loopback or a private range is only followed if the allowlist names that address as well, so 127.0.0.1:4848 has to be in the list for a name that resolves there.

• Nothing is intercepted. The proxy tunnels TLS (CONNECT) and streams plain HTTP; it never terminates TLS, rewrites a response or buffers a body. The browser's own rules therefore all still hold: a redirect to another allowed host loads as that host, with its own origin, storage and cookies, and the tab shows the URL it really ended up on.

• navigate and keepalive_set also check the allowlist up front, so the agent gets a clear refusal instead of a 403 page, and navigate's tool description names the hosts it may reach.

A tenant that is not named is unrestricted: no proxy, nothing intercepted, and a browser with upstream's own flags — exactly upstream behaviour, whatever another tenant's allowlist says.

What it does not do

• The allowlist is host and port, and a CONNECT tunnel is not inspected, so an entry without a port allows a tunnel to any port on that host. Prefer host:port entries.

• It is not a URL filter: a path or a query is never looked at (and never logged).

• Hosts you allow can talk to each other with the browser's normal rules; the allowlist decides reachability, not mutual trust. Keep it to the hosts a tenant needs.

• Plain http:// goes through the proxy as HTTP/1.1 (no h2c, no QUIC); https:// is end-to-end inside the tunnel.

• A firewall is still the outer boundary: the proxy is a per-tenant rule inside the service, not a replacement for network egress policy (see Security notes and docs/SECURITY-REVIEW-0.7.0.md).

Security notes

• One service is one trust domain per tenant (token): a tenant's connections share its persistent profiles; sessions are private to the connection. For hard isolation run one process (and data dir) per tenant.

• Web content is untrusted (prompt injection). Run the service where it can't reach private networks (block 10/8, 172.16/12, 192.168/16, 100.64/10, 169.254/16 egress) — otherwise a page can steer it at your LAN. The http(s)-only check on navigate does not replace that boundary. Run it as a user that can't read secrets.

• Live-view links are bearer tokens for the whole handoff (up to HANDOFF_TTL_SEC): anyone holding the link can open the live view whenever nobody else is connected, or 5 s after the connected viewer went silent. Serve them on a private network (VPN/mesh) or behind TLS, send them over a private channel, and press Done or Give up when finished — that revokes the link for good (no replay afterwards). A connected, responsive viewer can't be pushed out by a second one.

• No rate limiting yet beyond the connection, in-flight and session caps.

• A named tenant can be kept to a list of hosts, enforced by a per-tenant forward proxy that is the browser's only way out (see Per-tenant host allowlist). That narrows what the agent, the page, its workers and its service workers can reach; it does not replace the egress boundary above.

• The browser runs with its own sandbox (see Chromium sandbox). Without it (0.7.0 and earlier, or CHROMIUM_SANDBOX=0) Chromium runs with --no-sandbox: then a renderer compromise is directly on the service user, which is where the profiles and the service's environment (tokens included) are.

• Downloads are off unless a tenant is named in BROWSER_TENANT_DOWNLOAD_ALLOW, and then kept only from its hosts (or pages on them), streamed to the data volume and cancelled past a per-file cap and a per-tenant quota, so a page cannot fill the volume or RAM. Kept files leave once fetched, after an hour and at every start.

• Uploads are written to BROWSER_DATA/uploads (on the data volume, not a RAM-backed /tmp) under a per-file cap and a per-tenant quota, and deleted after use, after an hour and at every start. Once a file is in a page, the page can send it anywhere it can reach: upload only what you would hand that site.

• Secrets sent for type { secret_id } stay in the service's memory (at most 4 KB each, 20 per tenant, 10 min), never on disk or in its log, and are zeroed once typed. They keep the password out of the agent's transcript, not out of the page: the page that receives one can do with it what it likes.

• Credential masking around a handoff covers credential fields — their value, and values Bosun saw in them. A page that copies what the human typed into ordinary text, or into something it reads later, is not masked on any engine, and on chromium a page can also neutralize the masking helpers outright (they run in its own JavaScript world; patchright uses an isolated one, which stops that technique and nothing more). Treat the page where a credential is typed as trusted, or don't type it there. Details, probes and the standing test: docs/SECURITY-REVIEW-0.7.0.md (finding H2).

Roadmap

M1 meetings beyond the PoC (Teams, speaker separation, streaming TTS for lower speak latency), tenant quotas/audit log, mobile viewport for handoff, more push events (handoff done/timeout) · M2 vault (fill_credential), downloads as content references, watching pages outside keepalive visits · M3 recording macros from an agent's own calls (macro_record_start/stop).

License

Apache License 2.0 — see LICENSE and NOTICE.

Status: early preview (0.x). APIs, tool names and config may change without notice.