Return static platform metadata: library version + the list of SDK capabilities the glasses expose. TWO INTEGRATION PATHS — decide which one BEFORE scaffolding. Voice apps (assistant, transcriptions, recordDiscrete, speak, audio streams) reach the glasses through the phone's own Bluetooth audio routing: NO vendor SDK, NO vendor credentials, NO connection page, NO pairing flow — and the complete agent runtime (turn-taking, barge-in, tool calling, local + cloud models) works there. Only CAMERA and DISPLAY require a vendor integration. If the developer hasn't said they need camera or display, they are on the voice path — do NOT send them through Meta account setup, a Developer Center registration, or a GitHub PAT they don't need. Phase 4 assistant runtime (glasses.assistant.start(provider) { tool(name, description) { body -> ToolResult } }) is the canonical voice-assistant API — the model owns wake/turn-taking/intent/confirmation, the customer writes tool bodies that act on app state. Lower-level primitives are still surfaced for fine-grained control (audio.transcriptions, audio.recordDiscrete, audio.speak, audio.audioChunks, camera.capturePhoto, camera.videoFrames, toggles, connection.state, …). The right first call for any new task — primes the agent on what primitives are available before writing handler code. Default response is COMPACT — capability names and categories only — keeping the typical first-call cost ~2KB. Per-feature call shape + idiom + gotchas comes from getCapabilityGuide(feature); full compositional patterns (assistant_agent_loop for the canonical glasses.assistant.start loop; voice_qa_assistant for the manual composition) from getCodeExample(pattern); conceptual docs from searchDocs(topic: 'assistant_runtime') for the assistant surface, or searchDocs(topic: 'voice_integration') for the lower-level primitives. Pass expand: ['capabilities.full'] for the machine-readable catalog with params/payload/requires/constraints. glasses is optional — defaults to meta; android_xr is also supported (preview) and meta_rayban is accepted as a legacy alias. Note the vendorless baseline needs no value here at all — voice apps run without a vendor; pass it explicitly for forward compatibility once additional vendors land. Already built on raw Meta DAT? Call getMigrationGuide — it maps your existing DAT symbols to Extentos primitives and gives an ordered cutover plan, rather than starting from scratch. USE at session start before scaffolding or writing handler code. DON'T USE for what's installed in the project (use inspectIntegration).

Reference library — retrieve a complete SDK code example (Kotlin + Swift) for a use case. For voice-assistant work on any new app, start with assistant_agent_loop — the Phase 4 glasses.assistant.start { tool(name, description) { body -> ToolResult } } API where the model owns wake detection + turn taking + intent parsing + confirmation speech and the customer only writes tool bodies that act on app state. The provider abstraction covers OpenAI Realtime, xAI Grok, and Google Gemini Live (Gemini from SDK 1.8.0+) — the model id picks the vendor. The example pairs the customer code with the agent-driven E2E loop (injectAssistantUtterance → assertToolCalled) that verifies it without humans. For the dedicated agent-side test workflow (explicit two-step wake, multi-tool sweep, four-channel verification: event log + adb logcat + screencap + library state; real-OpenAi capable since iter5.2): agent_driven_e2e_full_loop. For the glasses-display two-view navigation pattern (browse ⇄ detail on the Ray-Ban Display, Neural-Band select + assistant tools driving one state machine): display_browse_detail. Other patterns still ship for apps that need fine-grained control: voice_qa_assistant (manual glasses.voice.onPhrase wake + glasses.audio.recordDiscrete + customer-side AnthropicClient — the pre-Phase-3 composition), barge_in_speak (manual TTS cancel on user interrupt), photo_describe_voice (wake → photo → vision LLM → speak), live_transcription_ui (transcripts into Compose/SwiftUI state), voice_notes (wake → record → persist), connection_page_setup (the minimum bootstrap wiring), byok_anthropic (an Anthropic HTTP client the CUSTOMER'S OWN handler code calls with the customer's own key — used by voice_qa_assistant / photo_describe_voice. It has nothing to do with the assistant runtime, which always runs on the Extentos managed gateway with no key of any kind), agent_test_loop (the legacy three-surface verification recipe — for Phase 4 use agent_driven_e2e_full_loop). USE when you're about to write handler code and want the canonical shape to peel from, OR when you want the agent test driver that asserts the handler works end-to-end. DON'T USE for capability discovery (use getPlatformInfo) or per-feature minimal usage (use getCapabilityGuide).

Per-feature SDK usage guide — minimal Kotlin + Swift snippet + gotchas + which getCodeExample patterns exercise the feature. Pairs with getPlatformInfo (which lists feature names and categories) by adding the actual idiom for using each feature. Covers: lower-level primitives (capture_photo / capture_video / record_audio / transcription_incremental / speak / video_frames / audio_chunks / connection_state / toggles / voice_command); and the assistant runtime (assistant_runtime as the umbrella, assistant_start, assistant_tool, assistant_provider_openai) — the canonical voice-AI surface. For voice-assistant work on new apps, start with assistant_runtime for the overview then drill into the individual primitives. USE when you know which feature you need but don't know the canonical call shape, or when you're hitting a confusing failure (the gotchas typically cover it). DON'T USE for a complete compositional pattern (use getCodeExample) or for capability discovery (use getPlatformInfo).

The entry point for a developer who ALREADY built their app against raw Meta DAT (the Device Access Toolkit / Wearables SDK) and wants to move onto the Extentos SDK. Extentos's production transport calls the SAME DAT underneath, so migration is a call-site swap, not a rewrite. Returns a map KEYED BY THE DAT SYMBOL you already have (Wearables.createSession, photoDataPublisher, addStream, Display, raw CaptureError, …) → the Extentos primitive that replaces it (glasses.connection, glasses.camera.capturePhoto, glasses.camera.videoFrames, glasses.display.show, ExtentosResult, …), plus a one-line 'what changes' per area and an ordered cutover plan. It re-embeds NO code — each mapping names the Extentos feature, so you drill into getCapabilityGuide(feature) for the Kotlin/Swift idiom and getCodeExample(pattern) for a full composition. The plan orchestrates the tools that already exist: generateConnectionModule (replaces Wearables.initialize/configure + your registration UI) → getPermissions (replaces hand-mapped manifest/plist + Meta scopes) → per-call-site swaps → validateIntegration → createSimulatorSession. Scope: FULL CUTOVER, not coexistence — running raw DAT and Extentos side-by-side in one process is not supported today (the DeviceSession is a single-owner handle), and the response says so. USE when a project already imports com.meta.wearable.dat.* (Android) / MWDATCore/MWDATCamera (iOS) and you're adopting Extentos. DON'T USE for a greenfield project with no existing DAT code (start at getPlatformInfo → generateConnectionModule).

One-shot project scaffold — emits the bootstrap module that wires Extentos.create(...) into the host app, build-script changes, dependencies, permissions, and the integration manifest. Run once per project; after this lands, the agent writes its own Handler classes against the SDK primitives surfaced by getCapabilityGuide / getCodeExample. PATH IS DERIVED FROM capabilities. An audio-only footprint emits the VOICE scaffold — SDK dependency, mic permission, bootstrap; NO connection page, NO Meta account, NO App ID or Client Token, NO credentialed repository, and NO placement round trip (single call). PLATFORM NUANCE on the vendor SDK itself: on Android the voice path genuinely does not pull it (com.extentos:glasses is vendorless; only com.extentos:glasses-meta carries the DAT artifacts, which is why the split exists — Meta's Android artifacts sit behind a credentialed repo that returns 401 without a PAT). On iOS the swift-glasses package is a single umbrella and GlassesCore links MWDATCore/MWDATCamera, so the DAT SDK IS in the graph for a voice app — but Meta's iOS repo is PUBLIC, so it resolves with no account, no token and no auth. The developer-facing promise holds on both; the dependency graph differs. Splitting iOS to mirror Android is a 3.0 item (breaking for existing consumers). A footprint including camera / video / display emits the DEVICE scaffold: adds com.extentos:glasses-meta plus the Meta DAT repository, and requires the vendor credential setup. Omitting capabilities defaults to VOICE. The two-call flow below applies to the DEVICE scaffold only. Two-call flow: call once WITHOUT placement to receive status: "needs_placement" plus the question to ask the dev (where should ExtentosConnectionPage live in the app?); after the dev answers, call again WITH placement set to one of the 5 ids to receive the full scaffold (files, suggestedRendering, etc.). Existing-app detection (Android): pass projectPath so the handler reads AndroidManifest.xml and detects whether you already have an Application subclass. If yes, the emitted ExtentosBootstrap is an object (init helper) plus a manual_patch for your existing class — no clobber. If no (greenfield) OR projectPath omitted, the bootstrap is emitted as a full Application subclass and the agent sets <application android:name> accordingly. Returns files[] with action values 'create' and 'manual_patch' — see searchDocs('file_actions') for how to apply each. Pass capabilities (the SDK feature names your app uses, incl. display) so the scaffold records them in the manifest AND emits ExtentosConfig.usedCapabilities — the connection page then shows one tile per capability, lit per the connected glasses. USE as the first generation tool when scaffolding a fresh project. DON'T USE when Extentos is already installed (use inspectIntegration to read current state).

Read the per-project ExtentosConnectionPage config (theming tokens + section visibility) the dashboard/server holds for an app. Returns { managed, config }: managed=true means the dashboard/server is the SINGLE source of truth (the SDK applies server.overlay(defaults) at render time) and the committed extentos.connection-page.json should mirror it; managed=false means UNMANAGED (committed file → code ExtentosTheme → built-in defaults). Read-only. USE to inspect what the dashboard currently drives, or before regenerateConnectionPageFile. DON'T USE to change it (use setConnectionPageConfig).

Persist the per-project ExtentosConnectionPage config (theming tokens + section visibility) to the dashboard/server, making the app MANAGED — the dashboard/server becomes the single source of truth and the SDK fetches it at render time (server.overlay(defaults)). Requires a linked Extentos account (run extentos-mcp login once; a 401 returns account_required with that hint). Validated against the connection-page schema — unknown keys are surfaced as a warning (the SDK ignores them), not blocked. After writing, run regenerateConnectionPageFile to mirror it into the committed extentos.connection-page.json. USE to author/update connection-page theming from the agent; the web dashboard's Connection section is the human-facing equivalent. DON'T USE just to read (use getConnectionPageConfig).

Regenerate the committed extentos.connection-page.json FROM the dashboard/server config (file ← server) — the generated mirror that makes the dashboard's tokens real in the repo and lets the app theme correctly offline. Returns file: { path, content } for you to write (it carries a generated _comment marker — do not hand-edit; edits belong in the dashboard). If the project is UNMANAGED (no server config) there is nothing to mirror. Pass projectPath to get a drift note when an existing committed file would be overwritten. Android-only for now (iOS file-sync is Phase 4). USE after setConnectionPageConfig / dashboard edits to sync the repo. DON'T USE to push local edits up (use adoptConnectionPageFile).

Adopt the committed extentos.connection-page.json UP to the dashboard/server (file → server) — the deliberate one-time seed of an existing code/file theme into the managed config. Reads the file at projectPath, validates it, and persists it (requires a linked account — run extentos-mcp login). DRIFT-PROTECTED: if the app already has a managed server config that DIFFERS from the file, it returns status: needs_confirmation with both versions and does NOT clobber — re-call with confirm: true to let the file win (or run regenerateConnectionPageFile to let the server win). Android-only for now (iOS is Phase 4). USE to seed an existing committed theme into the dashboard. DON'T USE for routine edits (author via setConnectionPageConfig / dashboard, then regenerateConnectionPageFile).

Read the per-project "Agent" (assistant) settings the dashboard holds for an app — the OpenAI Realtime model, the voice, the memory (compaction) model, and the within-session memory mode — plus the catalog of valid options for each, the dashboard defaults, and the SDK hard defaults. Returns { managed, config, options, dashboardDefaults, sdkHardDefaults }: managed=true means a dashboard row exists and drives these at session start; managed=false means none is set, so the SDK uses its hard defaults unless the app code sets values. Runtime precedence is code-set > dashboard config > SDK hard defaults; this tool sees the dashboard layer only. Account-scoped + read-only — requires a linked Extentos account (run extentos-mcp login; a 401 returns account_required). Consumed by BOTH SDKs at assistant start (iOS re-fetches on app relaunch — no rebuild needed). USE to see what models/voice a project is configured with, or to discover the valid option ids before recommending a change. DON'T USE to change them (use setAssistantConfig; the dashboard's Agent section is the human-facing editor) or for usage/cost (use getGatewayUsage).

Change a project's dashboard-managed "Agent" (assistant) settings via MCP — any of the OpenAI Realtime model, voice, memory (compaction) model, or within-session memory mode (PARTIAL update; pass only what you're changing). Validates each value against the catalog (call getAssistantConfig first to see valid ids) and echoes the COST IMPACT of a model change (per-1M rates, old → new) so a switch is never blind. GATED by the project's MCP access grant — the default is Read+Write so it works out of the box; the owner can restrict it per-project in the dashboard, and a permission_denied (403) means Assistant-config access was set to Read or Off. Requires a linked Extentos account (a 401 returns account_required, run extentos-mcp login). Applies to the next assistant session on BOTH platforms (a value set in app code still wins; Android picks it up in a fresh process, iOS on app relaunch). USE to configure a project's voice agent from the agent loop. DON'T USE to read current settings (use getAssistantConfig) or to set secrets (credentials are never written through the agent).

List a project's named-sound library — the sounds uploaded in the dashboard's Agent section or via addProjectSound. Returns { sounds: [{ id, name, url, createdAt }] }. Devices download + register the library at assistant start; app code plays any of them with glasses.audio.playSound(name). The SDK plays none of them on its own — every sound fires because app code asked for it. GATED by the project's MCP access grant (assistant_config.sounds — default Read+Write; a permission_denied 403 means the owner restricted it). Requires a linked Extentos account (a 401 returns account_required, run extentos-mcp login). USE to see which sound names app code can play, or before adding a sound to avoid a duplicate name. DON'T USE to add sounds (use addProjectSound).

Upload a local audio file into a project's named-sound library. Pass the name app code will play it with and a path to an MP3/M4A/AAC/WAV file under 1 MB. After the next assistant start (app relaunch), every device running the app can glasses.audio.playSound(name) — and the sound appears in the dashboard's Agent section, where a human can re-point that name at a different clip with no app change. The SDK plays none of them on its own. Code registrations (audio.registerSound) win over library sounds on name collisions. GATED by the project's MCP access grant (assistant_config.sounds — default Read+Write; a permission_denied 403 means the owner restricted it). Requires a linked Extentos account (a 401 returns account_required, run extentos-mcp login). USE to provision an app's UI sounds while building it — e.g. a camera app's shutter click ({ name: "shutter", filePath: "./sounds/shutter.mp3" }). DON'T USE for the assistant's VOICE (that's setAssistantConfig) or for sounds the app should bundle locally (registerSound in app code needs no upload).

Read account-scoped managed-AI-gateway usage + exact cost for a project over a recent window. Returns { window, totals, byModel, creditBalanceUsd }: totals carry token counts, response count, and the exact list-price USD cost from the billing ledger; byModel breaks the same down per model (biggest first). METADATA ONLY — token counts and cost, never transcripts/audio/prompt content. Only managed-gateway + dashboard-vault-BYOK sessions are metered; a code-direct setOpenaiApiKey() bypass goes straight to OpenAI and is invisible here. Account-scoped + read-only — requires a linked Extentos account (a 401 returns account_required, run extentos-mcp login). USE to report spend, spot the dominant model, or ground a model-choice recommendation in real cost (pair with getAssistantConfig). DON'T USE for live per-event traces (use getEventLog) or to change config (use setAssistantConfig).

Read whether a project's Meta DAT build identity is set — plus a MASKED hint and when it was updated. Returns { credentials: { metaDat: { set, appId, clientTokenHint, teamId, urlScheme, updatedAt } } }. There is NO provider-key entry — the assistant always runs on the Extentos managed gateway and there is no key to supply or check. SECRETS ARE NEVER EXPOSED — only presence + a masked hint; the encrypted value is never read or decrypted. Account-scoped — requires a linked Extentos account (a 401 returns account_required, run extentos-mcp login). GATED by the project's Credentials access grant (default Read+Write; a permission_denied (403) means it was set to Off). USE to check whether a project still needs an API key before recommending setup, or to confirm a key landed after setCredential. DON'T USE to read a secret value (impossible by design) — to ADD one, use setCredential.

Start WRITE-WITHOUT-KNOWING entry of the project's Meta DAT build identity. Takes NO secret argument BY DESIGN: the value must never pass through you. Returns a browser handoff { mode: "browser", url, credentialType, currentlySet } — a link to the project's dashboard credentials page where the signed-in owner pastes the secret; it goes straight from them into the encrypted vault (you can never read it back). GATED by the project's Credentials access grant at Read+Write (default Read+Write; a permission_denied (403) means it was set to Read or Off). Account-scoped (a 401 returns account_required, run extentos-mcp login). USE to wire up a project's API key from the agent loop without ever handling the secret: show the developer the returned URL, ask them to paste + save, then confirm with getCredentialStatus. DON'T pass a key value (there is no field for it) and DON'T ask the developer to paste a secret into the chat.

Read a project's PRODUCTION analytics — aggregate telemetry from the app's shipped (App Store / Play Store) installs over a recent window. Returns { window, totals: { events, activeInstalls }, byEvent, byDay, byVendor, byPlatform, hasData }. METADATA ONLY — event counts + active installs, NEVER transcripts/content/PII. Reads the prod-attested telemetry warehouse (the same data the dashboard Analytics page shows), so it is EMPTY until the app ships to a store and sends prod-attested events — dev/sideload telemetry is a separate tier and not included (use getEventLog for the live sim/dev event stream). Account-scoped + OWNERSHIP-CHECKED (you can only read your own projects; a 403 not_owner means your account has no data for that package) and GATED by the project's Analytics access grant (default Read; a permission_denied (403) means it was set to Off). A 401 returns account_required (run extentos-mcp login). USE to report a shipped app's real-world usage/adoption. DON'T USE for gateway spend (use getGatewayUsage) or live dev events (use getEventLog).

Analyze proposed voice phrases for UX issues (length, homophones, digit-usage, Meta wake-word collision, ambiguity with existing phrases) before wiring them into a wake trigger. Returns { results, generalGuidance, summary }: results is one entry per input phrase — { phrase, issues: [{ severity: 'error' | 'warning' | 'info', rule, message }], suggestions: [], collisions: [] } — where collisions lists exact-duplicate matches and issues carries per-rule findings (rules: too_short, homophone_risk, digit_usage, punctuation, collision, disambiguation, meta_wake_word_overlap, match_mode_hint). Only collision and meta_wake_word_overlap are error-severity (fix before wiring); everything else is advisory. generalGuidance is cross-phrase advice; summary is a one-line rollup. Applies equally to glasses.voice.onPhrase(phrase) { ... } registrations and direct glasses.audio.transcriptions() consumers — the issues this catches are the same regardless of which API surface dispatches the handler. USE before adding new phrase-match conditions to ANY voice-driven handler. DON'T USE for general voice docs (use searchDocs topic 'voice_integration').

Derive the Android permissions, iOS Info.plist keys, Meta DAT scopes, AND the runtime grants a list of SDK capabilities requires. Returns { android, ios, metaDat, runtimeGrants, summary }. runtimeGrants is the half that is not a manifest entry — the OS grants nothing until the app ASKS at runtime, and applying every returned plist key still ships a mute app if you skip it. Each entry says who asks (requestedBy: app | sdk), the exact call when it is yours, and what silence looks like when nobody does. On iOS the microphone is the app's job and its absence is invisible: no prompt, no error, no transcripts, every voice command dead on hardware while working perfectly in the simulator — ALWAYS all three platform blocks regardless of the platform argument (which only frames the one-line summary; it does NOT filter the response). android = { permissions[], manifestEntries[] (ready-to-paste lines), foregroundService: { required, types[], declaration, devInstructions }, notificationListener: { required, declaration, devInstructions }, minimumSdk, compileSdk, targetSdk }. ios = { plistKeys: [{ key, value, reason }] }. metaDat = { scopes[], registrationRequired, registrationSteps[] }. Capabilities that need no platform permission (e.g. speak, connection_state, earcon) contribute nothing and are accepted silently — they never error. USE after deciding which features (capture_photo, transcription_incremental, voice_command, …) the app integrates. DON'T USE for capability discovery (use getPlatformInfo).

Read-only snapshot of the current Extentos integration at a project path. Returns { found, manifest, generatedFilesStatus, dependency, drift, connectionPageConfig, gaps, summary }. found: false is NOT an error — when no extentos.manifest.json exists the call still succeeds, returning empty sub-objects plus a gaps remediation string that tells you to run generateConnectionModule. When found: true: manifest is the parsed extentos.manifest.json; generatedFilesStatus is per-file { path, exists, hasMarker, hashMatch }; dependency is { found, coordinate, file } for the Gradle/SPM dependency; drift is { fileDrift: [{ path, reason: 'missing' | 'marker_absent' | 'hash_mismatch' }], dependencyDrift: boolean }; connectionPageConfig is the committed connection-page snapshot (or null); gaps is an array of human-readable remediation strings; summary is a one-line rollup. USE before validateIntegration or for a 'what's wired so far?' read. DON'T USE for correctness checking / a pre-test gate (use validateIntegration — it severity-tiers the same surface and adds ~15 checks).

Pre-test gate that runs ~16 severity-tiered checks over the whole project — manifest present + parses, generated-file markers + hashes, dependency declared, library-version freshness, Android Meta-DAT repo declaration, bootstrap wiring (ExtentosGlasses.create / Extentos.create), connection-page config, toolchain (AGP/Gradle) floors, foreground-service hints, permissions, and the PROJECT KEY (project_key_present) — the account-bound gateway credential, checked for correct SHAPE and not merely presence, because an unresolved build-setting placeholder passes a build and then fails at runtime in every environment. Only generateConnectionModule mints one, including for voice-only apps that generate no connection page. The permissions check DERIVES the permissions your declared capabilities imply and verifies they're present in the REAL app/src/main/AndroidManifest.xml elements (Android) or the app target's Info.plist keys (iOS) — the runtime source of truth, not the manifest JSON's permissions array. Returns { valid, checks, summary }: each checks[] entry is { name, passed, severity?: 'error' | 'warn' | 'info', details?, fix? }. valid is true when zero ERROR-severity checks fail; warn/info are advisories that don't block. summary reads 'Safe to test' ONLY at zero warnings — any warning demotes it to 'review before testing'. So treat this as a graded readiness report, not a binary pass/fail. USE after making changes, before testing. DON'T USE for reading state without judging it (use inspectIntegration).

Get-or-create: provision a browser-based simulator session for this project, OR return the existing saved one. The simulator hosts your customer-built app running against a Meta-DAT-shaped transport stub — glasses.audio.transcriptions(), glasses.camera.capturePhoto(), glasses.audio.speak() and every other capability primitive flow through it exactly as they would on real hardware, so the agent can dogfood a handler in a browser before a Meta DAT is paired. Persistent-simulations model: each project + platform has at most one saved sim tied to the user's account. Calling this returns that sim's existing URL if it exists (status:'resumed') instead of creating a new one. Rotating to a new sim identity is a deliberate two-step act: deleteSimulatorSession({ sessionId }) first, THEN mint again. There is no force-fresh flag, on purpose — a discarded sim leaves device registrations behind that outlive it, so a silent replace can auto-bind the next mint to an app that is already gone. First-time per project: creates the sim (status:'active'); subsequent calls in any future session return the same sim. Auto-bind (response carries autoBind:'attached'): when the dev's running app is reachable via the MCP local bridge, attaches it to the session — no rebuild needed. Otherwise URL-bake fallback (Android: buildConfigField patch; iOS: plist write). For 'resumed' responses, the URL is unchanged from last time so no rebuild/patch is needed regardless. Iteration model — DO NOT call this per change: app-code edits rebuild + reinstall and reattach automatically; the simulator URL is stable. Calling this again on the same project just returns the same sim — cheap, idempotent. First-link auto-poll: when account-linking is required, by default polls completeAuthLink internally for autoLinkSeconds (default 30) and re-mints on success. autoLink: false for CI / non-interactive contexts.

Ensure a connected simulator browser tab for the session — opens one if needed and confirms the browser's WebSocket actually attached before returning. The browser tab is the hardware surrogate's viewport: camera + inject tools require it (setSimVideo returns browser_not_attached without it; the capture_photo / describe_scene flows the assistant runtime drives fail). Idempotent — if a browser is already connected it's a no-op (returns alreadyOpen: true); it never opens a redundant tab (the hub would reject a second live browser with role_conflict). When to call: right after createSimulatorSession before any camera-driven flow; after a backend deploy (which severs every sim WebSocket — the tab does NOT auto-reconnect); whenever the tab was closed; or any time getSimulatorStatus / a camera tool reports the browser isn't attached. How it works: checks the AUTHORITATIVE in-memory hub liveness (the same signal setSimVideo/inject enforce, so this can't disagree with them — unlike getSimulatorStatus.connectedRoles.browser, which is persisted state that can lag a dead socket ~25-50s after an unclean drop); if no browser is attached, the MCP server opens the session URL in the developer's default browser ITSELF (cross-platform — no shell command for you to run), then polls until the tab's WebSocket attaches or timeoutMs elapses. Headless / remote agents: pass autoOpen: false to get an immediate presence snapshot + the sessionUrl with NO spawn and NO wait — surface the URL to the developer to open on a machine with a display, then re-run with autoOpen:true to confirm. Returns { alreadyOpen, opened, browserConnected, browserClientId, appConnected, sessionUrl, waitedMs }; errors with browser_not_connected (carrying sessionUrl) when an auto-open didn't attach in time. DON'T USE to check whether the device APP is attached or hardware is ready — this ensures the BROWSER viewer only; use getSimulatorStatus for app/hardware state.

Poll the backend until the user finishes signup at the verificationUrl, then persist the bearer token to ~/.extentos/auth.json. USE after createSimulatorSession returns status:'auth_required' AND you set autoLink:false (or the inline poll timed out). DON'T USE preemptively. Note (Bundle 10+): createSimulatorSession's autoLink arg (default true) handles this inline for you in the typical case; you only need to call completeAuthLink directly if you opted out of autoLink or the user took longer to approve than autoLinkSeconds.

Fetch structured event trace inside a simulator session. Primary debugging tool for why is my handler not seeing what I expect — transcripts not arriving, photo capture failing, toggle changes not propagating, speak getting cut off, connection dropping, display not rendering. Events are grouped into six chips (errors / voice / camera / display / lifecycle / custom) — one chip per event, with errors absorbing every severity≥warn row regardless of modality. To see e.g. voice activity plus voice errors, fetch the chips separately and union them. USE to diagnose which capability primitive is misbehaving. DON'T USE for static configuration checking (use validateIntegration) or for live session phase (use getSimulatorStatus). Scope: captures the simulator's WebSocket relay — transport + SDK primitives (audio, camera, speak, toggles, voice triggers, runtime events). Customer-side direct-HTTP calls (BYOK Anthropic / OpenAI / Gemini / etc.) traverse api..com from the customer's app, NOT the simulator relay — invisible by default. Surface BYOK calls in the event log by wrapping them in glasses.observability.aiCall(label) { ... } — the wrapper emits ai_call_start and ai_call_end frames with timing + success/error metadata. These land under the 'custom' chip — the dedicated 'ai' chip was retired 2026-07-25 after never carrying an event in production. Without the wrapper, BYOK failures show as silent gaps (e.g. capture_photo + photo_result + speak with 3 unexplained seconds between) — those failures still need logcat / OSLog. Use the wrapper for any AI call that's part of your debug story; leave it off for true fire-and-forget background calls. Live watch: pass follow: true to block until new events land instead of returning an empty snapshot, and carry the returned cursor between calls — loop the pair to tail the log in real time (e.g. to follow a multi-turn AI conversation the developer built into the app, reacting to each speak as it happens). See getCodeExample('agent_test_loop').

Inject a synthetic STT transcript into a live simulator session, the same way the simulator browser tab's click-to-fire chips do. Closes the agent-driven end-to-end test loop: after createSimulatorSession returns and your app attaches, call this with the wake phrase text to drive your voice handler — no human in the loop. Frame travels through the same hub path as a real browser click, so handler dispatch + event-log entries (visible via getEventLog) are identical. Drives the glasses.voice.onPhrase matcher: a wake-phrase match dispatches the handler exactly as a real utterance would. USE for automated voice-flow validation (createSimulatorSession → injectTranscript → getEventLog). DON'T USE for static validation (use validateIntegration) or capability discovery (use getPlatformInfo). See getCodeExample(pattern: 'agent_driven_e2e_full_loop') for the full agent-driven recipe.

Phase 4 — drive an assistant turn from outside the live session for agent-driven E2E tests of glasses.assistant.start { tool(...) { ... } }. As of iter5.2 (2026-05-27) the text: path works for BOTH the Mock provider and the real OpenAi Realtime provider — same MCP call, provider-appropriate routing: (a) OpenAi (default in production apps) — injected as a synthetic user turn via conversation.item.create + response.create on the live Realtime WebSocket. Drives REAL model behavior with full conversation context + tool routing decisions; costs real tokens (~$0.005/turn); takes 500-2000ms for the model to respond + dispatch tool. (b) Mock — word-overlap-matches against registered tool descriptions + dispatches synchronously. Sub-ms, deterministic, $0. The library filters inject frames to source: "assistant_inject" only, so browser-mic STT transcripts (already flowing via the PCM audio path) are NOT double-injected. Wake first if Dormant. This handler does NOT auto-wake — if the session is Dormant (silence-timeout, fresh session, post-end_conversation), the library's injectUserTurn silently no-ops because connectionRef is null + the inject is dropped. Drive the wake faithfully via injectTranscript({ text: "<your wake phrase>" }), wait for assistant.session_started in getEventLog, THEN inject. This mirrors what a real user does on real hardware: phrase, wait, command. (The iter5.3 autoWake convenience was shipped + reverted in 0.1.4 — it collapsed the explicit two-step into one call, hiding the wake step and racing with onWake greeting hooks.) audioWavBase64 path is DEFERRED to v1.1 (needs Rust core changes) — handler returns not_implemented if passed. Returns watchCursor — a seq cursor captured immediately before the inject; pass it straight into assertToolCalled({ sinceCursor }) so the assertion anchors BEFORE this inject and never misses the tool call it triggers (the model fires the tool 0.5-2s later, often after your assertToolCalled call has already started). USE in the agent E2E loop: createSimulatorSession → injectTranscript("") → (wait for session_started) → const r = injectAssistantUtterance({ text: "..." }) → assertToolCalled({ name: "...", sinceCursor: r.watchCursor }) → getEventLog(types:['assistant.*']) → cross-verify via adb logcat + screencap. DON'T USE for Phase 3 glasses.conversation.onWake { listen / speak } flows — that's still injectTranscript's domain.

Phase 4 — wait for an assistant.tool_called event matching name (and optionally argsMatch partial-match). Re-scans the backend event log every ~200ms from a FIXED anchor until match or timeout, then returns the matched event payload directly (or errors with tool_not_called on timeout). Pass sinceCursor — the watchCursor from the injectAssistantUtterance that triggered the tool. That anchors the wait BEFORE the inject, so the triggered tool call (which the model fires 0.5-2s later) is always caught even if this call starts after the tool already fired. Without sinceCursor the call anchors at its own start ("now") and will MISS a tool that fired in the gap between your inject and this call — the dominant false-negative cause before the 2026-05-29 fix. The fixed-anchor re-scan (vs an advancing cursor) also means a late-committing event is never skipped, and the match is pure seq-comparison so it has zero clock-skew dependency. USE in the agent E2E loop right after injectAssistantUtterance. DON'T USE to inspect the full event trace (use getEventLog) or to check static tool registration (use inspectIntegration).

Read a live simulator session's current state — phase (active/paused/closed), hardware-ready, attached roles (app + browser), and the testVideos list available to drive setSimVideo (defaults bundled with the platform plus any MP4s uploaded for this project on the simulator page). Also returns a freshness advisory — each role's connectedAt plus the staleness rule — so you can catch the #1 silent failure in the test loop: driving a connection that went STALE after a code change (rebuilt/reinstalled the app → relaunch it so it's the fresh build; pushed a deploy → it severed the sim WebSockets, so re-ensure the browser). Session-level snapshot only. Does NOT report which capability streams are open — it never did: activeStreams/lastEventTimestamp existed but were never populated by anything (zero of 251 production sessions), so they were removed in 0.11.55 rather than left reporting 0 while streams ran. To see whether a subscription is live, read the event log (getEventLog) or your own app's state. USE during testing to confirm the session is healthy, the app role has attached, that a connection isn't stale after a change, or to discover which test videos you can pipe into capture_photo/capture_video. DON'T USE for event traces (use getEventLog).

Retire a simulator session on purpose. USE only when you genuinely need a NEW session identity — a different session id and URL. You almost never do. createSimulatorSession already RESUMES the saved sim for a project: same session, same URL, your app reattaches with no rebuild. That is the persistent-sim model and it is the intended way to work. DON'T USE to 'clean up' or 'start over' when something looks wrong — deleting disconnects any attached app and browser, and an app carrying a baked EXTENTOS_SESSION_URL needs a rebuild against the new one. Deleting is the FIRST HALF of a deliberate rotation: delete, THEN mint. There is no one-call force-fresh, precisely so a rotation is two explicit steps rather than a silent replace. Sim churn is what that costs you: discarded sessions leave device registrations behind that outlive them, and a later mint can auto-bind to an app that is already gone. Idempotent — deleting a session that is already gone returns alreadyGone: true rather than an error, so a retry is safe and you can still tell the two apart.

Pipe a Test Video into the simulator's camera input so capture_photo / capture_video / videoFrames run against a known scene instead of a blank viewport — closes the agent-driven test loop for camera-driven flows the same way injectTranscript closes it for voice flows. Discover available IDs via getSimulatorStatus.testVideos (bundled defaults + project uploads). Requires the simulator browser tab to be connected (browser role attached) — call ensureSimulatorBrowser({ sessionId }) first to guarantee it (idempotent: opens the tab cross-platform + polls until it attaches). Same precondition as the simulator's photo/video capture itself. The browser loads the video as its cameraSource (same code path as a human drag-dropping a file), and a camera_source_set lifecycle event lands in getEventLog so the source change is visible to your verification step. USE before triggering a capture_photo / capture_video flow that needs a scene. DON'T USE to upload new videos — that's the project simulator page's job (extentos.com/projects//simulator).

Switch the simulated glasses DEVICE MODEL — one of the eight simulatable models: rayban_meta, oakley_meta_hstn, oakley_meta_vanguard, rayban_meta_optics, meta_glasses (camera + audio, NO display), rayban_display (display + Neural Band), and the EXPERIMENTAL Android XR pair — android_xr_audio_glasses (camera + audio, no display) and android_xr_display_glasses (adds a display, driven by the temple touchpad, panel 450x394 rather than Meta's 600x600 square). The selected device's capability profile drives glasses.display.isAvailable on the connected app, so this is how an agent tests BOTH branches of a capability-gated feature headless: set rayban_display to exercise the display path (glasses.display.show renders), set any no-display model to verify the graceful degradation (the app's isAvailable guard declines; show() no-ops). The model identity ALSO surfaces to the app as glasses.device.type (and in the assistant's glasses-state context), so per-model polish — e.g. an Oakley-specific default voice — is testable by switching between models that share the same capability profile. Mirrors the sim's device dropdown — the change persists on the session AND is pushed live to a connected app (the gate flips without a reconnect), and a device_changed event lands in getEventLog(filter:'lifecycle'). USE before driving a display flow to pin which device you're testing; pair with getDisplayState / injectInput. DON'T USE for the camera scene (setSimVideo) or voice (injectTranscript).

Read what's currently rendered on the simulated glasses DISPLAY (track 5 — the native glasses.display.* capability). Returns the DisplayNode tree the app most recently rendered via glasses.display.show { column { text(); button(...) } } (or shown: false after glasses.display.clear() / before any show), the flat list of selectable node ids (button ids + clickable container ids), the node count, and the root kind. This is how the agent verifies what's on the glasses screen without pixels, and discovers which ids it can drive with injectInput. Source is the live in-memory hub snapshot (the latest show frame), so it's exact + immediate — no event-log redaction. Pairs with injectInput to close the agent-driven E2E loop for display flows: createSimulatorSession → (app calls display.show) → getDisplayState (read the tree + ids) → injectInput({ action: 'select', targetId }) → the dev's onClick runs → getDisplayState again to see the re-render. USE to confirm a display rendered, read its structure, or find selectable ids before injectInput. DON'T USE for the display event history (use getEventLog(filter: 'display')) or non-display session state (use getSimulatorStatus).

Drive the simulated glasses' DISPLAY input — the agent's way to "click" the display with no human and no hardware (track 5). Three actions: select (+ targetId) fires that node's onClick, running the developer's handler exactly as a Neural-Band index-pinch would on real glasses — THE action that closes the display E2E loop; navigate (+ optional targetId) moves the focus highlight (to a specific id, or the next selectable node when omitted); back emits a back gesture. The frame is delivered to the simulator browser tab (the display surrogate that owns the rendered tree + focus), which routes the selection back to the app — so an agent-injected select and a human clicking the sim's gesture panel are identical downstream (selection flows browser → backend → app, mirroring real hardware). Requires the simulator browser tab connected — the display only exists while the browser renders it; call ensureSimulatorBrowser({ sessionId }) first or this returns browser_not_connected. Discover targetId values from getDisplayState.interactiveIds. Typical loop: createSimulatorSession → ensureSimulatorBrowser → (app shows a display) → getDisplayState → injectInput({ action: 'select', targetId: 'end-run' }) → getEventLog(filter: 'display') shows display_select then the re-render (display_show). Mirrors injectTranscript (voice) / setSimVideo (camera) for the display capability. USE to exercise display button handlers + navigation end-to-end. DON'T USE for voice flows (injectTranscript / injectAssistantUtterance) or to read display state (getDisplayState).

Press the simulated glasses' hardware CAPTURE BUTTON (the right-temple controls) — the agent's way to exercise the wearer's hardware privacy gestures with no human and no hardware. Tap on a LIVE camera stream pauses it; a second tap resumes. While paused, every stream-needing capture (capturePhoto — both mechanisms —, captureVideo, videoFrames) fails with CaptureError.StreamPaused carrying the actionable "tap the right temple to resume" message, and a capture_denied row lands in getEventLog(filter: 'errors') — the same behavior a wearer's temple tap produces on real glasses, because it IS the same shared gate. Hold STOPS the stream (camera_stream_closed): on real glasses the hold stops the whole device session — the connection drops and the SDK auto-recovers over ~3-5s; the sim closes the stream and deliberately skips the connection blip (a surfaced substrate delta) — the next frame-grab photo / video / videoFrames use re-arms it. The stream-state transitions land as camera_stream_opened / camera_stream_paused / camera_stream_resumed / camera_stream_closed under filter 'camera', and the sim page's capture LED tracks them (lit = streaming, dark = paused/closed); videoFrames delivery halts while paused and dies on close. A gesture with NO live stream is invisible to the app (hardware-faithful — it falls through to Meta's first-party capture, which DAT apps never see): arm the stream first. Mirrors the sim page's Hardware-buttons panel — same hub path, identical downstream. Typical loop: createSimulatorSession → ensureSimulatorBrowser → setSimVideo → (app takes a photo — stream arms) → injectHardwareButton (pause) → drive a capture → getEventLog(filter: 'errors') shows capture_denied → injectHardwareButton (resume) → capture succeeds. USE to test paused/stopped-camera handling end-to-end. DON'T USE for display input (injectInput), voice (injectTranscript), or hardware alerts like thermal/hinges (those inject via the sim page).

Return a personalized production-readiness checklist based on the SDK capabilities the app uses, the handlers it declares, and the BYOK services it integrates. Returns { ready, steps, summary }: ready is true when no step has status 'needed'; each steps[] entry is { category, description, status: 'done' | 'needed' | 'optional', details, fix?, requiredScopes?, affectedHandlers?, codeChange?: { file, hint } }. Steps are conditional — streams add a foreground-service step, voice/camera/audio add a real-hardware-verification step, and declared BYOK services (or the Phase 3 conversation runtime) add an API-keys step. Pass projectPath so the Meta-account and credential-swap steps read the project's real build state and report 'done' instead of 'needed'. status is guidance, not a measurement: ONLY the vendor-setup and credential steps are derived from your project, and only with projectPath — every other step stays 'needed' however much of the work you have actually completed, so identical output before and after doing the work is expected rather than a bug. ready accordingly means 'nothing left on this list', NOT 'Extentos verified your app'; for checks that genuinely inspect the project use validateIntegration. Cross-links: getPermissions for the exact permission set the Permissions-Audit step describes, getCredentialGuide for the per-provider key setup the API-Keys step needs, validateIntegration for the structural pre-test gate. USE when going to production. DON'T USE during development.

Return step-by-step credential setup for the Meta DAT build identity plus each BYOK AI provider the app integrates. Returns { metaCredentials, serviceCredentials, keyStorage, summary }: metaCredentials = { required, requiredScope: 'real_hardware_only', scopeNote, steps[], configChange: { file, keys[], example, note? } } — the requiredScope / scopeNote say Meta DAT creds are needed ONLY for builds on real Ray-Ban Meta hardware; sim/emulator dev works without them, so DON'T lead the developer through Meta's developer-portal registration until the hardware milestone. serviceCredentials = one entry per BYOK service { service, forHandlers[], steps[], codeExample }. keyStorage = platform-specific { preferredPath, fallbackPath, doNotDo[], notes[] } secure-storage guidance. Cross-links: getProductionChecklist for where these steps sit in the ship sequence, getCredentialStatus / setCredential for the account-vaulted path (secrets pasted in the dashboard, never through the agent). USE when wiring a new BYOK provider or graduating from simulator to real hardware. DON'T USE during the simulator-only dev loop (Meta DAT creds aren't needed there).

Search Extentos documentation by topic or keyword. The conceptual / narrative layer that complements the action-oriented tools — read these to understand how the SDK is meant to be used, not just what calls exist. Canonical post-pivot topics: getting_started (full topic index — read first), custom_handlers (the central composition doc — how to write a Handler class that subscribes to capability primitives), assistant_runtime (the canonical voice-assistant surface; read first if you're building a voice assistant), voice_integration (lower-level primitives — glasses.voice.onPhrase + audio.recordDiscrete + customer-side LLM; for fine-grained control), connection_ui_placement, host_app_scaffold, auto_bind_session_lifecycle, local_bridge_discovery, device_code_flow, connection_state_model, permissions, production_checklist, concurrency_modes, multi_platform_projects, library_api, toggles, audio_video_coexistence, simulator_browser_mode, simulator_session_lifecycle (mint once, reuse, delete deliberately — read before minting a second sim for a project), event_log_schema, file_actions, agent_e2e_testing (how an AI coding agent verifies its own generated handler end-to-end — sim event log + adb-mediated emulator DB read + screencap, the dual-layer pattern that closes the agent loop without a human). Topic IDs are stable. At least one of topic or query is required (enforced by the handler — calling with neither returns an invalid_arguments error). Pass topic alone to fetch the full topic content (most common); pass query alone for keyword search across all topics; pass both to narrow keyword search inside a topic. mode: "snippets" returns only the matching paragraphs instead of full topic bodies — pair with a query when keyword-searching across topics so a single search doesn't dump ~20KB of unrelated content into context. USE to learn how primitives compose into a real flow, or to read up on a specific feature (permissions, toggles, connection lifecycle). DON'T USE for the platform capability list (use getPlatformInfo), call-shape per feature (use getCapabilityGuide), or project-installed state (use inspectIntegration).