ISyMCP
by DannyBaanks
README.md
<div align="center">
# ISyMCP
**A local HTTP + MCP bridge for ChatGPT and coding-agent workflows.**
**Un puente local HTTP + MCP para ChatGPT y agentes de programación.**
Scoped tools · Native Codex CLI integration · Verifiable execution receipts · Bun on Linux & Windows
[](https://github.com/DannyBaanks/chatgpt-http-mcp/actions/workflows/ci.yml)
[](https://github.com/DannyBaanks/chatgpt-http-mcp/releases/tag/v1.0.0-rc.2)
[](docs/verification/2026-10-14-visible-two-way-pass.md)
[](LICENSE)
**[Download rc.2](https://github.com/DannyBaanks/chatgpt-http-mcp/releases/tag/v1.0.0-rc.2)** · **[Guía en español](GUIA.md)** · **[Quickstart](#quickstart)** · **[Security & limitations](#limitations--risks--read-before-using)**
<img src="docs/images/chat-tools.png" alt="ISyMCP demo panel with tool execution receipts and success/failure cards" width="900">
<sub>Illustrative screenshots use demo data; regenerate with <code>bun run scripts/readme-screenshots.ts</code>.</sub>
</div>
> [!IMPORTANT]
> **Experimental and not affiliated with, endorsed by, or approved by OpenAI.** ISyMCP includes a browser adapter that automates parts of the consumer ChatGPT website. This is **not** the official OpenAI API and may be incompatible with OpenAI's terms or account restrictions. A successful technical test does **not** mean a workflow is permitted. For unattended programmatic AI calls, prefer the [official OpenAI API](https://platform.openai.com/docs/overview). See [risks](#limitations--risks--read-before-using) and the [internal policy assessment](docs/compliance/FINAL_VERDICT.md) **before connecting an account**.
---
## Start here
| I want to… | Go to |
|---|---|
| Download a versioned source package | [v1.0.0-rc.2 release](https://github.com/DannyBaanks/chatgpt-http-mcp/releases/tag/v1.0.0-rc.2) |
| Set up ISyMCP on Linux | [Quickstart](#quickstart) · [Spanish guide](GUIA.md) |
| Set up ISyMCP on Windows | [Windows guide](docs/WINDOWS.md) |
| See what was actually tested | [Verification records](docs/EVIDENCE.md) |
| Understand scopes, tokens and sandboxing | [Architecture](docs/ARCHITECTURE.md) · [Protocol](docs/PROTOCOL.md) |
| Check compatibility and account risks | [Limitations](#limitations--risks--read-before-using) · [Compliance review](docs/compliance/FINAL_VERDICT.md) |
| Review changes and publish a version | [Changelog](CHANGELOG.md) · [Release process](docs/RELEASING.md) |
### Releases & downloads
**Current published candidate: `v1.0.0-rc.2` (October 14, 2026).** [Release notes & assets](https://github.com/DannyBaanks/chatgpt-http-mcp/releases/tag/v1.0.0-rc.2) · [All releases](https://github.com/DannyBaanks/chatgpt-http-mcp/releases) · [Tags](https://github.com/DannyBaanks/chatgpt-http-mcp/tags).
Download `isymcp-v1.0.0-rc.2.tar.gz` or `.zip`, plus `SHA256SUMS`, from the release. **They are source archives, not standalone binaries.** Bun is required. The release workflow publishes only after CI succeeds against the tagged commit; existing tags are not moved. See [release maintenance](docs/RELEASING.md).
> [!NOTE]
> Why no green **Latest** badge? GitHub excludes prereleases (`-rc.*`) from its stable `/releases/latest` endpoint. The version **is tagged and released**; it is simply not marked stable yet.
---
## What you get
### 💬 A real chat, on localhost with Live Thinking Stream
`isymcp panel` opens a chat at **http://127.0.0.1:8798**. Each chat in the sidebar is its **own real ChatGPT conversation**, and you watch the answer **while ChatGPT is still writing it**: lists, code blocks, links, and animated, collapsible thought/reasoning boxes in real time.
The chat panel has three working views over the same conversation: **Conversación** for messages, **Evidencia** for saved tool receipts with filters and links to the owning message, and **Actividad** for the current bridge/chat status. The narrow left rail opens recent chats, session selection, activity, settings and host status; the drawers support Escape and return keyboard focus. Drafts save per chat and return after switching or reloading. The assistant avatar is a small local OpenAI Blossom asset; ISyMCP remains the product identity. Evidence cards show only fields retained in the receipt, so they do not invent or reconstruct stdout/stderr. See the [operator guide](GUIA.md#panel-de-chat-evidencia-y-actividad) and [browser verification record](docs/verification/2026-10-15-panel-chat-views.md).
<img src="docs/images/chat-live.png" alt="Live answer streaming with a blinking cursor and a running tool card" width="900">
### 🛠️ Tools you can actually verify
Turn on **Codex ISyMCP** in a chat and ChatGPT can run commands in a folder you choose, inside a sandbox. Linux hosts use `bubblewrap`; Windows defaults to the Docker Desktop Linux image and can opt into Codex CLI's native Windows sandbox. Every card above an answer comes from the **execution trace**: what really ran, its exit code and how long it took. It is not the model's account of what it did.
When something fails, the card says **who** failed: OpenAI blocked it, your ChatGPT session expired, Chrome crashed, or the local bridge didn't finish.
<img src="docs/images/chat-blocked.png" alt="Error card: 'Bloqueado por OpenAI — nothing ran on your machine'" width="900">
### 🎬 Local audio & video analysis
Import a public YouTube video/Short or register a local file, then inspect **timestamped frames, contact sheets, audio activity segments, waveforms, spectrograms and stereo channels**. The Codex ISyMCP media tools use reusable asset IDs and return verifiable measurements—not invented transcripts or sound identification. [Explore the media tools](#local-media-mcp).
### 🟢 Isolated & Decoupled Codex Profiles
No more mutating your global `~/.codex/config.toml`! ISyMCP provides isolated, side-by-side environments:
- **Native Codex (`codex`)**: talks directly to OpenAI, completely independent of the bridge.
- **ISyMCP Codex (`codex-isymcp` / `isymcp codex`)**: selects the named `isymcp_web` Responses provider for that process. An explicit native `-m` bypasses the bridge. The desktop shortcut opens this CLI in a terminal.
`isymcp up` and menu restart start services without installing global routing.
`isymcp models apply` prepares a private profile/catalog under
`~/.codex-web-http/codex-profiles/isymcp-web`; it preserves Codex's global
configuration, model cache and other applications' integration journals.
The native tool adapter translates supplied function/custom declarations into
Responses calls; **Codex executes them** and returns their real results. Calls
are correlated to a task, turn and call ID, with durable replay and explicit
handling of interrupted calls. WebSocket events use one JSON event per frame.
The MCP connector has been exercised through the native Codex CLI and
ChatGPT.com. The separate Web-provider path currently supports Codex CLI:
```bash
isymcp conversation new --client cli --cwd "$PWD" --model chatgpt-web/gpt-6
isymcp conversation list
isymcp conversation resume <launch-id-or-thread-id>
```
The first committed Web response binds the Codex thread to its ChatGPT URL;
that response may be a tool call, so a receipt alone does not prove the task
finished. Synthetic Desktop App profiles are disabled: they copied the
account's private `auth.json` into an undocumented secondary profile, which
the maintainer policy classifies as `UNKNOWN / NO EXECUTE`. Use the supported
Codex CLI flow above. The Web provider does not automatically retry failed or
ambiguous rounds.
### 🔄 Two-way execution: recorded evidence
The [October 14 verification report](docs/verification/2026-10-14-visible-two-way-pass.md) documents a native Codex round trip through the active owner with a real `printf` result (`ISYMCP_TWOWAY_4afa0c89`, exit `0`). A **separate** isolated sandbox canary returned `ISYMCP_E2E_VISIBLE_OK` (exit `0`, 18 ms). These are two distinct observations; do not combine their nonces or claim that every transport was covered by one run.
The exercised lifecycle includes `codex_turn_start`, verified `bootstrap.content`, `codex_tool_inventory`, `codex_exec`, and `codex_turn_complete`. Later requests were sometimes **blocked by OpenAI before execution**; see the report for scope and failures. A recorded PASS proves that specific round trip worked, **not ongoing compatibility, contractual permission or third-party certification**.
### 🛡️ Account safety, platform boundaries & disclosure
ISyMCP's local controls include scoped session tokens, revocation, loopback checks, sandboxing and execution receipts. They **do not** expand the permissions granted by OpenAI or other services.
- **MCP calls initiated within ChatGPT:** subject to that product's tools, confirmation requirements and safety enforcement; local owner consent is not a substitute for platform authorization.
- **ChatGPT Web automation:** the repository contains headless/visible browser integration. Its unattended automation and page extraction must **not be described as officially permitted or endorsed**. The [internal audit](docs/compliance/FINAL_VERDICT.md) flags these paths for disabling or further clarification.
- **Codex CLI/App profiles:** local custom-provider configuration and the particular browser transport are separate questions. Functioning locally does not establish permission to automate consumer web sessions or copy authentication.
- **Blocked or uncertain operations:** halt and preserve a sanitized receipt; never route around a refusal, change backend to evade it or automatically repeat an uncertain side-effect operation.
- **For unattended model-to-model requests:** choose an authorized, documented programmatic interface such as the official API.
The compliance documents are **maintainer assessments, not legal advice or OpenAI approval**. [Read the risks](#limitations--risks--read-before-using) and [audit details](docs/compliance/).
### A four-line ChatGPT command with a local task
Write your authorized task in `TASK.md` inside the project, then run:
```bash
isymcp session mint --cwd "$PWD" --write --request-file TASK.md
```
Select **Codex ISyMCP** in ChatGPT's composer and paste the four lines printed
by the command. `codex_turn_start` returns the installed local guide in
`bootstrap.content` and the bound task in `request.content`, with SHA-256
hashes. You do not need to paste either document. A changed task requires a new
token; files outside the authorized workspace are rejected. Omit `--write`
for a read-only task. Existing sessions without a task file remain supported.
The ChatGPT.com canary used exactly four lines and performed real command
execution, patching, file verification and turn completion.
### 🤝 Agent harness integrations
`chatgpt_ask` is an MCP interface for requesting a second opinion from another agent. Installation strategies are documented for Claude Code, Codex, Cursor, OpenCode, Copilot CLI, Hermes, OpenClaw, Pi and Gemini CLI. **A supported installation command is not evidence of an authorized unattended ChatGPT Web workflow**; evaluate the chosen backend and permissions first.
- **CLI Strategy (`mcp add`):** Claude Code, Codex, Copilot, Hermes, OpenClaw, Pi, Gemini, Grok, Qwen.
- **Declarative JSON Strategy:** Cursor IDE (`~/.cursor/mcp.json`) and OpenCode TUI (`~/.config/opencode/opencode.json`).
<img src="docs/images/harnesses.png" alt="Harness list: claude and codex connected, qwen and gemini selected, with the exact install commands shown before confirming" width="900">
### 📊 Real-time Telemetry & Prometheus Metrics
Track latency distributions (min/avg/max), token estimates (including reasoning tokens), error taxonomies, and RSS/Heap memory in real time via `isymcp metrics`, `/api/metrics`, and native Prometheus `/metrics` endpoint.
### 📟 Status & Diagnostics Dashboard
Server, tunnel, MCP and browser health, the latest soak test turn by turn, the daily canary (with a **Run canary** button), sessions and their expiry, **Codex tasks**, and automated E2E smoke tests (`isymcp smoke`).
<p align="center">
<img src="docs/images/status.png" alt="Status dashboard: everything online, latest soak 9/10" width="640">
<img src="docs/images/mobile.png" alt="The chat on a phone-sized screen" width="190">
</p>
---
## Quickstart
For Windows PowerShell, follow the [Windows setup guide](docs/WINDOWS.md).
Windows can run native PowerShell commands through Codex's Windows sandbox or
Linux commands through the Docker Desktop fallback.
```bash
git clone https://github.com/DannyBaanks/chatgpt-http-mcp.git
cd chatgpt-http-mcp
# For a published version, check out its tag from the Releases page.
bun install --frozen-lockfile
bun link # puts `isymcp` on your PATH
# 1. Import your ChatGPT session once (keep the cookie file chmod 600)
bun run scripts/import-cookies.ts --file /path/to/cookie.txt
# 2. Start the bridge and open the panel
isymcp server start
isymcp panel # starts it in the background and opens http://127.0.0.1:8798
```
For the experimental browser-backed chat, continue only after reading the [account-risk warning](#limitations--risks--read-before-using) and confirming that your intended use is permitted. In the panel choose **Chat → Nuevo chat**; from the terminal the local command is `isymcp ask "hola"`. Run `isymcp` without arguments for the menu.
### Optional extras:
```bash
isymcp tunnel connect # lets ChatGPT call the local tools (Codex ISyMCP)
isymcp session mint --cwd ~/projects/app --write # a folder ChatGPT may work in
isymcp harness install claude cursor opencode # plan only; add --apply to install chatgpt_ask
isymcp codex launcher # installs ~/.local/bin/codex-isymcp & desktop entry
isymcp smoke # automated end-to-end sanity verification
```
**Model and reasoning (experimental Web transport).** The browser adapter attempts to use whichever model and reasoning settings are available in the signed-in UI; availability and labels can change. In the chat's side panel, **Verificar** shows what ISyMCP sees (for example `GPT-5.6 Sol · High (3/3)`). **Sincronizar ajustes** opens a visible Chrome with your session: you set the model and reasoning by hand, close the window, and that becomes the bridge's default.
*Note on Reasoning Traces vs Internal CoT:* The captured thinking (`reasoning_content` in API / thought boxes in the panel) represents the **visible summary and UI-rendered thoughts** exposed by ChatGPT Web's DOM. Capturing this DOM presentation is NOT equivalent to obtaining the model's full, raw latent internal Chain-of-Thought tokens. We document and stream strictly what is visibly presented on screen.
📖 **Human Operator Guide (Spanish):** For comprehensive step-by-step instructions, see [`GUIA.md`](GUIA.md).
---
## Limitations & risks — read before using
> [!WARNING]
> **Account risk: experimental consumer Web automation.** ISyMCP drives or observes `chatgpt.com` through browser automation in some modes. OpenAI's [Terms of Use](https://openai.com/policies/terms-of-use/) restrict automated extraction and circumventing protective measures. This project is **not an officially supported ChatGPT-to-API conversion service**, and local technical success is not permission from OpenAI. Depending on behavior and enforcement, account restrictions are possible. If you need programmatic access, use an officially supported API or product integration.
- **Do not treat an internal compliance label as approval.** The [maintainer verdict](docs/compliance/FINAL_VERDICT.md) identifies browser automation and possible anti-bot behavior as serious compatibility risks. It also contains remediation recommendations, which must be verified in code before assuming they are implemented.
- **Fragile Web adapter.** UI changes, browser challenges, session expiration and platform decisions may break output capture. Capture failures may leave sanitized diagnostic files under `~/.codex-web-http/run/`—inspect their contents before sharing.
- **UI-rendered reasoning is not raw chain of thought.** The panel may display summaries visible in the interface, not the model's hidden reasoning.
- **Cross-agent prompt injection.** `chatgpt_ask` output is untrusted input to receiving agents. Labels are advisory and cannot replace their approval or permission model.
- **Local sandbox is not a perfect VM.** Bubblewrap hides the home/secret directories; Windows MXC denies the current user's profile and known credential paths. Other host paths can remain readable, but writes stay in the authorized workspace. Network is disabled by default. `CODEX_WEB_HTTP_SANDBOX=off` weakens confinement; `CODEX_WEB_HTTP_SANDBOX_NET=1` enables network.
- **Protect credentials.** Stored session data/cookies are sensitive; file mode `0600` is not protection against malware running as the same user. Never commit or share them.
- **Provider blocks are final for the attempted operation.** Do not evade, tool-hop or auto-retry pending/unknown requests. A block does not by itself establish a contractual violation.
- **Capacity & platform.** Browser-backed turns can be slow/serialized. Linux hosts use bubblewrap; Windows defaults to Docker Desktop's Linux engine, with an opt-in native Windows MXC sandbox through Codex CLI. Native Windows MCP E2E is demonstrated on Windows 11 25H2; Docker-backed Windows execution remains **NOT_DEMONSTRATED** in this branch. Chrome or Edge is used by the experimental Web adapter.
---
## How it works (including experimental Web paths)
```
your browser ──► isymcp panel (:8798) ─┐
OpenAI-compatible clients ─────────────┼──► bridge (:8791) ──► headless Chrome ──► chatgpt.com
other agents ──► MCP isymcp-chatgpt ───┘ │
├─ live thinking ──► streaming SSE (reasoning_content)
├─ compaction ──────► automatic context pruning (>40 msgs)
└─ tools: chatgpt.com ─► tunnel ─► MCP stdio ─► bwrap / Docker sandbox
```
- **Bridge** (Bun): drives one persistent ChatGPT tab. Turns are **serialized** (one at a time, in order), so chats never write into each other. It reads the answer from the page as ChatGPT writes it, and rebuilds the Markdown.
- **Live Thinking Stream**: extracts thinking bubbles dynamically and exposes them via SSE `delta.reasoning_content` and panel UI cards.
- **Context Compaction**: automatically prunes multi-turn conversations exceeding 40 messages, preserving the initial prompt/goal, inserting a context note, and retaining the recent message window.
- **Chats**: chat id → canonical `chatgpt.com/c/…` URL, stored in `~/.codex-web-http/chats/` (0700/0600). The browser only ever sends a chat id and your text; it never decides which conversation to open.
- **Tools**: each tool turn gets a **one-off token**. It inherits the folder and read/write mode of the session you picked, lives 15 minutes and is **revoked when the turn ends**. Only that token goes to chatgpt.com. The MCP logs every call under that token's fingerprint, and the tool cards are read back from that log.
- **Reviver**: if Chromium crashes mid-turn, the bridge relaunches it and re-reads the answer that was already generated. It never sends the turn twice.
- **Never Guess**: an answer only counts if it provably belongs to *this* turn (the last assistant turn changed identity, or its text changed). Otherwise the turn fails with a DOM dump. The bridge never returns the previous answer or "whatever looks closest".
- **Canary**: `isymcp canary` runs three real turns in one dedicated chat (echo A, echo B without A, Markdown capture). `isymcp canary schedule --apply` runs it daily and sends a desktop notification on failure.
- **Smoke Suite**: `isymcp smoke` runs unified end-to-end verification of health, metrics, harnesses, and isolated launchers.
<details>
<summary><b>Chat details</b></summary>
- Only the **new** message is sent each turn; ChatGPT already holds the context.
- Live text: the bridge re-reads the page about every 0.4 s, and each update replaces the whole partial answer.
- Markdown goes through a renderer that escapes everything first, so model HTML never reaches the page.
- Tools are **off** in every new chat. The session picker shows fingerprint, folder, read/write and expiry, never the token.
- The OpenAI-compatible route `/v1/chat/completions` keeps its own contract (full transcript) and its own conversation.
</details>
---
## Experimental HTTP compatibility endpoint (not the official OpenAI API)
```bash
curl -s http://127.0.0.1:8791/v1/chat/completions \
-H 'content-type: application/json' \
-d '{"model":"chatgpt-web/gpt-5.6-sol","messages":[{"role":"user","content":"hola"}]}'
```
- `POST /v1/chat/completions` (models `chatgpt-web/*`), plus native passthrough for `/v1/responses`, `/v1/models` and the websocket.
- **Idempotency**: send `x-isymcp-turn-id`. A replay returns the same response (`x-isymcp-replayed: 1`) without running the model again.
- **Typed errors** mapped to HTTP status (400/401/403/409/415/502/503/504). See [`docs/PROTOCOL.md`](docs/PROTOCOL.md).
- **Telemetry Endpoints**: `GET /api/health`, `GET /api/metrics` (JSON), and `GET /metrics` (Prometheus/OpenMetrics).
---
## Local media MCP
Process explicitly registered audio and video locally, then inspect the evidence from **Codex ISyMCP** or a separate media MCP entry point. Media queries use an opaque `asset.id` and **do not require a Codex turn token**. Audio analysis measures signals; it does **not** provide native listening, transcription, speaker recognition or voice/music separation.
### Quick workflow
| Start with | What to do |
|---|---|
| **Local file** | Register it with `bun src/isymcp.ts media add <file>`; keep the returned asset ID. |
| **Prepare a reusable YouTube analysis** | Run `isymcp media prepare` (or `bun src/isymcp.ts media prepare`) to paste a URL, choose a folder, review metadata and confirm before download. It creates a hash-verified package and keeps the private catalog. |
| **Script an existing flow** | Keep using `isymcp media prepare <url>` for the existing noninteractive JSON command; it does not open the picker or export a package. |
| **Look up a YouTube source in ChatGPT** | Call read-only `media_lookup(url)` first. If it is `ready`, inspect its stored audio results and request one relevant sheet with `sheet_index`; `not_found` and `not_prepared` never download or analyze. |
| **One-off public YouTube video or Short** | Call `media_import(url)`, poll `media_import_status(job_id, wait_seconds: 10)` until `complete`, and use `asset.id`. |
| **Inspect video** | Use `media_info`, `video_contact_sheet` for an overview and `video_frame` for a specific timestamp. |
| **Inspect audio** | Use `audio_scan` to scan the complete track in chunks of at most 120 seconds, then `audio_analyze` to zoom into a selected waveform, spectrogram or stereo measurement. |
For step-by-step commands and the verified ChatGPT prompt, see **[GUIA.md — Medios locales](GUIA.md#medios-locales-audio-y-video-2026-10-09)**.
### Local execution, entry points & limits
The independent `src/mcp/media.ts` entry point processes only explicitly registered local assets. The main **Codex ISyMCP** entry point exposes the media tools alongside its existing `codex_*` tools; both share `src/media/register.ts`. `media_list` discovers only user-registered assets, paginated in groups of 20. `media_lookup` resolves only a canonical public YouTube URL already in the catalog and never starts work. Codex session authorization and sandbox behavior remain enforced.
Both MCP entry points publish the bundled ISyMCP PNG through `serverInfo.icons` during initialization. ChatGPT or another MCP client may need to reconnect after upgrading the server before it refreshes the icon and tool catalog.
Audio activity windows, channel-aware waveforms, spectrograms, RMS, peaks and silence intervals run locally through **FFmpeg/ffprobe**. Video frames use a separately installed MIT **Video Vision** provider configured through `ISYMCP_VIDEO_VISION_ENTRY` (absolute `dist/index.js`) and optionally `ISYMCP_VIDEO_VISION_NODE`. Requires Bun and FFmpeg/ffprobe.
Original files remain local and read-only; requested images and statistics are returned to the MCP client. Limits: **500 MiB per file**, **120 seconds per audio window**, **60 seconds for ordinary queries**, **300 seconds for full-track `audio_scan`**, **one concurrent job** and **1 MiB per output image** (up to four images for separated stereo). Neither entry point provides an arbitrary filesystem path tool or audio transcription.
`media prepare` stores versioned manifests, verified audio-scan JSON and consecutive contact sheets privately under the media catalog. Audio is segmented in chunks of at most 120 seconds; video is sampled in gapless windows of at most 60 seconds. Failed runs can resume verified partial outputs, and only a complete generation becomes current. A later `media_lookup` returns the saved scan and sheet index; each call transfers at most one JPEG. Acoustic segments are energy measurements, not speech recognition or native listening.
The guided command exports a second copy only after the local preparation is `ready`. It records the canonical source and downloader-confirmed video ID, hashes the original and every artifact, stages under the selected destination filesystem, and publishes with an atomic no-replace operation. Existing packages are never overwritten. The contact sheets are chronological samples, not exhaustive frame extraction.
The MCP itself is a tool server, not a standalone chat model. The selected ChatGPT model produces the conversational response and decides when to call tools. Media tools do not require `turn_token`; `codex_*` execution tools require a real token from `isymcp session mint --cwd <directory>` and the explicit `COMANDO: @CODEX ISYMCP` marker. `codex_turn_start` consumes the supplied token; it does not mint one.
The standalone media stdio entry point has local compatibility coverage. Remote GPT.com calls documented below used the main Codex ISyMCP connection; they do not establish remote E2E compatibility for every standalone media tool.
### YouTube URL imports
`media_import(url)` starts an asynchronous local download of an individual public YouTube video or Short; `media_import_status` reports progress. **yt-dlp** is sourced from the Video Vision installation, `PATH` or `ISYMCP_YTDLP_BIN`, without browser cookies, configuration or plugins. Video Vision supplies frames and FFmpeg supplies audio graphics and measurements.
Import limits: **500 MiB**, **30 minutes**, a **180-second import deadline**, and one shared download/analysis worker. Sources and SHA-256 are recorded. Completed downloads persist; partial files are retained, while in-memory job status resets when the server restarts. See [GUIA.md](GUIA.md) for the verified Angel Engine import workflow.
### Video contact sheets
`video_contact_sheet(asset_id, start_seconds, end_seconds, columns?)` returns a single **1920 × 1080 JPEG** with chronological labelled frames and a timestamp/index map. Default layout: **4×4** (16 samples); **3×3** and **6×6** are also available. FFmpeg preserves aspect ratio.
The maximum interval is 60 seconds, under the shared worker/deadline and 1 MiB image cap. Sample times exclude the interval endpoint; requested seek times may differ from decoded frame timestamps. Use `video_frame` for details or a narrower interval for fast-changing scenes. **A contact sheet is a sampled overview, not video playback or OCR.**
### Audio segments, waveforms & stereo
`audio_scan(asset_id, start_seconds?, end_seconds?, chunk_seconds?, threshold_dbfs?, merge_gap_seconds?, min_segment_seconds?)` scans the complete audio track by default in chronological chunks of up to 120 seconds (`chunk_seconds` can be reduced to 1–120). It returns each chunk, acoustic activity segments with absolute source timestamps, effective parameters and a total segment count. The full scan has a 300-second deadline and returns measurements without images. Each chunk is segmented independently, so continuous activity can split at chunk boundaries. This measures acoustic energy, not speech, transcription or native listening.
`audio_segments(asset_id, start_seconds, end_seconds, threshold_dbfs?, merge_gap_seconds?, min_segment_seconds?)` finds activity intervals using **10 ms RMS windows on mono 16 kHz audio**. It accepts windows up to 120 seconds. `algorithm_version` identifies the segmenter independently of the `isymcp-media/1` response schema. Version `audio-rms-activity/2` uses these defaults:
| Parameter | Default |
|---|---:|
| `threshold_dbfs` | `-32` |
| `merge_gap_seconds` | `0.10` |
| `min_segment_seconds` | `0.12` |
Pass `merge_gap_seconds: 0.15` to reproduce the earlier six-segment grouping on the sample below. Segmentation detects **acoustic energy, not speech**: music, effects and noise can form segments, and quiet speech can be missed.
Use `audio_analyze` on a selected window to zoom into its waveform and spectrogram. The optional `channel` parameter is `mix` (default), `left`, `right` or `separate`. In stereo `separate` mode the tool returns independent left/right measurements and **four images** (L waveform, L spectrogram, R waveform, R spectrogram), plus Pearson correlation and L−R difference RMS/peak. These quantify channel similarity; they **do not separate voices, music or effects**. Source and analyzed channel counts are reported.
<details>
<summary><b>Reproducibility canary — Angel Engine Short (2026-10-09)</b></summary>
The public [Angel Engine Short](https://www.youtube.com/shorts/lHkDE3BahB0) was analyzed through Codex ISyMCP from GPT.com using the same registered asset and SHA-256. Default settings returned **eight intervals**; explicitly setting `merge_gap_seconds: 0.15` reproduced the **six earlier intervals**. Both responses reported `isymcp-media/1`, `audio-rms-activity/2` and effective parameters.
| `merge_gap_seconds` | Activity intervals (seconds) |
|---:|---|
| `0.10` (default) | `0.17–1.49`, `1.72–2.41`, `2.65–2.80`, `3.15–4.30`, `4.43–5.72`, `6.13–6.42`, `6.93–8.00`, `8.13–9.992` |
| `0.15` | `0.17–1.49`, `1.72–2.41`, `2.65–2.80`, `3.02–5.72`, `6.13–6.62`, `6.93–9.992` |
A reported `audio_analyze(channel="separate")` call over seconds 3–5 measured Pearson correlation **0.9958708812**, L−R difference RMS **0.0140555501**, and difference peak **0.1249013543**. This shows similar, non-identical channel signals; it cannot identify or separate sources.
The protocol report is user-provided rather than a captured raw protocol transcript; segment RMS/peak values were said to match but were not included in that report. See the [E2E report](docs/evidence/audio-segments-gptcom-e2e-user-report-20261009.json), [local test log](docs/evidence/audio-segment-versioned-tests-20261009.txt), and [version manifest](docs/evidence/audio-media-final-manifest-20261009.sha256).
</details>
---
## CLI Reference
| Command | Description |
|---|---|
| `isymcp` | Interactive menu with everything below (or prints status when piped) |
| `isymcp panel` | Opens the chat + status panel on :8798 (detached); `panel start` / `stop` / `status` / `run` |
| `isymcp ask "text" [--new]` | Real ChatGPT turn from the terminal; keeps one terminal chat, stdin works too |
| `isymcp codex [args...]` | Runs Codex with isolated ISyMCP web profile (without mutating `config.toml`) |
| `isymcp codex launcher` | Installs a desktop entry on Linux or Start Menu shortcut + `.cmd` launcher on Windows |
| `isymcp conversation new [--client cli]` | Interactive Codex CLI launch with isolated `isymcp_web` provider |
| `isymcp conversation list` | List confirmed conversations and linked thread IDs |
| `isymcp conversation resume <id>` | Resume an active conversation thread |
| `isymcp version` / `--version` | Display installed package version |
| `isymcp up` / `down` / `status` | Server + tunnel lifecycle and honest status |
| `isymcp server start` / `stop` | Starts or stops the HTTP bridge |
| `isymcp tunnel connect` / `stop` / `status` | MCP tunnel for ChatGPT tools |
| `isymcp session mint --cwd <dir> [--write] [--ttl h]` | Creates a scoped session folder (read-only by default, 7-day TTL) |
| `isymcp session list` / `revoke <fp>` | Audits or revokes tool session tokens |
| `isymcp harness list` | Lists detected coding agents and their `chatgpt_ask` installation status |
| `isymcp harness install` / `uninstall <ids\|--all> [--apply]` | Installs `chatgpt_ask` (dry-run plan by default, `--apply` writes) |
| `isymcp canary` / `canary status` | Real turn verification: capture, turn identity, Markdown |
| `isymcp canary schedule [--apply]` | Daily systemd user timer or Windows Task Scheduler task for regression alerts |
| `isymcp health` | Live health and diagnostic status of the bridge |
| `isymcp doctor [--json]` | Offline local diagnostics (binaries, versions, sandbox, disk, last media failure + `error_code` and remedy) |
| `isymcp metrics [--prom]` | Real-time telemetry (turns, tokens, latencies, memory) or Prometheus format |
| `isymcp smoke` | Unified E2E verification suite (health, metrics, harnesses, launcher) |
| `isymcp logs` | Live bridge log viewer and exporter |
---
## Security model
- **Local only, for real.** The bridge and the panel bind `127.0.0.1` *and* reject foreign `Host`/`Origin` headers and non-JSON POSTs (protection against CSRF and DNS rebinding).
- **Token Authentication (`CODEX_WEB_HTTP_TOKEN`):** Required for any non-loopback exposure. Clients must send `Authorization: Bearer <TOKEN>` or `x-isymcp-token`.
- **Sandbox Isolation.** `bubblewrap` on Linux, Docker Desktop on Windows, and the optional Codex native Windows MXC sandbox hide bridge secrets, deny network by default, and scope writes to the selected workspace. The native Windows MCP E2E is demonstrated in [the Windows guide](docs/WINDOWS.md); Docker-backed Windows execution is not demonstrated here.
- **Session Tokens.** Tokens expire (24 hours by default) and can be revoked. Only sanitized fingerprints (`sha256[:12]`) are stored in logs and UI.
- **Platform safety boundaries:** Reported OpenAI blocks must halt the affected operation without bypass. This safeguard is not a blanket claim that every browser path has passed a compliance audit; see [open risks](docs/compliance/FINAL_VERDICT.md).
---
## Verification & Testing
- **Full Test Suite:** **545 tests passing, 1 skipped, 0 failed across 81 files** (`bun test`; verified on 2026-10-14).
- **Two-Way E2E Round Trip:** Documented through separately recorded native-owner and sandbox canaries, including an 18 ms sandbox execution. [Receipts and limits](docs/verification/2026-10-14-visible-two-way-pass.md).
- **Unified E2E Smoke:** Verified with `isymcp smoke` and `scripts/e2e-all.ts`.
- **Soak Testing:** 99/100 successful real turns in soak baseline; 9/10 with tool execution inside sandbox.
- **Canary:** Regular verification against live chatgpt.com DOM changes.
- **Internal compliance assessment:** Supporting registers and threat model in [`docs/compliance/`](docs/compliance/); not an external certification or approval.
See [`docs/EVIDENCE.md`](docs/EVIDENCE.md), [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md), and [`docs/PROTOCOL.md`](docs/PROTOCOL.md).
---
## License
MIT — see `LICENSE`. Derivative mechanics from MIT projects are credited in `NOTICE`.
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo issues