Chat Unleashed

Chat Unleashed is a local MCP server that lets ChatGPT Chat use a Mac the way a person does: run commands, read and write files, look at the screen, drive apps through their UI, and reach the everyday apps (browser, Calendar, Reminders, Notes, Contacts, Music). It is built for Chat, reaches the Mac through an OpenAI Secure MCP Tunnel, and keeps every machine-specific detail on the machine.

Chat is where most work starts, and on its own it cannot touch the Mac. Work and Codex can act on a machine, and both draw on the shared agentic allowance. This connector closes that gap: the chat you already use can read files, run commands, drive apps and finish the job instead of describing the steps. Chat is not part of that shared allowance today, and its thinking is still unlimited, so the local agent runs on the Chat you already use.

The package is local-agent-mcp. The connector and the chat-unleashed command are the product surface.

Tool activity is recorded as JSON lines in the per-user application-data directory under logs/audit.log.

Quickstart: Clone, Build & Run

Clone the repository and build the server executable right away:

# 1. Clone the repository
git clone https://github.com/PotterSkxawng/chat-unleashed.git
cd chat-unleashed

# 2. Install dependencies and compile
npm install
npm run build

# 3. Link CLI tool to your PATH (or run directly with npm run unleashed)
npm link

# 4. Run setup walkthrough
chat-unleashed setup
# (or: npm run unleashed -- setup)

The build script compiles TypeScript with tsc and bundles the standalone executable into dist/index.js using esbuild.

Related MCP server: chatgpt-mac-bridge

Chat Unleashed CLI

The connector is named Chat Unleashed, with local mac as its alias. The chat-unleashed command is the terminal front end for it. It reads the machine before it claims anything: the launchd jobs, the tunnel profile, the env file, /health, /readyz, and a real MCP handshake.

chat-unleashed            # what is running, and the one next step
chat-unleashed setup      # guided install, skips every step already true
chat-unleashed doctor     # every check, with the fix for each failure
chat-unleashed connect    # prints exact connector settings for ChatGPT
local-mac                 # the same tool under its alias

New machine, from nothing

ChatGPT cannot reach a server on your loopback interface. The supported path is an outbound Secure MCP Tunnel:

ChatGPT -> OpenAI tunnel -> tunnel-client -> http://127.0.0.1:3344/mcp

1. Create a tunnel at platform.openai.com and copy its ID, which starts with tunnel_.

2. Create a runtime API key at API keys with Tunnels Read and Use. It is shown once.

3. Run chat-unleashed setup in a terminal and paste both. The key input is hidden and is never echoed.

4. Setup writes the tunnel profile for this server, stores the key at mode 0600, installs the launchd jobs, starts the server, waits for the MCP handshake, starts the tunnel, and checks that the tunnel reports ready.

5. Run chat-unleashed connect --open and fill the connector form with these values:

   • Name: Chat Unleashed

   • Description: Runs commands, reads and writes files, looks at the screen and drives apps on my Mac. Use it when a request needs work done on the local machine instead of only an answer.

   • Connection: Tunnel

   • Tunnel: the ID from step 1

   • Authentication: None. The runtime key stays inside the tunnel on this Mac; ChatGPT never gets a token for the server.

   Save, refresh the tools in the connector details, then start a new chat and pick Chat Unleashed from the plus menu.

6. Grant macOS permissions when a tool first needs them: Accessibility, Screen Recording, Automation per app, and optionally Full Disk Access.

Chat is the surface this unlocks. Work and Codex act through the shared agentic allowance; Chat is not part of that pool today, so a chat that calls these tools gives you a local agent without spending it.

See docs/CONNECT_CHATGPT.md for the full walkthrough and what to do when a chat session goes stale.

Secrets never print. The runtime key is read from ~/.local/share/local-agent-mcp/.env.tunnel (mode 0600) or from CONTROL_PLANE_API_KEY, and masked input is the only way setup accepts it.

Saved state

Saved settings are kept outside the published package at ~/Library/Application Support/local-agent-mcp/config.json on macOS ($XDG_CONFIG_HOME/local-agent-mcp/config.json elsewhere, or $LOCAL_AGENT_MCP_HOME when set). Tunnel credentials and profiles are stored separately by the tunnel client and must not be committed.

The file is created from defaults on first run, written atomically with mode 0600, and holds the device name, workspace/download/screenshot paths, screenshot size and format, shell timeouts, the Computer Use plugin location and any API keys you add. Change it with the app_config tool, the JSON API below, or any editor: the server re-reads it on every tool call, so edits apply without a restart, and typed values are validated instead of silently accepted. Secrets are masked whenever settings are read.

Local JSON API (for a desktop frontend)

With --http (loopback by default) the server also serves a small JSON API for a settings UI. Loopback callers only.

Permissions

The server asks macOS for what it needs and names the exact binary that has to be trusted, whatever launched it.

• Accessibility - read UI trees, click, type, arrange windows

• Screen Recording - screenshots with real pixels

• Automation - Apple Events to Calendar, Notes, browsers and the rest (per app, prompted on first use)

• Full Disk Access - optional, only for protected stores

mac_permissions reports each one; action=request triggers the OS prompts and action=fix opens the right panes. Tools that need a grant say so in their error, with the path to add.

Tools

Pages (native)

The pages_* tools drive the actual Pages app over Apple Events. There is no Word, no docx and no conversion step anywhere in the path - a document created here is a normal .pages file that opens, edits and exports like any other.

Two rules are baked into the implementation:

• Documents are never closed and Pages is never quit. Anything the tools create is saved and left open for you.

• No third-party format is used as an interchange layer. Content goes straight into Pages' rich-text model.

Because Pages' scripting dictionary exposes font/size/color but not paragraph styles, format: "styled" maps each block type onto a concrete font + size + color instead of naming Heading 1. Pass format: "plain" (the default when you supply a template) to leave the template's own paragraph styles in charge.

What an edit preserves

Every editing op falls into one of three buckets, and knowing which is which is the difference between a clean edit and a mangled document.

Never rewrites anything it was not pointed at. append, prepend, insertParagraph, deleteParagraph, duplicateParagraph, moveParagraph and pages_table's structural ops work on real paragraph and cell objects. Every other paragraph keeps its runs, including bold/italic inside the text.

Rewrites one paragraph, keeps its look. replaceParagraph, insertText, deleteText, splitParagraph and mergeParagraphs set the paragraph's contents and leave its font, size and colour alone. The paragraph looks the same afterwards - but any character-level formatting inside that one paragraph (a bold phrase mid-sentence) is flattened, because the text is being replaced wholesale. Everything outside the paragraph is untouched.

Formats without touching text. pages_format and pages_replace never change what a paragraph looks like except in the way you asked for. pages_format can address a character range (op: "characters") or every occurrence of a string (op: "matches"), which is how you bold a term everywhere without disturbing the rest of the paragraph.

setBody is the one op that rebuilds the whole body from plain text. Treat it as the "start over" escape hatch, not the default.

Editing without wrecking formatting

pages_replace is deliberately narrow: it only rewrites the units you point it at, and never rebuilds the body.

• scope - document (all paragraphs + all cells), paragraph (one index), table, row, column, cell. Table scopes take table, row, col.

• keepFormat - true (default) re-applies the paragraph's own face, size and colour after the rewrite; false lets the paragraph's style default take over. Table cells keep their formatting either way: only the cell value is written, so fill, borders and alignment survive.

Inserting text without disturbing the neighbours

Pages has no "insert paragraph at index N" command. The workaround is to splice into the anchor paragraph's own contents (anchorText & linefeed & newText), which creates the new paragraphs in place and lets the anchor keep its formatting. insertParagraph and prepend both work this way, so an insert in the middle of a styled document does not reset the paragraphs around it.

One consequence: the anchor paragraph is rewritten, so inline runs inside that single paragraph are flattened. If that matters, anchor to an empty paragraph or use append.

Default styles

pages_styles owns a persisted default at ~/.config/local-agent-mcp/pages-style.json. set writes it; pages_create picks it up whenever no style is passed, so "the look of new documents" is a machine setting rather than a per-call argument. get prints the resolved type scale so you can see exactly what each role will become.

apply restyles an existing document: it infers each paragraph's role from its size relative to the document's dominant body size (and from italic/mono faces), then rewrites font, size and colour per role, leaving every character of text alone. The heuristic is size-based, so a document with unusual proportions can be misread - pages_inspect first, then pages_format for surgical fixes.

Styles

style takes a preset name - classic, serif, modern, compact - or an object of overrides (family, baseSize, accent, muted, headerFill, headerText, zebra). Families: helvetica-neue, georgia, avenir-next, palatino, times. Every size is derived from baseSize through a fixed type scale, so one number re-proportions the whole document.

Tables

A table block builds a real Pages table: header (creates a true header row), rows, headerColumn, per-column align, columnWidths, and zebra. Tables must be the last blocks, because Pages cannot insert an inline table mid-flow and reordering content silently would be worse than refusing.

Existing tables are edited in place with pages_table:

• values - setCell (one cell, plus font, size, color, bold, italic, fill, align), setCells (a rectangular grid at once). Writing a value resets the run to the table's body style, so the cell's own face, size and colour are captured first and written back afterwards - a value edit does not quietly restyle the cell.

• structure - insertRows/deleteRows, insertColumns/deleteColumns, merge, unmerge, resize, clear, sort (by column, ascending or descending).

• formatting - format with font, size, color, fill, align, verticalAlign, wrap, headerRows, headerCols, footerRows, columnWidths, rowHeights. Omit rows/cols to format the whole table.

bold/italic work by swapping to the matching face of the cell's current family, so a Georgia cell stays Georgia when you bold it. Row and column inserts shift values, not cell formatting: the cell keeps its own styling and the data moves into it.

Set a cell value beginning with = to write a formula (=SUM(B2:B5)); Pages evaluates it, and pages_inspect will report the computed value afterwards.

What Pages does not expose

Worth knowing before you promise something the dictionary cannot do. Pages' scripting interface covers font, size, colour, cell fill, cell alignment, table geometry and object placement. It does not expose, for body text:

• paragraph alignment (left/centre/right/justify)

• line spacing, paragraph spacing, tabs, indents

• text highlight / background colour behind a run

• paragraph styles by name (which is why format: "styled" maps block types onto concrete font + size + colour instead of naming "Heading 1")

Table cells do have alignment and vertical alignment, so use a table when you need centred or right-aligned content.

Large documents

Reading used to cost four Apple Events per paragraph - a 20-page document meant thousands of round trips, and the output string was built with quadratic concatenation. Both are gone:

• Bulk property reads. font of every paragraph, size of every paragraph, color of every paragraph and contents of every paragraph each come back in a single event, so a long document is read in a handful of round trips. If any coercion fails the code falls back to the per-paragraph loop automatically.

• One join at the end. Records are accumulated in a list and joined once with text item delimiters, instead of out & ... inside the loop.

• Argument-size safety. osascript -e is bounded by the kernel's argument limit, so any script over ~60 KB is written to a temp file and run from there. Large setBody/append payloads no longer fail with "argument list too long".

• Timeouts scale with the document rather than cutting a long read off.

pages_verify is the companion tool: point it at a 20-page document and it reports counts, the typography actually in use, table structure, and any structural problems - implausible point sizes, missing fonts, empty or ragged tables, runs of blank paragraphs, merged cells that cannot be addressed. Pass expectParagraphs / expectTables to assert an exact structure.

Running in the background

The Pages tools never bring Pages to the front. Nothing in the generated AppleScript calls activate; every edit is sent as an Apple Event to the running app, so the document is modified while you keep working in whatever window you are actually using.

The single exception is opening a document that is not already open - macOS raises the app for that. When it happens the script remembers which app had focus and hands it straight back, so even that case does not leave you looking at Pages.

Verified mechanically: of the 34 scripts the Pages suite generates, zero contain activate. The only scripts in the whole server that do are the four Keynote slideshow commands and Finder's reveal/open, where taking the screen is the point.

Numbers

Numbers is not Pages with a grid. Three differences matter, and the tools are built around them:

• Content lives on sheets. numbers_sheets addresses them; every table and cell operation takes sheet (default 1).

• Cells have A1 names. numbers_cells speaks B4 and A1:C10 rather than row/column indices, and get renders the result as a grid.

• Structure is native. Numbers exposes add row above, add row below, add column before, add column after, transpose, sort, merge and unmerge as real commands. Pages tables needed value-shifting workarounds; Numbers does not, so rows and formulas move correctly.

Values are typed by shape: 42 is stored as a number, true as a boolean, =SUM(B2:B5) as a live formula, anything else as text.

Keynote

A slide's text lives in two placeholder shapes - default title item and default body item. keynote_content writes through those, which keeps the theme's typography; dropping a text box on top would look wrong on every theme. If a layout has no title placeholder the tool says so instead of silently creating an orphan text box.

Reordering is copy-then-delete, because Keynote exposes duplicate ... to <location> but no plain move. Both halves run in one script so the document never sits in a half-moved state.

keynote_show drives a real presentation - start, goto, next, previous, stop. It activates Keynote, because that is what presenting means.

Cross-app iWork

Pages, Numbers and Keynote share one object model: documents hold shapes, images, text items, tables, charts, movies and lines, and each is an iWork item with position, width, height, rotation and opacity. The only real difference is where content lives:

iwork_shapes, iwork_export and iwork_document are written once against that shared model. Pass path and the app is inferred from the extension (.pages, .numbers, .key); pass app to use the frontmost document.

iwork_script is the escape hatch for anything the structured tools do not cover: your AppleScript runs inside the target app with doc already bound. Scripts containing close or quit are refused - the never-close rule is enforced, not just documented.

Computer use

Seeing and driving the machine itself. These are the tools to reach for when an app has no scripting interface, or when the point is what is on screen.

System, workflow and everyday apps

Durable tasks

A task is a plan on disk, not a plan in a conversation. task_plan writes it, task_run executes the steps that can run now, and every step has to be proven by a check before it counts as done.

Three rules make the stored state trustworthy:

• Executed is not verified. A step with no verify check ends unverified. A task with an unmet criterion cannot close as done unless task_close force writes the gap into the reason.

• Evidence belongs to a definition. Editing a verified step clears its proof and puts it back in the queue, because the old check proved the old command.

• Approval is a stored decision, not a conversation. Steps that run rm, sudo, git push, force flags or shutdown wait until task_approve records a timestamp for that exact step.

Checks cover a shell command and its output, a file that exists or does not, a window title, interface text, another tool call, and polling with wait. The same vocabulary verifies a step and tests a success criterion.

Everything lives in <appHome>/tasks/<id>.json with an append-only <id>.jsonl action log, so a run survives a dropped conversation, a new client, or a restart of this server. Long work can run with no client attached:

node dist/index.js --run-task task_ab12cd34 --max-steps 10

That entry point loads the same plan, runs what can run, prints the report, and exits non-zero when a step failed or another process holds the lock.

Screen control

Two tools replace guessing at coordinates.

A window that exists but is not drawn is reported as such, which is how a minimized window or one on another Space shows up instead of as an element that cannot be found. The context trail is kept on disk across calls, so a workflow that moves between Pages, Numbers and Finder does not have to re-establish where it was.

The method ladder is the point. In order:

1. Accessibility action when the element exposes a press, because it survives a window moving, a display changing scale or a Space switch.

2. Menu shortcut when the app binds the intent (save, find, print) to keys, which needs no target at all.

3. Scripting command when the app is scriptable and answers the intent directly, such as make new document in Pages.

4. Pointer click on the element's current center, only when nothing above is available, and always verified afterwards.

A click that changes nothing is reported as verified: no rather than as success. Pixel-only change is labelled as weak evidence. Focus drift and a window moving under a click get one automatic retry at the new bounds, and no more than one.

Documents

The document model says a paragraph exists; the page says whether it fits. Tasks can require a clean render with a pdf check, which is how a compile and verify loop refuses to call a document finished while a page is clipped.

Coding

code_loop is the goal engine: it runs the acceptance commands, prints the failing output and records the iteration. It records the machine's resource state with every run, because a benchmark taken at load 200 on eight cores measures the thermal state, not the code.

Browser

Clicks and typing are verified against the url and the page text. The browser is started with its own profile, separate from the one the person is signed into, so a page that needs their session has to be signed into once in that profile. Without a Chromium, engine=safari gives tab listing, navigation and page text, and says what else is unavailable instead of pretending.

Skills

Success rates are Laplace smoothed, so a new skill is not punished for having no history and one lucky run does not read as certainty. A compiled skill is a task, which means it inherits retries, checkpoints, approval gates and evidence.

System automation

mac_* tools cover the parts of a Mac that are not documents:

• mac_apps - what is running, what is in front, activate/launch/hide.

• mac_ui - System Events: windows, menu clicks, keystrokes. This is the long tail for apps with no scripting interface, and it needs Accessibility permission in addition to Automation. The error message tells you where.

• mac_system - clipboard text, notifications, volume, battery, disk, uptime.

• mac_finder - reveal, open, inspect, Trash.

• mac_shortcuts - list and run Shortcuts, optionally with a file input. Prefer this over UI scripting when a shortcut exists.

The mac_apps and mac_finder convenience tools favor explicit app activation and recoverable Trash operations. Other tools have broader authority: app_control can quit or force-quit applications, and shell access can modify or delete files. Restrict which agents can connect and review the per-user audit log. iwork_apps only queries already-running apps and does not launch them.

One-time permission

macOS blocks Apple Events until the host app is granted Automation access. The first call raises a prompt; if it was declined, every call returns -1743 / -10004 and pages_status prints the exact System Settings path plus the process chain, so you know which app to authorise:

System Settings → Privacy & Security → Automation → [host app] → Pages → on

Pages is addressed by bundle id (com.apple.Pages) rather than by name, so a renamed copy of the app still works.

Install and validate

Requires an Apple Silicon Mac with Node.js 20.11.1 or newer. Run from a checkout or unpacked package:

npm install
npm run build
npm run smoke:portability
npm run validate:release

npm run build typechecks and bundles the current source in dist/; it does not deploy or restart the running service. npm run deploy:local installs the bundle atomically into ~/.local/share/local-agent-mcp/dist/, retaining the previous installed file as index.js.previous when one exists. Deploying the file does not replace the code already loaded by a running Node process. Do not restart a tunnel carrying an active chat merely to update the bundle.

A checkout with Git metadata enables Git-aware project_context and project_review features. A source copy without .git can still build and run, but those review features cannot inspect history or diffs.

Connect ChatGPT through the Secure MCP Tunnel

Normal ChatGPT sessions do not connect directly to localhost on your Mac. The remote architecture is:

ChatGPT -> OpenAI Secure MCP Tunnel -> tunnel-client -> local-agent-mcp

The recommended private local target is the Streamable HTTP endpoint on 127.0.0.1:

npm run http
curl --fail http://127.0.0.1:3344/health

Use tunnel-client init --help to generate a profile for your own tunnel ID and endpoint, selecting --mcp-server-url http://127.0.0.1:3344/ for an HTTP-backed tunnel. The profile belongs in the tunnel client's user configuration directory. Refer to your tunnel provider's connection flow for the control-plane key and ChatGPT plugin binding. Never commit a real tunnel ID, credential, private profile or LaunchAgent environment file to the package.

A correctly supervised tunnel can run with a user-specific LaunchAgent. Provide credentials through a protected environment or secret file (mode 0600) and keep application-data directories restricted to the current user. The installer does not currently create the tunnel or register a LaunchAgent automatically; those are separate setup steps. The default HTTP endpoint is loopback only and must not be exposed directly to a network or public reverse proxy.

Stdio MCP remains supported for compatible clients, including local CLIs. If a stdio tunnel reports MCP server is not initialized, consult the initialization-specific troubleshooting guidance before changing transports. The MCP_STDIO_SEND_INITIALIZED_NOTIFICATION option is specific to tunnel-client; it is not required for HTTP-backed targets.

See Connection and runtime troubleshooting for diagnosing Session terminated, missing tunnel credentials, server exits, 503/413 responses, permissions, app discovery and browser CDP issues.

Deferred updates without cutting off ChatGPT

After validating a candidate, npm run update:install registers an independent per-user launchd watcher and npm run update:stage stages the bundle by hash. The watcher only activates it after at least 20 minutes without observed local/remote tool traffic. Only the separately supervised HTTP MCP server is restarted; the Secure MCP Tunnel stays alive. A watchdog cannot prove that an idle ChatGPT session has disconnected, so a request arriving during the small restart window may still need retrying. Failed health checks trigger a rollback.

See Deferred updates for the idle criteria, staging, rollback, status and cancellation commands.

Optional dependencies

Pages/Numbers/Keynote are needed for their native document tools. A supported Chromium-family browser and debugging target are needed for CDP automation. Python, MLX, Metal development tools, and Homebrew are optional and discovered at runtime when present. Features that rely on them should report a missing dependency instead of assuming an exact installation path or chip generation.

Chat acts during the turn: it can call tools, read the results and keep going until the answer is done, but it does not keep working after the turn ends. Work and Codex provide that background loop. This connector is local reach for Chat, not a second scheduler.

Audit log

The audit history is stored under the per-user application-data root. With the default macOS configuration:

tail -f "$HOME/Library/Application Support/local-agent-mcp/logs/audit.log"

Setting LOCAL_AGENT_MCP_HOME changes the application-data root. The audit log may contain file paths, command arguments and tool summaries: treat it as sensitive local history and exclude it from release archives.

Security notes

• No built-in HTTP authentication - the endpoint is loopback-only and rejects foreign Host/Origin headers. Do not pass --host 0.0.0.0 and do not port-forward it.

• The tools have full user-level permissions: anything your account can do, the connected agent can do. The audit log is your accountability trail, not a guardrail.

• ChatGPT's replies can be manipulated by content it reads on this machine (a webpage or file telling it to run a command). Treat the audit log as must-review after agent sessions, and run it as a non-admin user if you want a harder ceiling.

• Disconnect or disable the ChatGPT connector to revoke ChatGPT access. Stop the tunnel for an immediate remote cutoff; stopping only the local HTTP process does not revoke tunnel credentials. Keep audit history private.

Contributing and project policies

Chat Unleashed is licensed under the MIT License. The repository has a contribution guide, code of conduct, security reporting policy, support guidance and governance notes. Please use the GitHub issue and pull request templates when proposing changes, and avoid including secrets, personal data or unredacted audit logs in reports. CI checks the build, logic tests and release hygiene on Apple Silicon macOS; passing CI is not a replacement for real macOS permission and UI testing.

Development

npm run stdio   # stdio transport
npm run http    # Streamable HTTP on 127.0.0.1:3344
npx tsc -p tsconfig.json --noEmit   # typecheck