duet-mcp
This server lets an LLM and humans share and coordinate edits to a live GUI document over MCP.
Edit shared note text via
set_text, guarded by a requiredbaseRevision.Configure canvas presentation via
set_settings: caption, notes, visibility, grid, style, color, and opacity.Move or resize the canvas box via
set_boxwithin a 720×480 canvas, minimum 64×64.Detect stale edits using
baseRevision; conflicts return the latest document instead of running the handler.Get the human-facing GUI URL via
gui_url.Observe the current document and wait for changes or specific ops via
await_change.Capture a PNG screenshot of the rendered GUI, optionally at a given path, via
render_screenshot.Read stored blobs by ID via
read_blob, returning images as images when applicable.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@duet-mcpChange the document text to 'Q3 goals'."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
duet-mcp
English | 日本語
One shared state for humans through a GUI and LLMs through MCP. Define each operation once.
Start a project
Requires Node.js 22+.
npx --package=duet-mcp duet init my-app
cd my-app && npm install
npm run devOpen the URL printed by Vite. Edit text or drag the box; the shared state updates for connected clients. duet init creates the template, build configuration, dependencies, and scripts in an empty directory.
Read template/app.ts for shared state and operations, then template/ui/main.tsx for the screen. Add an action in app.ts and call it from the GUI. Keep the template's rootDir/webDist settings when editing it.
Related MCP server: Express REST API and MCP Server Framework
Define an action once
import { defineApp, createAction } from "duet-mcp";
import { appInfo, awaitChange, observerOps } from "duet-mcp/assets";
import { z } from "zod";
type Doc = { text: string };
const action = createAction<Doc>();
export const app = defineApp({
id: "notes",
version: "1.0.0",
initialDoc: (): Doc => ({ text: "" }),
actions: {
set_text: action({
description: "Replace the shared text.",
input: z.object({ text: z.string() }),
handler: ({ doc }, { text }) => {
doc.update(state => { state.text = text; });
return { text };
},
}),
url: appInfo().url,
await_change: awaitChange(),
...observerOps(),
},
});The handler's document, input, and result are typed. Custom actions and built-in actions share the same actions object.
In the GUI, import useDoc from ../duet/browser, call const doc = useDoc(), and handle the initial null. Then call await doc.set_text({ text: "hello" }). A Node caller imports getDoc from ./duet/node and calls await getDoc() first. MCP takes the same { text: "hello" } input.
Keep drafts in React useState / useRef. Clear a draft only after its action succeeds; use the value calculated in the release handler when submitting a drag.
Connect an MCP client
Run npm run build, then configure your MCP client to launch node with the absolute path to dist/template/main.js. The template provides GUI URL, observation, screenshot, blob, and undo/redo actions. Screenshots automatically use installed Chrome, Edge, then Playwright Chromium. If none is installed, run npx playwright install chromium. Use renderScreenshot({ channel: "chromium", timeoutMs: 10_000 }) to select a browser and per-step capture timeout.
Use npm run typecheck to check your app. Generated template/duet/browser.ts, node.ts, and connection.ts need no manual editing.
State lives in memory; your application owns persistence. For update behavior, observation and guarded edits, assets, and takeover, see doc/SEMANTICS.md.
Available Tools
7 toolsawait_changeawait_changeA
省略すると現在の doc と revision を即時取得する。sinceRevision は観測した文字列をそのまま渡す。その版以後のコミットを待ち、既に変更があれば即返す。until は待つ op 名を絞る。全応答に doc が載る。truncated は差分を説明できないという意味。doc を確認すること。再起動や履歴の保持範囲外でも最新 doc を即返す。timedOut の場合も doc と revision を組で読む。changes は対象 op の変更説明で、連続した同じ参加者の同じ op は count にまとまる。activity は最終活動からの経過ミリ秒で、編集完了や優先権は保証しない。activity 自体では起床しない。
Inputs: until, timeoutMs, sinceRevision.
gui_urlgui_urlA
人が操作する GUI の URL を返す。ポートは app.id から導出されるので、人に「どこで開いているか」を聞かれたらこれで答えること。
Inputs: no named parameters.
read_blobread_blobB
blob を読む。画像なら画像として返す。id は doc から得る。
Inputs: id (required).
render_screenshotrender_screenshotA
同じ文書を GUI の別セッションで描画した PNG を返す。人間の下書きやスクロール位置は共有しない。path を渡すとその画面へ移動してから撮る(GUI が解釈する URL をそのまま書く)。
Inputs: path.
set_boxset_boxA
Move or resize the box within a 720×480 canvas. The minimum size is 64×64. baseRevision は意図を決めるために観測した revision(文字列)をそのまま渡す。版が変わっていたら handler を実行せず conflict と最新 doc を返す。最新 doc を見て意図を見直すこと。同じ引数を新しい版で自動再送しない。通信エラーは適用結果が不明な場合がある。まず await_change で現在の状態を確認する。
Inputs: x (required), y (required), width (required), height (required), +1 more.
set_settingsset_settingsA
Set the canvas heading, notes, visibility, grid, style, color, and opacity. baseRevision は意図を決めるために観測した revision(文字列)をそのまま渡す。版が変わっていたら handler を実行せず conflict と最新 doc を返す。最新 doc を見て意図を見直すこと。同じ引数を新しい版で自動再送しない。通信エラーは適用結果が不明な場合がある。まず await_change で現在の状態を確認する。
Inputs: grid (required), color (required), notes (required), style (required), +4 more.
set_textset_textA
Replace the shared note text. baseRevision は意図を決めるために観測した revision(文字列)をそのまま渡す。版が変わっていたら handler を実行せず conflict と最新 doc を返す。最新 doc を見て意図を見直すこと。同じ引数を新しい版で自動再送しない。通信エラーは適用結果が不明な場合がある。まず await_change で現在の状態を確認する。
Inputs: text (required), baseRevision (required).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
7 tool updates
v0.6.1- First observed
await_change - First observed
gui_url - First observed
read_blob - First observed
render_screenshot - First observed
set_box - First observed
set_settings - First observed
set_text
TDQS
Scored across 7 tools
set_text and set_settings overlap somewhat because set_settings can update 'notes', which could be confused with the note text set_text replaces. The other tools are clearly distinct, but this boundary ambiguity makes mis-selection possible.
Most tools follow a clean verb_noun pattern: set_text, set_settings, set_box, await_change, render_screenshot, and read_blob. gui_url is the outlier and would be clearer as get_gui_url, but the overall convention is still predictable.
Seven tools is well-scoped for this collaborative canvas server. Each tool serves a distinct role: mutating text/settings/box, observing changes, rendering views, and reading blobs.
The surface covers the main observe-modify-render workflow: await_change provides current state and revision, set_* tools make changes, and render_screenshot/read_blob provide outputs. Minor gaps exist around explicit create/delete operations for boxes or blob uploads, but they may not be required by the intended domain.
Maintenance
Related MCP Connectors
Hosted MCP server for AI-driven data ops. Create apps, manage schemas, and CRUD structured data.
Build and publish full-stack apps from your coding agent: models, rules, pages, auth, per-app MCP.
Build, validate, deploy — HTTP APIs, cron jobs, webhooks and MCP tools — from your AI client.
Host any HTTP API as MCP tools with a tamper-evident cross-agent settlement ledger.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceMCP server enabling AI agents to collaborate in shared rooms with a document edited via compare-and-swap optimistic concurrency, plus a chat channel and change deltas.34 npmMIT
- AlicenseNot gradedqualityAmaintenanceEnables LLMs to discover and invoke generated API actions as MCP tools and read context through MCP resources for list and get-by-id operations. It provides HTTP-based MCP access for clients such as Cursor and Claude without extra glue code.MIT
- AlicenseNot gradedqualityAmaintenanceEnables LLMs to discover and invoke generated CRUD operations as MCP tools and read data through MCP resources, alongside standard REST endpoints, using one codebase.MIT
- AlicenseNot gradedqualityBmaintenanceEnables LLMs to interact with generated CRUD services through MCP tools and resources, so they can create, update, list, and fetch records via a standard protocol. It also provides REST API access from the same codebase for traditional clients.MIT