Summary: mcp-screen gives a terminal coding agent a local, browser-based infinite canvas to show visual artifacts, read back what the human points at, ask visual questions, and post standup reports.
Place widgets on the canvas — markdown, text, tables, bar/line charts, metrics, key/value, status lists, and architecture livemaps (with live file-edit highlighting).
Update or remove widgets — shallow-merge data (or replace with merge:false), resize, retitle, or stop live refresh.
Read widgets back — including the human's selection (clicked table row, chart bar, MCP App message) and unmappedFiles for livemaps.
Inspect the whole canvas with read_canvas to list every widget, kind, title, live status, and selection.
Clear the canvas entirely, with an undo option (standup cards are kept).
Ask visual questions — 2–8 clickable choice cards (optional descriptions/previews) that block until the human clicks or the timeout passes, returning the choice or answered:false.
Post standup reports — cards with summary, changed files, test results, and risks, grouped by worktree/agent.
Run live widgets from a command, URL, or MCP tool source that refreshes on an interval with no LLM in the loop — nothing runs until the human approves it.
Supports Amp as a terminal agent client, registered via amp mcp add screen -- npx mcp-screen or the amp.mcpServers entry in ~/.config/amp/settings.json, so the agent's tool calls can place widgets, ask visual questions and post reports to the shared canvas.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@mcp-screen chart last quarter's revenue, then ask me which region to focus on"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An infinite canvas next to your terminal. Your agent puts charts, diffs, flows, logs, plans and maps on it. You circle the bar that looks wrong and type "why?", and the agent reads exactly what you pointed at. Local, one command, no API keys.
claude mcp add screen -- npx mcp-screenThe agent in this clip is scripted (labelled on screen): no model is running. Everything else is real. Each widget comes from a real MCP tool call to a real mcp-screen process, the circle is drawn with real mouse input, and the caption quotes what read_canvas actually returned (scripts/record-hero.mjs, also as hero.mp4). The older demo of ask_user and the agent theatre is demo.gif.
Why · Features · Install · Tools · Security · Contributing
Claude Code, Codex, opencode, Amp, Cursor CLI and Gemini CLI live in a terminal, and a terminal can't show a chart, a table, or the UIs that MCP servers now ship as MCP Apps. Those UIs reach a terminal agent as a wall of JSON, and when the agent needs a decision you get a numbered list to type back into.
mcp-screen opens a browser tab next to your terminal and gives the agent somewhere to put things, and gives you something to click. It runs entirely on your machine (127.0.0.1, token-protected, nothing uploaded).
First time? Run npx mcp-screen serve, then click Try a sample canvas to play with it before wiring up an agent. Press ? for shortcuts. It stays smooth with 100+ widgets (off-screen cards are culled; measured budgets in QA.md).
Related MCP server: ui-mcp
One canvas, 17 widget kinds. The yellow circle ("why?") has the agent's reply threaded under it; the sticky note on the diff is still waiting. Circled data points stay attached when you scroll the card. Screenshot from scripts/e2e.mjs.
Canvas
place_widget charts, tables, metrics, markdown and status lists. Clicks flow back: the agent can read_widget and see the bar or row you picked.
Visual questions
ask_user puts choice cards on screen and blocks until you click one. The agent gets your choice back.
Agent theatre As agents read, edit and create files, their orbs glide across a live architecture map, files fly in and modules glow.
Standup board
Claude Code Stop/SubagentStop hooks turn each finished run into a card: files changed, tests, summary, risks.
MCP Apps for terminal agents
mcp-screen wrap -- <server> renders every ui:// result in a null-origin sandbox.
Live widgets
A widget can carry a command/URL/tool source that refreshes with no LLM in the loop, after you approve it.
Plus Share (PNG/HTML of the whole canvas, filename includes the project; Copy link opens that named canvas on localhost; Notifications can mute a canvas or set quiet hours), named canvases (git root by default, header switcher), and Clear (confirm + undo). Circle a bar or long-press / Alt+click a row to leave a note; the agent answers in a thread (reply_annotation). With install-hooks, every prompt injects “N notes waiting on the canvas” when you left questions unanswered.
The canvas opens at http://127.0.0.1:4747. Several agent sessions share one canvas: the first one starts it and the others attach.
claude mcp add screen -- npx mcp-screen
npx mcp-screen install-hooks --write # optional: standup, theatre, and “notes waiting” on each promptcodex mcp add screen -- npx mcp-screen, or ~/.codex/config.toml:
[mcp_servers.screen]
command = "npx"
args = ["mcp-screen"]opencode.json:
{
"$schema": "https://opencode.ai/config.json",
"mcp": { "screen": { "type": "local", "command": ["npx", "mcp-screen"] } }
}amp mcp add screen -- npx mcp-screen, or ~/.config/amp/settings.json:
{ "amp.mcpServers": { "screen": { "command": "npx", "args": ["mcp-screen"] } } }~/.cursor/mcp.json, or .cursor/mcp.json in a project:
{ "mcpServers": { "screen": { "type": "stdio", "command": "npx", "args": ["mcp-screen"] } } }Check it with cursor-agent mcp list. A project-level server may show as needing approval; run cursor-agent mcp enable screen.
gemini mcp add -s user screen npx mcp-screen # writes mcpServers to ~/.gemini/settings.jsonGemini CLI disables MCP servers in folders you haven't trusted, so trust the project folder when it asks. gemini mcp list should show screen as Connected.
How these were checked (Oct 2026): opencode ran end to end with a free model. For Amp, Cursor CLI and Gemini CLI, the configs above were written (via amp mcp add and gemini mcp add for those two); Cursor (cursor-agent mcp list-tools screen) and Gemini (gemini mcp list → Connected) started the server and saw its tools. Prompting a model through Claude Code, Codex, Amp, Cursor and Gemini needs an account login, which we didn't do here.
Tool | What it does |
| Put a widget on the canvas (kinds below). Optional |
| Change a widget; shallow-merges |
| Read a widget, including your |
| List every widget (with |
| List named canvases (one per project) and switch the browser to one. Default name = git root folder, or |
| Answer (or follow up on) a human's note/circle. The reply appears as a thread bubble next to their mark. |
| One-click: answer the oldest unanswered note (optional |
| remove_widget | Remove one widget. |
| clear_canvas | Remove everything (undo: true brings it back). |
| ask_user | Show 2–8 choice cards and wait for your click; returns the choice, or answered: false with a reason. |
| ask_form | Show a small form (text, textarea, number, slider, select, checkbox) and wait for Submit; values are validated before the agent gets them. |
| tidy_canvas | Re-pack all widgets into a neat grid (same as the Tidy up button). |
| post_report | Add a card to the standup board. |
With npx mcp-screen install-hooks --write, Claude Code also gets a UserPromptSubmit / SessionStart hook: when notes are waiting, each prompt injects them as additionalContext so the agent can call reply_annotation without being asked again.
kind | data |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| created by |
The canvas is infinite. Pan by dragging the background or scrolling, zoom with Ctrl/⌘+scroll or +/−, F fits everything, and Tidy up re-packs the grid. Drag a card by its header to move it; positions are kept by the hub and show up in read_canvas. The minimap (bottom right) jumps anywhere.
The left toolbar lets you talk back with ink: arrow (A), circle (O), highlighter (H) and sticky note (N), in five colours. Each mark is attached to the widget it lands on, plus the data points under it (bars, rows, log lines, diagram nodes, map markers, diff lines…), so the agent sees for example {kind:"circle", text:"why?", target:{widgetId:"ci-chart", dataPoints:[{label:"10/09", value:9}]}}. Select a mark and press Delete to remove it. It's deliberately not a whiteboard: marks exist to point the agent at things.
ask_user { "question": "Which pricing layout should I build?",
"choices": [ { "id": "tiers", "label": "Three tiers", "description": "Good / better / best", "preview": "- Free · Pro · Team" },
{ "id": "slider", "label": "Usage slider" } ],
"timeoutSeconds": 180 }
// → { "answered": true, "choice": { "id": "tiers", "label": "Three tiers" }, "widgetId": "question-…" }2 to 8 choices, each with an optional description and markdown preview. The call blocks (sending MCP progress notifications every 10 s) until you click, the timeout passes (default 3 min, max 15), the card is removed, or the agent cancels the call. It then returns answered: false with the reason.
Ask: "draw a livemap of this repo with place_widget, one box per module, with globs". Once the hooks are installed, every Read, Write and Edit plays out live: the agent's orb moves to the module, the file chip flies in and lands with a ripple, the box glows (blue for reads, warm for edits, a NEW badge for created files), and a caption says who is doing what. Theatre mode hides the side panel for a full-screen view. If you installed hooks before this version, re-run install-hooks --write so the PostToolUse matcher includes Read. read_widget on the map returns unmappedFiles, the edited files no box covers, so the agent knows when to redraw.
wrap)claude mcp add flaky -- npx mcp-screen wrap -- node examples/demo-app-server.mjsThe agent sees the wrapped server's tools unchanged, plus the screen tools. When a tool declares _meta.ui.resourceUri, mcp-screen fetches the ui:// resource and renders it on the canvas. The bridge handles ui/initialize, tool-input and tool-result notifications, tools/call and resources/read proxied back to the server, ui/open-link, ui/message and ui/update-model-context. Whatever the app reports becomes the widget's selection, so the agent can read it.
place_widget { "kind": "status-list", "title": "Pipelines", "data": { "items": [] },
"source": { "type": "command", "argv": ["gh", "run", "list", "--json", "..."], "parse": "json" },
"intervalMs": 30000 }The widget appears with an Approve button that shows the exact command. Nothing runs until you click it. Approved sources are saved to ~/.mcp-screen/allowlist.json. Commands run without a shell, with a timeout and an output cap. Stop revokes the approval. See SECURITY.md.
npx mcp-screen install-hooks --write # merges Stop / SubagentStop / PostToolUse hooks into ./.claude/settings.json
# (--local points them at a local checkout instead of npx)When an agent or subagent stops, the hook reads git diff --numstat (plus untracked files) in its worktree. The summary and risks come from Claude Code's last_assistant_message. Subagent cards are named by agent_type, and the internal agents Claude Code runs (with an empty agent_type) are skipped. The hook also reads the transcript for the last test command and its result (vitest, jest, pytest, go and cargo output are parsed), falling back to the transcript for the summary if needed. These become a card. Agents can also call post_report themselves.
mcp-screen stdio MCP server + canvas
mcp-screen wrap -- <cmd...> proxy an MCP server and render its MCP Apps
mcp-screen serve canvas only
mcp-screen hook hook entry point (reads the hook JSON on stdin)
mcp-screen install-hooks [--write]
--port <n> (default 4747) --no-opennpm install
npm run build
npm test # vitest unit tests, one file per part
npm run screenshots # end-to-end: real stdio MCP client, hooks, Playwright screenshots + GIF
npm run demo # records the ask_user + theatre demo: screenshots/demo.gif + demo.mp4
node scripts/record-hero.mjs # records the README hero: screenshots/hero.gif + hero.mp4 + hero.png
npm run ux # first-timer checks at 4 viewport sizes (screenshots/ux/)
npm run soak # 25 clear/undo/re-place cycles through the UI; DOM + heap must stay flat
npm run perf # performance budgets with 120 widgets (see QA.md)
npm run interop # renders the official ext-apps basic-server-vanillajs example through wrapSee CONTRIBUTING.md. The design reuses pieces of opencanvas: the fan-out event bus, the widget-registry idea (kind → schema → renderer) and the null-origin plugin sandbox. It leaves out tldraw for a lighter grid canvas and a zero-build browser client.
The UI uses the Inter variable font (SIL OFL 1.1, public/fonts/LICENSE-Inter.txt), bundled locally so nothing loads from the network.
v0.1, early. Named canvases live under ~/.mcp-screen/canvases/<id>.json (legacy canvas.json is migrated) and come back after a restart (MCP_SCREEN_NO_PERSIST=1 turns that off). Known gaps: diagrams cover Mermaid flowcharts only; maps are an offline world outline (no tiles, no zoom inside the map); PNG export of an embedded MCP App shows a placeholder, because the sandboxed frame can't be captured; MCP-tool live sources and app tools/call only work in the session that runs wrap and owns the canvas; no single-binary build yet.
mcp-screen binds to 127.0.0.1, needs a per-run token, sandboxes MCP Apps in a null origin, and never runs a live source until you approve it. Details and known limits: SECURITY.md.
If mcp-screen helps your agent show its work, give it a star so other people can find it.
MIT © Krunal Ashar
Ask the human to fill in a small form on the canvas and BLOCK until they press Submit (or the timeout passes). Field types: text, textarea, number, slider (needs min/max), select (needs options), checkbox. Returns {submitted:true, values:{fieldId: value}} or {submitted:false, reason}. Values are checked against the fields (required, min/max, options) before you get them.
Inputs: fields (required), prompt (required), detail, submitLabel, +1 more.
Ask the human a visual question: places clickable choice cards on the canvas and BLOCKS until they click one (or the timeout passes). Use it for decisions that are easier to make by looking: layouts, copy options, which bug to fix first. Returns {answered, choice:{id,label}} or {answered:false, reason}.
Inputs: choices (required), question (required), detail, timeoutSeconds.
Remove every widget from the canvas (standup cards stay). The human can undo it from the canvas, or you can call clear_canvas with undo:true right after.
Inputs: undo.
List named canvases (one per project). The default name is the git root folder. Switch with open_canvas; place_widget takes an optional canvas argument.
Inputs: no named parameters.
Switch the browser to a named canvas (creates it if needed). Pass the project name or an id from list_canvases.
Inputs: id (required), title.
Show something to the human on their mcp-screen canvas (a browser tab next to the terminal). Returns the widget id. Widget kinds and their data:
markdown {markdown}
text {text} (monospace log/output)
table {columns: string[], rows: cell[][]} (sortable; row clicks become the widget selection)
chart {type: "bar"|"line", labels: string[], values: number[], unit?, linkTo?: tableId} (bar clicks become the selection and filter the linked table)
metric {value, label?, delta?}
kv {items: {key: value}}
status-list {items: [{label, status: "ok"|"fail"|"pending"|"unknown", detail?}]}
diff {before, after, path?, language?} or {patch: "unified diff / git diff text"}, mode?: "split"|"unified" (syntax highlighted)
diagram {mermaid: "flowchart TD A[Start] --> B{Ok?} B -->|yes| C"} (Mermaid flowchart subset) or {direction?: "TD"|"LR", nodes: [{id, label, shape?}], edges: [{from, to, label?}]}
image {path: "/abs/file.png"} (local png/jpg/gif/webp/svg, served safely) or {url: "https://..."}; alt?, caption?
log {lines: string[], maxLines?} (terminal tail; update_widget {data:{append:["more lines"]}} appends; a live command/url source with parse:"text" tails it)
checklist {items: [{id?, label, status: "todo"|"doing"|"done"|"failed"|"skipped", detail?}]} (progress bar; update_widget {data:{set:{"": "done"}}} changes single items)
code {code, language?, filename?, startLine?, highlight?: lineNumbers[]} (syntax highlighted, copy button)
map {markers: [{lat, lon, label, value?, color?}], bounds?: [west, south, east, north]} (offline world map, no tiles)
timeline {items: [{at: ISO date or label, title, detail?, status?: "ok"|"fail"|"pending"|"info"}]}
livemap {root?, nodes: [{id, label, globs: string[], layer?, description?}], edges: [{from, to, label?}]} (architecture diagram; file edits matching a node's globs light it up live)
Inputs: data (required), kind (required), title (required), h, +7 more.
Post a standup card when you finish a task: what you changed, test results, a short summary and risks. Cards are grouped by worktree and agent on the Standup board.
Inputs: summary (required), agent, risks, tests, +3 more.
List every widget on the canvas with its id, kind, title, live status, selection and position/size in pixels, plus every annotation the human drew (kind arrow|circle|highlight|note, text, replies, and target: the widget and data points it points at). pendingQuestions lists marks with a question you have not answered yet: answer each with reply_annotation. Check this when the human says "look at this" or "see my note".
Inputs: canvas.
Read a widget back, including selection: what the human clicked (table row, chart bar) or what an MCP App reported, and annotations: arrows, circles, highlights and sticky notes the human drew on it, with text (their note) and target.dataPoints (e.g. the chart bar they circled). For livemaps, unmappedFiles lists edited files no node covers (time to update the map).
Inputs: id (required).
Remove a widget (and stop its live refresh).
Inputs: id (required).
Answer a question the human left on the canvas (a circle, arrow, highlight or sticky note with text). Your reply appears in a thread bubble next to their mark. read_canvas lists unanswered ones under pendingQuestions, with the widget and data points they point at; answer about exactly that. A human follow-up in the thread makes it pending again.
Inputs: id (required), text (required).
One-click: answer the oldest unanswered note on the canvas (see read_canvas → pendingQuestions). Use when the human said "answer my notes" or a hook told you notes are waiting and you can answer the first one now. Optional canvas limits to that project.
Inputs: text (required), canvas.
Re-pack every widget on the canvas into a neat grid (same as the Tidy up button). Returns the new positions.
Inputs: columns.
Update an existing widget. data is shallow-merged into the current data by default (send only the keys you change, e.g. {edges} on a livemap); pass merge:false to replace it entirely. Shortcuts: log widgets take data.append (lines to add); checklist widgets take data.set ({"": "done"}). x/y move it on the canvas; changing w/h pushes covered widgets down.
Inputs: id (required), h, w, x, +4 more.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Scored across 14 tools
Most tools target clearly different objects or actions: widget CRUD, canvas management, annotations, and blocking human questions are separated well. A couple of pairs are close (read_canvas vs read_widget, reply_annotation vs reply_oldest_pending), but the descriptions make their scope distinction clear.
All tool names use consistent snake_case and mostly follow a verb_noun or verb_phrase pattern (place_widget, update_widget, read_canvas, reply_annotation). There is no mixing of camelCase or vague standalone verbs.
14 tools is well within the expected range for a canvas interaction server and each tool covers a distinct part of the workflow: canvas selection, widget display/editing, human input, annotation replies, and reporting. No obvious padding tools.
The surface covers core widget lifecycle operations (place, update, read, remove, clear, tidy), canvas switching, annotation reading/replying, and blocking human input. Minor gaps remain around full canvas lifecycle management, such as deleting or renaming named canvases.
Live, editable canvas for AI agents — author graphs via MCP, then watch and reroute runs live.
Display delivery platform for AI agents. Push HTML, dashboards and live data to screens.
Deploy AI-generated HTML/CSS/JS to instant public HTTPS URLs from any MCP-compatible agent.
Charts and dashboards as images for AI agents. No browser, same data same bytes. Local and free.
