Start here. Connection, app version (and whether it is the tested build), open editor windows, recording state, ffmpeg / edge-tts / input helper and the Accessibility and Screen Recording permissions, supported features, the named recipes and the saved brand kits. Works when the app is closed.

Launch Screen Studio with a local automation connection (bound to 127.0.0.1). A running app is never killed or restarted.

Quit Screen Studio like Cmd+Q when the work is done, which also closes the local automation connection. The app asks about unsaved changes itself; refuses while recording.

List the person's saved Screen Studio presets, or apply one to a project file (the app's own preset apply). Apply to a project that is not open in the editor: an open editor keeps its old settings and its next save writes them back over the preset.

Everything Screen Studio ships that settings can point to: wallpapers (styles.background.systemName), cursor sets (cursor.set.id), device frames (device.frameKey), music tracks, narration voices, camera layout types, mask types, looks and output aspects.

List capturable windows, display IDs, microphones and cameras.

Inspect the current recording session. Null means idle.

Start recording one exact window, a display, or an area of a display (displayId plus area in display points). Audio and camera default off. Rehearse first; record each beat with screenstudio_desktop_perform.

Pause, resume, add a marker, restart (discard and record again), cancel (discard), or finish and return the saved project. Idle time between beats is cut later by the edit plan, so pausing for short thinking is unnecessary.

Inspect an exact native window and get a 60-second, single-use observation token.

Screenshot the target native window before deciding where to act. Scaled to width points wide (default 1280) so large windows stay readable and small.

One visible native step on an observed window: focus, move, click, doubleClick, drag, taps (a burst of count taps every intervalMs at one point, for mash buttons), hold (press for ms, then release), type, key (with modifiers, e.g. {key:'k',modifiers:['command']}; keys follow the person's keyboard layout) or scroll (optional x/y to scroll there; otherwise the pointer stays if it is over the window or moves to its middle). Coordinates are window-local points including the title bar. Pointer glides at a human pace and settles before clicking. Input stops if the window loses focus, another window covers the point, or the person moves the mouse. Terminals, password managers and System Settings are off limits unless SCREENSTUDIO_ALLOW_APPS names them; system shortcuts (command+space, command+tab, command+q) are never sent.

Perform a whole beat in one take while recording: up to 12 steps, each followed by a hold so its result is visible (defaults: click 1.2s, type 0.9s, key 1s, scroll 1.2s). One continuous take gives natural pacing; pace 1.3 is calmer. markers:true adds a recording marker before the first step, so screenstudio_analyze and screenstudio_narrate (pinToMarkers) know where the beat starts. minDurationMs stretches the last hold so the beat lasts at least that long (a narration line's durationMs from screenstudio_voice_lines plus about 400ms).

Copy a project bundle to a fresh destination, to keep the original untouched.

Read what happens in the raw recording, in source ms: clicks and drags (0-1 frame coordinates), typing bursts, shortcuts, cursor travel, screen changes (page = whole view replaced), idle stretches, recording markers (one per beat when recorded with markers, mapped to source ms; a marker dropped while paused lands where recording resumed) and, with a microphone, the audio valley between each pair of sentences (where the plan may cut when the transcript shows no pause). Typed text is omitted unless includeText is true.

See raw recording moments (source ms) before deciding what to keep or where to zoom. Up to 8 frames; clicks within 400ms are marked with a red box.

Propose an edit from the footage (read the craft resource for taste): keep each beat with lead-in and a readable hold, cut dead air, speed up only typing and waiting, and add few long zooms grouped like Screen Studio's auto-zoom (clicks under 5.3s apart share one zoom that pans). Returns ops ready for screenstudio_editor_apply, its pacing check, scored beats and director's notes (each decision at its playback time with the rule behind it). Styles: balanced (default, product demos), calm (docs and tutorials), snappy (short social clips). keep/drop take source-ms spans to override (keep also protects narrated stretches). Speech from the transcript is never cut or sped up; tightenPausesMs (250-400 for talking videos) trims silences between phrases, removeFillers drops um/uh. speedUps:false for narrated videos. look applies a starting frame (wallpaper, gradient, dark, light, minimal, social; none is a default, choose for the content; keep leaves the project's own frame, even under a recipe); aspect sets the output frame. targetMs fits the edit to a length by dropping the lowest-value beats (never the payoff), listed in droppedBeats; structure hook-demo-payoff also suggests a coldOpen. markers (keep by default when the recording has markers): keep starts a beat at each marker and protects it, retake drops the take before a repeated line, chapters numbers the beats. markerBeatsMs (one length per marker, a narration line's beatMs) keeps [marker, marker + ms] so a hold stretched for a voiced line survives. talkingHead plans camera layouts too (skipped when the recording has no camera). recipe (keynote, social-vertical, changelog, founder-talking-head, tutorial, launch-teaser, docs-walkthrough; see screenstudio_recipes) sets style, look, aspect, plan options, captions, cursor and click sound in one word, and returns its music, voice and export settings; explicit arguments win over the recipe, and the notes say what replaced what. No recipe is applied unless named. brand (a kit saved with screenstudio_brand, or the recipe's brand) lays its own colours, captions, cursor and sound over the plan's config (under a recipe or look, its starting look is left out), and its voice and music replace the recipe's where the recipe has them.

Named recipes: one word for a whole direction (pacing style, frame, aspect, plan options, captions, cursor, click sound, music, voice, export). Without a name, lists each with when to use it; with a name, returns the full recipe with any overrides merged and what it sets in plain words; with brand (or overrides.brand), the saved kit's own settings, voice and music laid over it. None is a default: pick the one that fits the content and where it will be watched, then pass it as recipe to screenstudio_plan_edit.

Camera director for talking-head recordings: plans camera layouts from the speech, clicks and zooms. The cutout camera is the default layout, on the side away from the clicks; you full screen for the opening line and the sign-off; screen only while a zoom asks the viewer to read; split screen under long talk with nothing clicked. Every change lands in a pause between sentences, at least 5s apart (minStretchMs when longer). from timeline (default) plans over the open editor's current cut; from plan plans over a fresh screenstudio_plan_edit plan, so pass the same recipe and plan options you applied it with. Returns ops for screenstudio_editor_apply (clears the layouts track first), the stretches, a layout check and notes saying why. A recording with no camera gets no layouts.

Judge a timeline the way a viewer feels it, in playback time: short or back-to-back zooms, zoom rate, choppy cuts, speed ramps, sped-up clicks, snappy springs, abrupt endings, camera layouts changing too often or mid-sentence, plus taste checks on the look (frame cropping, cursor flicker, loupe depth and overuse, loud click sounds, heavy blur). With recipe (plus the recipeOverrides and brand you planned with), also flags settings that work against it (off-recipe). Uses the open editor's live state when available.

List projects open in Screen Studio editor windows.

Open a project in a Screen Studio editor window (or find it if already open) and return its timeline. The person can watch every edit happen there.

The open editor's live timeline (slices, zooms, camera layouts, masks in source and playback ms), playhead, undo state, a style summary and, with includeConfig, every project setting (crop, cursor, captions, audio, camera, animations, zooms, defaultLayout, styles, device, output). Any setting shown can be changed with a config op.

Edit the open project live in the editor window. Ops run in order, all times in source ms: setSlices (full ordered list of kept ranges with speed, optional per-clip volume, hideCursor, disableSmoothMouseMovement), addZoom (follow:true pans to clicks, otherwise target x/y 0-1; presentation 'loupe' for the glass loupe with optional loupe radius/bevel/chromaticAberration/glassOptics), updateZoom, removeZoom, clearZooms, addLayout (camera layout for a stretch: camera-overlay, cutout-camera, split-screen, fullscreen-camera, screen-only, with options), addMask (sensitive-data blur or highlight, rects 0-1), updateItem/removeItem/clearTrack (layouts, masks, zooms; updateItem on a mask takes fields.rects 0-1 like addMask, or fields.bounds in capture points as editor_state shows them), config (any dotted setting from editor_state includeConfig, e.g. crop.rect01, captions.position01, defaultLayout.type, camera.background.edgeFalloff01, styles.background.systemName, device.frameKey). Timeline ops also include splitAt, cutRange (split both ends and delete the middle), removeSlice, mergeSlices, updateSlice (speed, volume up to 1, system/device audio, hideCursor, disableSmoothMouseMovement), resetCuts, restoreAutoZooms (Screen Studio's own click zooms), duplicateItem, setTrackDisabled, and clearTrack/removeItem for voiceOvers. With show (default) every step plays out in the editor window wherever it is, without taking focus: the playhead jumps to each edit, the timeline is split and pieces deleted one by one, new zooms, layouts and masks drop in selected, and each setting changes with its sidebar panel open (stepMs per step, 350 by default; show:false applies instantly). Returns a checkpoint, the new timeline and a pacing and taste check. Cmd+Z in the app also works. Saves after each apply unless save:false. If an op fails, the result has partial:true, failedAt and error; the ops before it stay applied, nothing is saved, and screenstudio_editor_restore with the checkpointId undoes the batch.

Put the editor back to the timeline and settings saved in an apply checkpoint.

Step the editor's own undo/redo history, like pressing Cmd+Z.

Drive the editor interface: open a sidebar panel (background, cursor, captions, transcript, shortcuts, audio, animation), select a timeline item (zooms, masks, layouts, slices by id), set the visible timeline span in ms, show or hide the sidebar and timeline, or switch to preview-only. Changes nothing in the project.

Save the open project exactly like Cmd+S in the editor.

Capture the editor's rendered preview at playback times (up to 6) to check framing, zoom targets and legibility without exporting. The editor window must be on screen.

Move the editor playhead to a playback or source time.

Play a range in the editor window so the person can watch it.

Voice over the video. Each line is spoken by a neural voice (edge-tts, an online Microsoft voice service, so the line text leaves this Mac) and pinned to a moment: sourceMs (stays on its moment through later edits) or playbackMs. pinToMarkers puts each line without a time 300ms after its recording marker (line i on marker i). Lines are mixed, optionally over Screen Studio library music ducked under speech (e.g. 'commercial/Product Uplift'), and attached as the project's audio track through the app. captions:true captions the voice: as Screen Studio's own captions when the recording has no microphone, otherwise as .srt and styled .ass files next to the project for burn-in. Returns each line's playback start/end, notes on what it chose and warnings when a line runs into the next or past the end. Call again without lines after changing the cut to re-sync. Screen Studio's own AI and recorded voiceover tracks are disabled in this app build. Re-sync keeps the last voice, rate, music, volumes and captions unless you pass new ones; music laid by screenstudio_music stays under the voice unless music is null.

Voice a script before recording, without a project: returns each line's spoken durationMs and beatMs (line plus 400ms) so every beat can be recorded to last as long as its line (screenstudio_desktop_perform minDurationMs, with markers:true). Uses edge-tts (online; the text leaves this Mac). The clips are cached, so screenstudio_narrate with the same text, voice and rate reuses them.

Lay a track from Screen Studio's music library under the edited video, faded in and out. duckUnderSpeech dips it smoothly while anyone speaks (from the transcript). Uses the project's background audio track, shared with narration (narration has its own music option). If the track holds narration, the music is mixed under it instead of replacing it.

Transcribe the microphone with Screen Studio's own recognizer. Default is the on-device system recognizer (no upload); pass a generator like {type:'whisper', model} only when the person wants it (it can download a model). Captions come from this transcript. Replaces the current transcript, including caption fixes made with screenstudio_transcript_edit. editor is 'updated' when the open editor now shows it, or 'not-open'.

Read the transcript: every word with its index and source ms, phrases (speech separated by pauses), filler words. Does not generate or upload anything.

Fix the transcript the captions show: correct a misheard word or name (keep the word's leading space), or remove words from the captions. Indexes come from screenstudio_transcript_read. Updates the open editor live; editor is 'updated' when it now shows the change, or 'not-open'.

Render what the editor shows (including unsaved edits) to a local MP4 or GIF without a save dialog. height is the frame's short side, like every target and recipe: 1080 renders 1920x1080 at 16:9, 1080x1920 at 9:16 and 1080x1350 at 4:5 (a portrait frame asks Screen Studio for its full height). Returns a job; poll status. Results say whether the render came from the live editor or the saved project (source live/disk). Existing files are never overwritten: a taken or unwritable output ends with status delivery_failed and the renderedPath. keepRender keeps the app's temporary QA copy.

Inspect an export job; when complete, returns the delivered file with encoded media properties.

Cancel a known export job without deleting the source project.

A frame of the completed render at playback milliseconds.

Render one edit for every place it goes, in one call: x (16:9 1080p60), landing (16:9 1080p60), linkedin (1:1 1080p30), shorts (9:16 1080p60), portrait (4:5 1080p30), docs (16:9 1080p30), gif (16:9 480p24 GIF). Each target renders its own copy of the live project (aspect, padding, caption size and position, camera cutout size; vertical targets get fixed zooms on the main actions so they stay readable, and portrait targets keep the screen zoomed between zooms, about 1.8x in 9:16, so it fills the frame's width instead of a thin band), one after another, named -. in outputDir, never overwriting. The editor is not changed. A copy whose frame would crop every wide shot has avoid-empty-zoom-area turned off, and says so. Narration captioned for burn-in is burned into a --captioned.mp4 beside each video. The kit (default on) adds a poster PNG per aspect at the payoff moment, .srt and .vtt captions when there is a transcript or narration, a chapters list (yours are put in order, the first at 0:00, at least 10s apart), a manifest with each file's duration, size and limit checks, and a contact sheet of every variant. With recipe, targets default to the recipe's and its export settings apply (frame rate on every video, resolution on its own aspect). Returns files, warnings and notes explaining each choice. Takes minutes; renders run in Screen Studio one at a time, with a progress notification per target, and a render past renderTimeoutMs (default 20 minutes) is cancelled before the next starts. A second call for the same project and outputDir is refused while one runs.

Make a seamless loop for a landing page hero or a README: finds a stretch of the recording (targetMs, 4-15s) whose last frame matches its first (beat edge to beat edge, or a window inside one long beat cut at sentence breaks and still screens), applies it to a copy of the project ( Loop.screenstudio, opened in the editor) so the main edit stays intact, mutes sound, brings the cursor back to its start and ends zooms before the restart, then renders MP4 (1080p60), a silent WebM and a GIF under maxGifBytes, plus a poster. Returns the files, the source range, the similarity score (SSIM), an embed snippet, notes, and the first and last frames side by side to judge the join. apply:false only plans and shows the join from the raw recording.

Review a whole edit in one image: a labelled grid of the open editor's preview at the moments that matter (opening, each cut and the frame before it, each zoom's midpoint, layouts, masks, pacing issues, the ending), or at playbackMs you choose (up to 24). Each tile reads 'n · m:ss.s what'. With jobIds instead, one row per finished export at its opening, midpoint and ending. Use it before editor_frame, which is for a close look at one moment. The editor window must be on screen.

Save a client's brand once and apply it to any project as one live, undoable config op. A brand holds colours (primary, optional secondary and text), a backdrop (gradient from primary to secondary, flat colour, or the backdrop of a starting look), caption colours and font, cursor set and size, click sound, corner radius, shadow, padding, and the narration voice and music. Applying checks caption contrast (WCAG 4.5:1) and picks a readable caption box when the brand's is not, and checks the cursor set against the app. Actions: save (brand; replaces a saved brand of the same name and says so), list (unreadable brands are listed with their error), show (name), delete (name; removes it for good, even when it no longer reads), apply (name, projectPath; shows each setting changing in the open editor unless show is false), toPreset (name, projectPath; asks the app to save the project's settings with the brand laid over them as a Screen Studio preset, and says so when this build cannot). Image backdrops are not supported yet. Returns the settings it changed and why.

Find private text in the footage before publishing and propose blur masks for it. Reads frames (every everyMs plus just before and after each screen change) with macOS text recognition and looks for emails, phone numbers (including +E.164 and labelled digits), API keys and tokens (sk-, ghp_, xox, AKIA, AWS secret keys, Stripe, Google, private key headers), JSON web tokens, hex keys and hashes (32+ hex digits with letters; digit-only IDs such as post IDs in links, and IDs run into hex at either end, are not flagged), long random strings, card numbers (Luhn checked), IP addresses, links carrying tokens (with or without https://), passwords and secrets in settings, .env lines and connection strings, and any extraTerms (plain words, or /regex/flags; a term inside a longer secret widens the blur to the whole secret). Long recordings are read end to end: the frame grid widens to fit maxFrames. Each finding is followed across frames into one span with a padded box (0-1 of the capture). Returns detections with a masked preview (never the full value), a preview sheet of up to six frames that show the findings (each at least 960px wide, every blur filled red, existing blur masks included, every finding seen there ringed in yellow and numbered, with its kind and masked value listed under the frame), notes explaining the choices, and addMask sensitive-data ops for screenstudio_editor_apply. With apply: true the masks are added live in the open editor (undoable, then saved). An existing blur mask grows to cover new findings; a highlight or disabled mask blocks a blur, and the notes say where. ignore drops kinds that are expected on screen (custom, api-key, jwt, url-token, credential, email, card, phone, ipv4, hex-secret, base64-secret).