voiceover-mcp

Use voiceover-mcp to examine a web page through the actions and speech of a screen-reader user. The server exposes four Model Context Protocol (MCP) tools and keeps one Playwright WebKit session open while an agent moves through the page, listens to each announcement, and decides what to do next.

This guide takes you from installation through a practical accessibility check. See the interface reference for exact fields, limits, results, and backend support. For the development story and tradeoffs, read Building voiceover-mcp: I Made an AI Use VoiceOver.

Choose a Backend

Start with the virtual reader, then confirm important findings with real VoiceOver.

Safety: The voiceover backend turns on system-wide VoiceOver and takes over the Mac's speech and keyboard until the session stops. Run it only on a prepared machine, and always call stop_session.

Related MCP server: accessibility-mcp-server

Install and Build

You need Node.js 20 or later.

cd voiceover-mcp
npm ci
npx playwright install webkit
npm run build

These commands install the locked dependencies, install WebKit, compile the TypeScript server, and bundle the virtual reader for browser injection. The virtual backend is now ready.

Prepare Real VoiceOver

Real VoiceOver needs one-time macOS setup:

npx @guidepup/setup

Then configure the Mac:

1. Grant the terminal, editor, or MCP client that starts the server Accessibility and Automation access in System Settings → Privacy & Security.

2. Open VoiceOver Utility and enable Allow VoiceOver to be controlled with AppleScript.

3. Run the real-reader smoke check only after completing both steps.

See Guidepup's manual VoiceOver setup guide for platform details.

Connect an MCP Client

Register the built server with an absolute path:

{
  "mcpServers": {
    "voiceover": {
      "command": "node",
      "args": ["/absolute/path/to/voiceover-mcp/dist/index.js"],
      "env": {
        "SCREEN_READER": "virtual"
      }
    }
  }
}

Restart the client after changing its configuration. SCREEN_READER=virtual gives you the safer default; an explicit backend in start_session overrides it. Without either setting, the server defaults to voiceover.

The server communicates through standard input and output. Keep standard output reserved for MCP JSON-RPC messages; application diagnostics belong on standard error.

Run Your First Session

1. Open a Page

Call start_session with the virtual backend:

{
  "url": "https://www.guidepup.dev",
  "backend": "virtual"
}

The result confirms the backend, current cursor position, and supported jump targets. A landing warning means the server could not confirm that the cursor entered page content; navigate to the URL again or try one next action.

2. Inspect the Page

Send related actions in one screen_reader call:

{
  "steps": [
    { "do": "findNextHeading" },
    { "do": "next", "count": 2 },
    { "do": "findNextLink" }
  ]
}

The response shows where the cursor started, what the reader announced after each action, and where the cursor ended:

Started on: "banner"

Step 1 · findNextHeading
    heading, Guidepup, level 1
Step 2 · next ×2
    Introduction
    Automate screen readers
Step 3 · findNextLink
    Get started, link

Now focused: "Get started, link"

The reading cursor persists between calls. Continue from the reported Now focused item, or use navigate to return to a known page position.

3. Interact

Use act for links and buttons, type for text fields, and press for keys:

{
  "steps": [
    { "do": "findNextFormControl" },
    { "do": "type", "text": "screen reader testing" },
    { "do": "press", "keys": "Enter" }
  ]
}

If an action opens another page, the server detects the navigation, waits for the page, and moves the reader back into its content. The transcript reports the new URL.

4. Stop the Session

Call stop_session when the check is complete. The server also attempts cleanup when the MCP connection closes or the process stops, but an explicit stop gives you a clear result.

Perform a Screen-Reader QA Pass

Use the same order for every page so results remain comparable.

1. Check structure. On virtual, move through landmarks with findNextLandmark. Then inspect the heading outline with findNextHeading. Look for a clear page title, logical heading levels, and named regions.

2. Check navigation. Move through links with findNextLink. Flag empty names, raw URLs, vague phrases such as “click here,” and duplicate names that lead to different places.

3. Check controls and media. Use findNextFormControl and findNextGraphic. Confirm that each control has a useful name and each meaningful image has appropriate alternative text.

4. Check reading order. Use readAll to read up to 20 items at a time. Confirm that the spoken order matches the page's meaning and that no empty or hidden content interrupts it.

5. Check keyboard access. Use press with Tab and Shift+Tab. Confirm a sensible focus order, keyboard operation, a visible focus indicator, and a way out of every component. Use screenshot to inspect the visible focus indicator.

6. Confirm on VoiceOver. Repeat findings that affect task completion, image text, embeds, or reader-specific wording with the real voiceover backend.

Record the exact announcement, action, backend, and URL for each finding. A concrete transcript makes the result reproducible.

Use the Action Vocabulary

The tool exposes the keyboard actions an ordinary screen-reader user needs:

• Move: next, previous, interact, and stopInteracting.

• Activate: act.

• Jump: next or previous heading, link, landmark, form control, table, list, or graphic.

• Read: readAll.

• Enter input: type with text, or press with keys.

• Open a page: navigate with url.

A screen_reader call accepts 1–64 steps. Add count from 1–50 to repeat an action. Repeated sweeps stop when the announcement no longer changes, and readAll stops after 20 items or the end of content. The interface reference lists every exact action name.

Interpret Results Correctly

• Reading cursor and keyboard focus differ. next and previous move the reader cursor. press with Tab moves browser focus. Test both: an element may be readable but unreachable by keyboard, or focusable but absent from the reading order.

• No speech is evidence, not a verdict. (no new speech) means the reader added nothing to its speech log. Check the cursor, take a screenshot, or try a nearby action before calling it a defect.

• Errors stop the batch. The first failed action reports an error, later actions appear as skipped, and the MCP result sets isError: true.

• Backend wording differs. The virtual reader often announces role first and includes container boundaries. VoiceOver uses its own phrasing. Compare meaning and order, not literal strings.

Troubleshoot Common Problems

The server supports one WebKit tab and one session per process. The virtual backend provides fast accessibility-tree feedback; it cannot replace confirmation with a real screen reader.

Run Checks Without an MCP Client

Use the smoke scripts to verify the built server:

npm run build
npm run smoke:virtual
node scripts/smoke-virtual.mjs https://www.guidepup.dev

smoke:virtual runs headlessly and saves a screenshot. On a prepared Mac, run npm run smoke to perform the same check with real VoiceOver.

Develop and Verify Changes

The main implementation paths are:

• src/index.ts: MCP tools, schemas, backend selection, and shutdown.

• src/verbs.ts: action names, required inputs, and execution limits.

• src/executor.ts: validation, ordered execution, and captured speech.

• src/backends/voiceover.ts: real VoiceOver and headed WebKit.

• src/backends/virtual.ts: injected virtual reader and headless WebKit.

• src/format.ts: human-readable transcripts.

• test/voiceover.test.ts: unit coverage with a fake reader.

Before submitting a change, run:

npm test
npm run build
npm run smoke:virtual

Run npm run smoke when the change affects real VoiceOver behavior and the Mac is ready for takeover. Keep the interface reference synchronized when a tool, action, limit, or result changes.