Browserose MCP

MCP Tool for Agents to control the Google Chrome browser.

Browserose MCP is an MCP (Model Context Protocol) server that lets AI agents and IDEs control Google Chrome via Playwright, with full iframe support: snapshot and interact inside iframes (e.g. ALM SCORM players, embedded apps).

The toolset is sufficient for full end-to-end automation. You can complete entire flows—login, multi-step navigation, nested iframes, quizzes, and course completion—without manual steps or switching tabs. Use navigation, locators, frame selectors, coordinate fallbacks, and diagnostics as needed.

Requirements

• Node.js 18+

• Chrome installed (or set PLAYWRIGHT_MCP_USE_CHROMIUM=1 to use Chromium)

• Playwright browsers: npx playwright install chromium (or chrome if available)

Related MCP server: OLTestStack

Install and build

npm install
npx playwright install chromium
npm run build

Cursor / IDE configuration

Add to ~/.cursor/mcp.json (or project .cursor/mcp.json):

{
  "mcpServers": {
    "playwright-chrome": {
      "command": "node",
      "args": ["/absolute/path/to/Browserose-MCP/build/index.js"]
    }
  }
}

Restart Cursor after editing mcp.json.

Environment variables

• PLAYWRIGHT_MCP_HEADLESS — set to 1 or true to run the browser headless (default: headed so you see the window).

• PLAYWRIGHT_MCP_USE_CHROMIUM — set to 1 or true to use Playwright's Chromium instead of system Chrome.

• PLAYWRIGHT_MCP_VIEWPORT_MAXIMIZED — window opens maximized by default. Set to 0 or false to use a fixed size (see width/height below).

• PLAYWRIGHT_MCP_VIEWPORT_WIDTH — viewport width in pixels (default: 1280). Used only when maximized is off.

• PLAYWRIGHT_MCP_VIEWPORT_HEIGHT — viewport height in pixels (default: 800). Used only when maximized is off.

Examples: To use a fixed size instead of maximized, set PLAYWRIGHT_MCP_VIEWPORT_MAXIMIZED=0 and optionally PLAYWRIGHT_MCP_VIEWPORT_WIDTH=1920, PLAYWRIGHT_MCP_VIEWPORT_HEIGHT=1080.

Popup handler

When a link or button opens a new tab (e.g. Go to activity on IBM SkillsBuild opens the SCORM player in a new window), the MCP automatically attaches to that new tab. The browser context listens for the page event; when a new page is created, the internal page reference is updated so that all subsequent tool calls (navigate, click, snapshot, frame_probe, etc.) run in the new tab, not the original one. You do not need to switch tabs manually or use a separate “switch to tab” tool: click Go to activity, wait for the player to load, then continue with the same tools—they will already target the player tab. Implementation: in src/browser.ts, context.on("page", (newPage) => { page = newPage; }) runs on every new page (popup or target=_blank). Restart the MCP server (or Cursor) after pulling changes so the handler is active.

Tools (quick reference)

Tools reference and use cases

Detailed description of each tool and when to use it. Optional frameSelector uses >> for nested iframes (e.g. iframe#a >> iframe#b).

Navigation

Snapshot and ref-based interaction

These tools use an accessibility/DOM snapshot to get refs (e.g. s1e2). You then pass that ref to click, type, hover, or select. Best when the page (or iframe) is same-origin and has a normal DOM.

Locator-based interaction (no snapshot)

These tools use Playwright locators (role+name, text, or css) and do not require a snapshot. They work in cross-origin iframes and are the main escape hatch when refs aren't available or snapshot is empty.

Customizing by situation: Use nth when multiple elements match (e.g. 4th button named "SUBMIT"). Use enabledOnly to click the first enabled match when the DOM has several disabled copies. Use scopeCss to restrict the search to a container (e.g. .quiz-card, [role="dialog"]) so you don't match the same text or role in the sidebar or another panel.

Coordinate-based interaction (escape hatch)

When snapshot and locators both fail (e.g. canvas, custom-rendered UI, or wrong frame depth), use coordinates relative to a frame.

Diagnostics (finding the right frame / layer)

When a frame shows no buttons or empty text in the snapshot, the real UI is often in a child iframe, canvas, or shadow DOM. These tools help you find it.

Utility

Use cases in practice

• Normal web automation (main page)
  Use browser_snapshot (or browser_snapshot_frame with no/minimal nesting) to get refs, then browser_click, browser_type, browser_hover, browser_select_option with those refs. Optional browser_screenshot for verification.

• Login flows
  Often on the main page: browser_type_locator and browser_click_locator with role/name or text (e.g. email → Continue → password → Log in). No snapshot required.

• Single iframe, same-origin
  browser_snapshot_frame with frameSelector: "iframe#id" → get refs → browser_click / browser_type with the same frameSelector.

• Cross-origin or "empty" iframe
  Snapshot may be empty or refs may not work. Use locators: browser_list_clickables with frameSelector to see what's there, then browser_click_locator and browser_type_locator with the same frameSelector and role/text/css.

• ALM / SCORM (nested iframes)
  The visible lesson UI is often in a third-level iframe. If browser_frame_probe on iframe#pplayer_iframe >> iframe#modulePlayerIframe shows clickables: 0, run browser_frame_inventory on that chain; it will list child iframes (e.g. iframe#content-frame). Extend the chain to ... >> iframe#content-frame and use browser_list_clickables and browser_click_locator (e.g. role: "button", name: "Next") there. One-line takeaway: when the frame has no DOM content, use frame_inventory to find the real content iframe, then add it to the chain.

• Canvas or custom-rendered UI
  If frame_inventory shows a large canvas and no useful iframe, or locators don't match: use browser_click_at_rel_debug to see where (rx, ry) lands, then browser_click_at_rel with adjusted (rx, ry), or browser_frame_bbox + browser_click_at with computed (x, y).

• Quizzes / multiple identical buttons
  When several "SUBMIT" or "Next" buttons exist and only one is enabled, use browser_click_locator with enabledOnly: true (and same role/name) so the first enabled match is clicked. Or use browser_list_clickables with role: "button", name: "SUBMIT" (and optionally enabledOnly: true) to see indices, then click with nth. Use scopeCss (e.g. .quiz-card) to restrict to the current question card and avoid matching the sidebar.

• Debugging "click does nothing"
  Check: (1) Correct frame? → browser_frame_probe and browser_frame_inventory. (2) Right element? → browser_list_clickables in that frame; browser_hit_test_rel to see what's under (rx, ry). (3) Right coordinates? → browser_click_at_rel_debug. (4) Multiple matches? → use nth or enabledOnly.

End-to-end: IBM SkillsBuild / ALM course

You can run the full course flow with only Browserose MCP tools: from the IBM page through login, learning plan, launching the activity, and completing lessons and quizzes.

1. Start from the IBM page and log in

• Navigate to the course or plan URL (e.g. https://skills.yourlearning.ibm.com/activity/PLAN-...).

• If redirected to login: browser_click_locator with text: "Log in with ibm" (or equivalent). On the IBM login page:

  • browser_type_locator with role: "textbox", name: "IBMid", and the email.

  • browser_click_locator with role: "button", name: "Continue".

  • browser_type_locator with role: "textbox", name: "Password", and the password.

  • browser_click_locator with role: "button", name: "Log in".

• Use browser_wait and browser_screenshot as needed to confirm the next page.

2. Open the learning plan and the module

• From the plan page: browser_click_locator with text: "Microcredential 1: Data Classification" (or the right section).

• Then browser_click_locator with text: "Classifying and Sourcing Data" (or the target module).

• Launch the activity: browser_click_locator with role: "button", name: "Go to activity".

• Wait for the player: Clicking Go to activity may open the SCORM player in a new tab. The MCP attaches to new tabs automatically (popup handler), so the next tool calls then run in the player tab. Use browser_wait (e.g. 5–8 seconds), then browser_frame_probe or browser_frame_inventory on iframe#pplayer_iframe (and then iframe#pplayer_iframe >> iframe#modulePlayerIframe if needed) until the content frame is present. The visible lesson UI is in the content frame:
  iframe#pplayer_iframe >> iframe#modulePlayerIframe >> iframe#content-frame

3. Content frame: lessons and “Continue”

• Continue / Next (content):
  browser_click_locator with frameSelector: "iframe#pplayer_iframe >> iframe#modulePlayerIframe >> iframe#content-frame", css: "button.continue-btn".

• Use browser_list_clickables with that frameSelector to discover buttons (e.g. “Next”, “Continue”). Use browser_screenshot with that frameSelector when you need to see what’s on screen.

4. Practice quiz: two-pass strategy

The fastest way to pass is: (1) first pass—answer arbitrarily, submit, read the correct answer from feedback, memorize it; (2) TAKE AGAIN; (3) second pass—answer with the memorized correct options and submit.

• Start or restart quiz:
  browser_click_locator in the content frame with text: "START QUIZ" or text: "TAKE AGAIN".

• Select an option:
  If locators like #qmc-X-label are covered by overlays, use browser_click_at_rel in the content frame with e.g. rx: 0.5, and ry roughly: first option ~0.48–0.52, second ~0.56–0.6, third ~0.64–0.68 (tune if layout differs). Or use browser_hit_test_rel / browser_click_at_rel_debug to confirm.

• Submit:
  browser_click_locator in the content frame with role: "button", name: "SUBMIT", enabledOnly: true (so the active SUBMIT is clicked).

• Read feedback:
  After submit, use browser_frame_probe on the content frame and read textSample (or use browser_screenshot) to get “Correct answer: …” and store it per question index (Q1, Q2, …).

• Next question:
  browser_click_at_rel in the content frame with rx: 0.65, ry: 0.85 (NEXT button area). Repeat until the quiz is done (no more SUBMIT or you see completion).

• Second pass:
  Click TAKE AGAIN, then for each question click the option that matches the stored correct answer (by ry band or by locator if the correct option text is clickable), SUBMIT, then click (0.65, 0.85) for NEXT until the quiz is complete.

5. After the quiz and finishing the module

• When the quiz is complete, use browser_click_locator in the content frame with css: "button.continue-btn" (or equivalent) to continue to the next lesson or close the module.

• Repeat the same pattern for further lessons and quizzes until the module/course is marked complete.

Summary: All steps—login, plan navigation, “Go to activity”, waiting for the player, lesson Continue, quiz (two-pass with feedback reading and TAKE AGAIN), and completion—can be done with the existing tools (navigate, click_locator, type_locator, list_clickables, frame_probe, frame_inventory, click_at_rel, screenshot, wait). No manual tab switching or external tools are required.

Using iframes (including cross-origin / ALM SCORM)

1. Navigate to a page that contains an iframe (e.g. ALM course page).

2. Call browser_snapshot with includeFrames: true to get the main page plus same-origin iframes, or call browser_snapshot_frame with frameSelector: "iframe" (or iframe#id) to get only that frame's tree.

3. Use the returned refs with browser_click, browser_type, etc., and pass the same frameSelector. For nested iframes use a chained selector with >>, e.g. iframe#pplayer_iframe >> iframe#modulePlayerIframe.

Example: click "Next" inside the first iframe:

• browser_snapshot_frame with frameSelector: "iframe" → get ref for the "Next" button (e.g. f1e2).

• browser_click with ref: "f1e2", frameSelector: "iframe".

Escape hatch (cross-origin / SCORM): For frames where snapshot fails, use Playwright locators directly (no AX/DOMSnapshot):

• browser_list_clickables with frameSelector: "iframe#pplayer_iframe >> iframe#modulePlayerIframe" → lists buttons/links with text and enabled/disabled.

• ALM SCORM: The visible lesson UI (e.g. "Next", "Learning objectives") lives in a third-level iframe. Use browser_frame_inventory on iframe#pplayer_iframe >> iframe#modulePlayerIframe to see child iframes; then chain to the content frame: iframe#pplayer_iframe >> iframe#modulePlayerIframe >> iframe#content-frame. Use that selector with browser_list_clickables and browser_click_locator (e.g. role: "button", name: "Next").

• browser_click_locator with the same frameSelector and role: "button", name: "Next" (or text: "Next") → clicks the element. Works because Playwright targets the frame's context directly.

Cross-origin / SCORM (AX tree empty): The server also uses a 3-tier snapshot for frames:

1. Tier A — CDP Accessibility.getFullAXTree (refs with backendDOMNodeId; click via box model).

2. Tier B — If AX is empty, CDP DOMSnapshot.captureSnapshot (refs with viewport coordinates; click via Input.dispatchMouseEvent).

3. Tier C — Use browser_screenshot with frameSelector, then browser_click_at with the same frameSelector and (x, y) to click by coordinates (e.g. canvas or when both AX and DOM snapshot fail).

License and author

• License: This project is open source. Use and modify it freely.

• Developer: ETTALBI OMAR