BrowserAgent

✨ Highlights

• One model, not two. observe returns the a11y snapshot and the pixel overlay in a single call, so the model never reconciles text and images itself.

• Diffs, not dumps. The diff engine returns only what changed since the last observation, with fingerprint-based uid rebinding across navigation.

• Events, not polling. Console, network, DOM, and navigation events are collected and pushed, so the model doesn't re-read the page every turn.

• Strict TDD at 100/100. Every module lands at 100% coverage and 100% mutation score, enforced by CI.

• TypeScript-only, zero tolerance. No .js/.mjs/.cjs anywhere; no as, any, !, or @ts-ignore in the codebase.

Related MCP server: hermes-computer-use

The thesis

Most browser integrations treat "read the page" and "act on the page" as separate worlds, one returning text and the other returning pixels. That split forces the model to re-read the page every turn and to guess at the relationship between what it sees and what it can click.

BrowserAgent's core idea: build one unified, event-driven, visually-rich model where the semantic layer and the visual layer are the same object, a model that can both watch and explain.

Why another browser MCP server?

It's not a fork of chrome-devtools-mcp. It borrows that project's proven patterns (stable element uids keyed by loaderId_backendNodeId, a ContextPage abstraction that hides Puppeteer behind a narrow contract, an "act then wait for stable DOM/navigation" wrapper, and token-optimized formatters) and rebuilds them from scratch around a different product shape.

• Unified observe, not split snapshot/screenshot. One call returns the a11y tree with the pixel overlay.

• Diff, don't re-read. The diff engine (with fingerprint-based uid rebinding) means the model doesn't pay to re-read the whole page every turn.

• Events, not polling. The server pushes changes instead of the model polling.

• Fewer round trips over micro-optimization. The browser and the LLM are the bottlenecks, so performance comes from the diff engine and event-driven observation rather than shaving milliseconds.

Architecture

flowchart TB
    Client[MCP Client / LLM]

    subgraph Protocol["MCP Protocol Layer"]
        Server[McpServer<br/>tools/list · tools/call · server/discover]
    end

    subgraph Framework["Tool Framework"]
        Handler[ToolHandler<br/>defineTool · gating · write mutex]
    end

    subgraph Core["Core Model"]
        direction LR
        Observe[observe<br/>snapshot + overlay]
        Diff[Diff engine<br/>changes + rebinding]
        Events[Event layer<br/>buffer + collector]
        Actions[Action layer<br/>log + act-then-wait]
    end

    Browser[ContextPage<br/>over Puppeteer / CDP]

    Client <-->|MCP| Server
    Server -->|registerTool| Handler
    Handler -->|read| Observe
    Handler -->|write| Actions
    Observe --> Diff
    Observe --> Events
    Actions --> Events
    Observe --> Browser
    Actions --> Browser

The layers are thin and testable: the protocol layer bridges our tools onto MCP, the ToolHandler enforces gating, and the ContextPage hides Puppeteer behind a narrow contract so nothing touches the raw page.

The product surface, in dependency order:

Getting started

Requirements

Install

git clone https://github.com/PremierStudio/BrowserAgent.git
cd BrowserAgent
npm install

Run the full gate chain

This is exactly what CI enforces:

npm run ci

Individual gates

Software stack

Engineering: strict TDD with a 100/100 gate

Every module is written test-first (RED → GREEN → refactor) and must clear a hard, CI-enforced gate before it is committed:

typecheck → lint → format → knip → unit tests → coverage (100%) → mutation (100%) → survivor registry

• 100% coverage (lines/branches/functions/statements) via Vitest + @vitest/coverage-v8.

• 100% mutation score via Stryker. Coverage alone is a lie; mutation testing proves the tests actually catch real faults. The only escape from the mutation gate is a pre-approved, documented entry in mutation-survivors.json (currently empty).

• No dead code. knip runs with zero findings. Unused files, exports, and dependencies are removed, not ignored.

• TypeScript only, everywhere. No .js/.mjs/.cjs anywhere, including source, configs (eslint.config.ts, stryker.config.ts, vitest.config.ts), and scripts (emitted to dist-scripts/ via tsconfig.scripts.json).

• No banned constructs. as casts, any, ! non-null assertions, @ts-ignore/@ts-nocheck/@ts-expect-error, and .forEach are all lint errors.

• Deterministic tests. Clocks and timers are injected, so there are no sleeps and no flaky timing.

The full engineering spec lives in docs/mvp.md; every deviation and protocol decision is recorded in docs/decisions.md.

Repository layout

src/
  actions/       ActionLog (replay seed), ActionRunner, StabilityWaiter (act-then-wait)
  context/       ContextPage abstraction + CDP a11y-tree conversion (axTree)
  diff/          diff engine, fingerprint-based uid rebinding, DiffTracker
  events/        event types, bounded EventBuffer, normalizer, EventCollector
  protocol/      MCP tool bridge to @modelcontextprotocol/server v2
  snapshot/      a11y snapshot builder + uid→box overlay
  tools/         tool framework: defineTool, ToolHandler, ToolMutex, Response, observe
  uid.ts         stable loaderId_backendNodeId uid generation
docs/
  mvp.md         the product plan and non-negotiable engineering requirements
  decisions.md   every amendment (supersedes mvp.md where they conflict)
scripts/         survivor-registry checker (TypeScript, emitted to dist-scripts/)
.github/workflows/ci.yml

Roadmap

gantt
    title BrowserAgent milestones
    dateFormat  YYYY-MM-DD
    axisFormat  %d

    section Foundation
        M0 Tooling + test infra      :done, m0, 2026-08-11, 1d
    section Core model
        M1 Page model (observe)      :done, m1, 2026-08-11, 1d
        M2 Diff engine               :done, m2, 2026-08-11, 1d
        M4 Event layer               :done, m4, 2026-08-11, 1d
        M3 Action primitives         :done, m3, 2026-08-11, 1d
    section Protocol
        M5 MCP protocol layer        :active, m5, 2026-08-12, 3d
    section Intent & UI
        M6 Intent tools              :m6, after m5, 3d
        M7 MCP Apps replay UI        :m7, after m6, 3d
    section Hardening
        M8 Hardening                 :m8, after m7, 2d

Contributing

This repo follows the rules in AGENTS.md (binding for every agent) and the spec in docs/mvp.md. The short version:

• Strict TDD. Write the failing test first (RED), confirm it fails for the right reason, then implement (GREEN), then refactor.

• The 100/100 gate. No module merges unless it's at 100% coverage and 100% mutation score, with typecheck/lint/format/knip clean.

• No survivor silencing. A surviving mutant is fixed by strengthening the test, never by weakening the code or adding ignore comments.

• TypeScript only. No .js/.mjs/.cjs, including configs and scripts.

License

Apache License 2.0 © Premier Studio. See LICENSE.