Semantic Hints MCP

A small local stdio MCP server that exposes compact semantic observations from a UI annotated with data-agent-* hints.

It is a companion to the official Playwright MCP, not a replacement. It only reads semantics; all browser actions (click, type, navigate) stay in Playwright MCP. The intended split:

1. semantic_snapshot → a compact map of hinted elements (~2–3× smaller than a Playwright ARIA snapshot of the same page).

2. Pick a semantic id, e.g. checkout.submit.

3. Act via Playwright MCP using the selector [data-agent-id='checkout.submit'].

4. semantic_observe → the current value/state of one hinted element.

Browser / session architecture

The semantic-hints MCP uses the Playwright library directly. It does not call the Playwright MCP internally.

However, the semantic-hints MCP and the official Playwright MCP must observe/control the same browser state during an experiment. Avoid accidentally creating two unrelated browser sessions where:

• Playwright MCP clicks in browser A

• semantic-hints MCP observes browser B

That would invalidate the workflow.

Semantic-hints MCP must not silently launch a separate browser by default.

Preferred design: one shared Chromium over CDP

1. Start Chromium with remote debugging enabled, e.g.:

   chromium --remote-debugging-port=9222

   (or the equivalent using your locally installed Chrome/Chromium binary).

2. Configure the official Playwright MCP to connect to that shared browser, if supported by the installed version/config.

3. The semantic-hints MCP connects to the same browser via Playwright:

   chromium.connectOverCDP("http://127.0.0.1:9222")

4. It reuses the active page where possible. If no suitable page exists and a URL is provided, it may open/navigate a page. If no page exists and no URL is provided, it returns a clear error asking for a URL or active page.

Configuration

Behavior:

• If SEMANTIC_HINTS_CDP_URL is reachable, connect over CDP (the normal mode).

• If the CDP connection fails and SEMANTIC_HINTS_LAUNCH_BROWSER is false, fail clearly with setup instructions — never silently launch.

• If SEMANTIC_HINTS_LAUNCH_BROWSER is true, the server may launch its own browser, but every tool response then carries a warning field making clear this is a standalone session that may not share state with the Playwright MCP.

The normal research workflow should use the shared CDP browser mode.

Related MCP server: Scout

Tools

semantic_snapshot

Returns hinted elements grouped into regions, actions, inputs, observables, navigation, other, plus counts, url, and screen.

{
  "url": "http://localhost:8082/products", // optional: navigate first; else read current page
  "scope": "products.filters",             // optional: data-agent-id or CSS selector subtree
  "includeHidden": false                    // optional: include hidden elements (default false)
}

Per element (fields included only when present): id, role, name, action, state, target, controls, observes, value (inputs/observables only), enabled (interactive only), visible. Never returns HTML, class names, or DOM subtrees.

semantic_observe

{ "id": "cart.total" }

Resolves [data-agent-id="<id>"] and returns its compact current state. Returns a clear error if zero or multiple elements match.

Setup

cd semantic-hints-mcp
npm install
npx playwright install chromium   # one-time browser download
npm run build

See Configuration above for the environment variables.

Register in Claude Code

From this folder, after npm run build:

claude mcp add semantic-hints \
  --env SEMANTIC_HINTS_CDP_URL=http://127.0.0.1:9222 \
  --env SEMANTIC_HINTS_TARGET_URL=http://localhost:8080 \
  -- node "$(pwd)/dist/index.js"

Or add it to .mcp.json / your Claude Code config manually:

{
  "mcpServers": {
    "semantic-hints": {
      "command": "node",
      "args": ["/absolute/path/to/semantic-hints-mcp/dist/index.js"],
      "env": {
        "SEMANTIC_HINTS_CDP_URL": "http://127.0.0.1:9222",
        "SEMANTIC_HINTS_TARGET_URL": "http://localhost:8080"
      }
    }
  }
}

(For development without a build step, use "command": "npx", "args": ["tsx", "/abs/path/src/index.ts"].)

Usage example

Expected Claude Code workflow:

1. Start the app under test, e.g.

   npm run dev          # → http://localhost:8080 for the annotated WebTestBench app

2. Start a shared Chromium with CDP enabled, e.g.

   chromium --remote-debugging-port=9222

3. Register/start both MCP servers:

   • the official Playwright MCP, configured for the shared browser if possible

   • the semantic-hints MCP, configured with SEMANTIC_HINTS_CDP_URL=http://127.0.0.1:9222

4. Agent calls:

   semantic_snapshot({ "url": "http://localhost:8080" })

5. Agent interacts using the official Playwright MCP:

   browser_click({ "target": "[data-agent-id='checkout.submit']" })

6. Agent calls:

   semantic_observe({ "id": "cart.total" })

The click (step 5) and observe (step 6) must operate on the same browser page/session — which is exactly what the shared CDP browser guarantees.

Tests

npm test

Vitest loads test/fixture.html in headless Chromium and checks: grouped compact output, hidden-element handling, scope, observable/input value reads, missing & duplicate-ID errors, and that output contains no raw HTML/DOM.