# Ghostlight

> *The single light left on a theater stage when the house is dark, so no one falls off the edge.*

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![MCP Compatible](https://img.shields.io/badge/MCP-Compatible-brightgreen)](https://modelcontextprotocol.io/)
[![Node.js 18+](https://img.shields.io/badge/Node.js-18+-339933?logo=node.js)](https://nodejs.org/)
[![Playwright](https://img.shields.io/badge/Playwright-Powered-2EAD33?logo=playwright)](https://playwright.dev/)

A QA verification MCP server that prevents AI hallucination. High-level tools for accessibility, visual regression, and UI verification — with **adversarial verification** that proves agents actually looked at the screenshot.

## The Trust Problem

When an AI sees a screenshot that looks familiar, it may pattern-match and answer from training data instead of actually reading the page. "That looks like Wikipedia" → answers about what Wikipedia *should* contain, not what it *actually* shows.

**Ghostlight's solution: Adversarial Verification with Subagents**

```
Orchestrator: "Is this page professionally styled?"

Ghostlight: Here's the screenshot and a subagent prompt.
            The subagent must extract specific text to prove they looked:

Subagent extractions:
  1. Site name: "Hacker News" ✓
  2. First headline: "Cloudflare acquires Astro" ✓  ← can't guess this
  3. Nav links: "new, past, comments, ask..." ✓

Result: TRUSTED (verified actual perception)
```

The headline is today's content — not guessable from training data. The agent had to actually read the screenshot.

## Why Ghostlight?

AI coding agents need to verify their UI work, but existing approaches have problems:

- **Pattern matching** — Agents recognize "Wikipedia-shaped" pages and answer from memory
- **Shallow engagement** — Quick glance at header, then guess the rest
- **Context contamination** — URL or site name triggers prior knowledge
- **No verification** — Agents claim they "see" things without demonstrating perception

Ghostlight solves this with:
- **Adversarial framing** — "This screenshot may differ from expected content"
- **Text extraction challenges** — Prove you read by quoting specific text
- **Subagent isolation** — Fresh context with only the screenshot, no URL hints
- **Deep reading questions** — Target content that can't be guessed

## Installation

```bash
npm install
npx playwright install chromium
```

## Usage

### As an MCP Server

Add to your project's `.mcp.json`:

```json
{
  "mcpServers": {
    "ghostlight": {
      "command": "node",
      "args": ["/path/to/ghostlight/dist/index.js"]
    }
  }
}
```

Or run directly:

```bash
npm run build
npm start
```

The server communicates via stdio using the [Model Context Protocol](https://modelcontextprotocol.io/).

## Tools

| Tool | Purpose | Returns |
|------|---------|---------|
| `ping` | Health check | Echo response with timestamp |
| `get_console_logs` | Capture console errors, warnings, logs | Structured log entries with severity |
| `verify_accessibility` | axe-core WCAG audit | Violations with severity + remediation |
| `capture_dom_state` | DOM structure snapshot | Serialized DOM tree with snapshot ID |
| `compare_snapshots` | Diff two DOM states | Added/removed/changed elements |
| `measure_element` | Pixel-perfect measurements | Dimensions, spacing, typography |
| `test_viewports` | Responsive testing at breakpoints | Layout report per viewport |
| `capture_visual_state` | Screenshot + DOM + styles | Full visual context (saves to temp files) |
| `detect_layout_shift` | CLS (Core Web Vital) measurement | Shift score + shifting elements |
| `detect_flash` | Rapid visibility changes | Flash events with timestamps |
| `visual_verify` | Start verified visual check | Session ID, screenshot path, challenges |
| `submit_visual_response` | Submit challenge answers | Trust status, score, hints |

### Tool Categories

**Verified Visual Tools** (anti-hallucination):
- `visual_verify`, `submit_visual_response`

**Text-only Tools** (fast, no images):
- `get_console_logs`, `verify_accessibility`, `detect_layout_shift`, `measure_element`, `capture_dom_state`, `compare_snapshots`

**Standard Visual Tools** (screenshots saved to temp files):
- `capture_visual_state`, `test_viewports`, `detect_flash`

## Visual Verification Workflow

The `visual_verify` and `submit_visual_response` tools form a challenge-response system for verifying that an AI agent can actually "see" a page. The recommended pattern uses **subagent delegation** to keep the main conversation context clean.

### The Problem: AI Hallucination

When an AI agent sees a screenshot that looks familiar (e.g., "looks like Wikipedia"), it may:
1. Glance briefly at the image
2. Recognize the pattern
3. Answer from training data instead of actual screenshot content

This causes answers about what the page "should" contain, not what it actually shows.

### The Solution: Subagent + Adversarial Framing

```mermaid
flowchart LR
    A[Orchestrator<br/>calls visual_verify] --> B[Subagent<br/>fresh context]
    B --> C[Verification<br/>Result]

    A -.->|Returns| D[session_id<br/>screenshot_path<br/>challenges<br/>subagent_prompt]
    D -.-> B

    B -->|reads screenshot<br/>answers + submits| C
```

**Key insight:** Dispatch a fresh subagent with ONLY the screenshot and questions. No URL, no site context. The subagent must actually read the image to answer.

### Example: Complete Verification Flow

**Step 1: Orchestrator calls visual_verify with criteria**
```typescript
// Orchestrator defines what to check BEFORE seeing the page
{
  url: "http://localhost:3000/dashboard",
  query: "Is this page professionally styled?",
  criteria: [
    { question: "What is the main heading text?", type: "text" },
    { question: "Is there a navigation bar?", type: "boolean" },
    { question: "How many buttons are visible?", type: "count" }
  ]
}

// Response includes ready-to-use subagent prompt:
{
  session_id: "vs-abc123-xyz",
  screenshot_path: "/tmp/ghostlight-verification/verification-123.png",
  challenges: [...],
  subagent_prompt: "VISUAL VERIFICATION\n\nThis screenshot may differ..."
}
```

**Step 2: Dispatch subagent with the provided prompt**
```typescript
// Use the Task tool to dispatch a subagent
Task({
  prompt: response.subagent_prompt,  // Use the pre-built prompt
  subagent_type: "general-purpose"
})
```

**Step 3: Subagent handles everything**

The subagent will:
1. Read the screenshot
2. Extract specific text (proves they looked)
3. Answer challenges
4. Call `submit_visual_response`
5. Retry if not trusted (up to 3 times)
6. Return final result

### Adversarial Framing

The subagent prompt uses adversarial framing to force deep engagement:

```
VISUAL VERIFICATION

This screenshot may differ from expected content.
Extract exact text - do not guess from familiar layouts.

Required extractions (read first, then fill in):
1. Site name/logo text (exact words): ___
2. First headline (exact words): ___
3. Navigation links visible (list them): ___

If text is unclear, write UNCLEAR rather than guessing.
```

This works because:
- **"May differ from expected"** — breaks pattern-matching shortcuts
- **Required extractions** — forces systematic reading before answering
- **Exact quotes** — can't be guessed from training data
- **UNCLEAR option** — prevents fabrication when uncertain

### Orchestrator-Defined Criteria

Instead of auto-generated challenges, the orchestrator can define exactly what to verify:

```typescript
criteria: [
  { question: "What is the main heading text?", type: "text" },
  { question: "Is there a footer with copyright?", type: "boolean" },
  { question: "How many form fields are visible?", type: "count" }
]
```

**Benefits:**
- Orchestrator commits to acceptance criteria BEFORE seeing the page
- Questions are specific and meaningful
- No ambiguity about what "professionally styled" means
- Subagent just verifies visual perception

### Challenge Types

| Type | Example Question | Validation |
|------|------------------|------------|
| `text` | "What does the heading say?" | 50%+ word overlap, substring match |
| `count` | "How many buttons visible?" | ±2 absolute or ±50% relative |
| `boolean` | "Is the nav bar visible?" | Accepts true/false/yes/no |
| `color` | "What color is the button?" | Fuzzy RGB matching (~5% tolerance) |
| `position` | "Where is the sidebar?" | Keyword-based region matching |

### Trust Threshold

Responses are marked `trusted: true` when **80%+ of challenges pass**. The generous tolerances (±2 for counts, 50% word overlap for text) focus on "did they look?" rather than "did they count exactly?"

### Why Subagents?

1. **Context isolation** — Visual analysis stays out of main conversation
2. **Fresh perspective** — No accumulated context bias
3. **Anti-hallucination** — Subagent has no prior knowledge of what page "should" be
4. **Clean results** — Main agent gets structured pass/fail result

### Session Lifecycle

- Sessions expire after **5 minutes** (configurable)
- Maximum **1000 concurrent sessions** (configurable)
- Screenshots saved to temp directory

## Tool Reference

### get_console_logs

Captures console output from page load. Attaches listeners BEFORE navigation to catch everything.

```typescript
{
  url: string;           // Required: Page URL to monitor
  levels?: string[];     // Optional: ['error', 'warning', 'log', 'info', 'debug']
  wait_ms?: number;      // Optional: Time to wait for async logs (default: 1000)
}
```

Returns status (`clean`, `has_errors`, `has_warnings`), summary counts, and detailed entries with source locations.

### verify_accessibility

Runs axe-core WCAG 2.1 audit. Returns actionable violations with remediation guidance.

```typescript
{
  url: string;           // Required: Page URL to audit
  selector?: string;     // Optional: Scope audit to specific element
  rules?: string[];      // Optional: Specific axe-core rule IDs to check
}
```

### capture_dom_state / compare_snapshots

Capture DOM state, then compare two snapshots for regression testing.

```typescript
// Capture
{ url: string; selector?: string; max_depth?: number; }

// Compare
{ before_id: string; after_id: string; }
```

### measure_element

Returns precise measurements: dimensions, box model, typography, visibility.

```typescript
{
  url?: string;          // Optional if page already loaded
  selector: string;      // Required: CSS selector
}
```

### test_viewports

Tests responsive design across breakpoints.

```typescript
{
  url: string;
  selector?: string;     // Element to track visibility
  breakpoints?: Array<{name: string; width: number; height: number}>;
  // Default: mobile (375x667), tablet (768x1024), desktop (1920x1080)
}
```

### capture_visual_state

Comprehensive visual capture. Screenshots and DOM snapshots are saved to temp files to prevent token overflow.

```typescript
{
  url: string;
  selector?: string;     // Capture specific element instead of full page
  scroll_to_load?: boolean;  // Scroll to trigger lazy-loaded content
  max_depth?: number;    // DOM tree depth (default: 10)
}
```

### detect_layout_shift

Measures Cumulative Layout Shift using PerformanceObserver API.

```typescript
{
  url: string;
  wait_time?: number;    // Observation time in ms (default: 1000)
}
```

Returns CLS score with thresholds: `good` (<0.1), `needs-improvement` (0.1-0.25), `poor` (>0.25).

### detect_flash

Detects rapid visibility/style changes that cause flicker.

```typescript
{
  url: string;
  wait_time?: number;    // Observation time in ms (default: 500)
}
```

### visual_verify

Initiates a verified visual verification session. Captures screenshot and generates challenges. Supports orchestrator-defined criteria for precise control over what to verify.

```typescript
{
  url: string;           // Required: Page URL to verify
  query: string;         // Required: Verification question (e.g., "Is the button red?")
  selector?: string;     // Optional: Focus on specific element
  criteria?: Array<{     // Optional: Orchestrator-defined verification criteria
    question: string;    // The verification question
    type: 'boolean' | 'text' | 'count';  // Expected answer type
  }>;
}
```

Returns:
- `session_id` — Use this in `submit_visual_response`
- `screenshot_path` — Path to captured screenshot (temp file)
- `user_query` — Echo of the verification query
- `challenges` — Array of questions (expected answers withheld)
- `subagent_prompt` — Ready-to-use prompt for dispatching a verification subagent

### submit_visual_response

Submits answers to a verification session. Validates against expected values with fuzzy matching.

```typescript
{
  session_id: string;           // Required: From visual_verify
  challenge_answers: Array<{    // Required: Your answers
    id: string;
    answer: string | number | boolean;
  }>;
  query_answer: string;         // Required: Answer to original query
}
```

Returns:
- `trusted` — Boolean, true if >= 80% challenges passed
- `verification_score` — 0.0 to 1.0 score
- `challenges_passed` / `challenges_total` — Pass counts
- `failed_challenges` — List of questions that failed (without revealing expected values)
- `query_answer` — Echo of the provided answer

**Note:** Failed challenges show only the question text, not expected values. This prevents gaming the system by retrying with revealed answers.

## Development

```bash
# Run tests
npm test

# Type check
npm run typecheck

# Build
npm run build

# Development mode (no build required)
npm run dev
```

### Architecture

```
src/
├── index.ts                      # Entry point
├── server.ts                     # MCP server with tool registry
├── browser-manager.ts            # Playwright browser lifecycle
├── snapshot-store.ts             # In-memory DOM snapshot storage
├── verification-session-store.ts # Visual verification sessions (TTL-based)
├── challenge-generator.ts        # DOM-based challenge creation
├── answer-validators.ts          # Fuzzy matching validators
└── tools/
    ├── index.ts                  # Tool exports
    ├── get-console-logs.ts
    ├── verify-accessibility.ts
    ├── capture-dom-state.ts
    ├── compare-snapshots.ts
    ├── measure-element.ts
    ├── test-viewports.ts
    ├── capture-visual-state.ts
    ├── detect-layout-shift.ts
    ├── detect-flash.ts
    ├── visual-verify.ts          # Visual verification initiation
    └── submit-visual-response.ts # Challenge answer validation
```

### Browser Session Management

Ghostlight uses a shared browser with isolated contexts:
- Browser launch is expensive (~1-3s), so one browser is shared across tools
- Each tool call gets an isolated browser context for clean state
- Contexts are automatically cleaned up after use

## Requirements

- Node.js 18+
- Chromium (installed via `npx playwright install chromium`)

## License

MIT