# Page Vision MCP

> Structured visual analysis pipeline for coding agents — give your AI agent the ability to **see**.

[![PyPI](https://img.shields.io/pypi/v/page-vision-mcp.svg)](https://pypi.org/project/page-vision-mcp/)
[![Python](https://img.shields.io/pypi/pyversions/page-vision-mcp.svg)](https://pypi.org/project/page-vision-mcp/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

[中文文档](README_CN.md)

## What It Does

Page Vision MCP turns visual analysis from a prompt into a **tool**. Instead of asking an LLM to "look at this image," your coding agent calls a structured API that returns consistent, machine-readable results with coordinates.

**The Problem:** AI coding agents modify frontend code but can't verify the result. They guess effects from source code alone.

**The Solution:** Screenshot → VLM Analysis → Structured JSON with coordinates → Agent makes informed decisions.

**5 Tools:**

| Tool | What It Does | VLM | Playwright |
|------|-------------|:---:|:----------:|
| `capture_page` | Screenshot a webpage (fallback) | - | Yes |
| `analyze_ui` | Structured UI analysis with region-level localization | Yes | - |
| `compare_ui` | Screenshot diff with mode constraints | Yes | - |
| `extract_text` | Block-level OCR with coordinates | Yes | - |
| `crop_analyze` | Progressive crop refinement with depth guards | Yes | - |

**3 Resources:** `vision://health` | `vision://version` | `vision://models`

## Pipeline

```
Screenshot ──> analyze_ui ──> Structured JSON
                  │
                  ├── Need detail? ──> crop_analyze (depth +1)
                  ├── Need text?   ──> extract_text
                  └── Need diff?   ──> compare_ui
```

Typical workflow:

```
"Check this page" ──> capture_page ──> analyze_ui ──> Agent finds issue
                                                          │
                                                    Agent fixes code
                                                          │
                               capture_page ──> compare_ui ──> Verified ✓
```

## Quick Start

### Install

```bash
pip install page-vision-mcp

# Optional: Playwright for screenshot fallback
pip install "page-vision-mcp[playwright]"
playwright install chromium
```

### Configure

Set environment variables in your MCP client config (recommended) or create a `.env` file (see `.env.example`):

```bash
VISION_API_KEY=your-api-key
VISION_BASE_URL=https://api.openai.com/v1
VISION_MODEL_ID=gpt-4o
```

Works with any OpenAI-compatible VLM: GPT-4o, GLM-4V, Qwen-VL, etc.

### MCP Client Config

Add to your MCP client configuration (Claude Code, Trae, Cursor, etc.):

```json
{
  "mcpServers": {
    "page-vision-mcp": {
      "command": "page-vision-mcp",
      "env": {
        "VISION_API_KEY": "your-api-key",
        "VISION_BASE_URL": "https://api.openai.com/v1",
        "VISION_MODEL_ID": "gpt-4o"
      }
    }
  }
}
```

See `examples/mcp_config.json` for the full configuration with all options.

## Tool Details

### `analyze_ui` — UI Analysis

```python
analyze_ui(
    image_source="screenshot.png",  # URL/path/base64/data_url
    prompt="",                       # optional focus
    task="ui",                       # ui/screenshot/qa/general
    detail="high"                    # low/auto/high
)
```

Returns structured observations with region-level localization (page → section → component → text), inferences, uncertainties, and suggested regions for further analysis.

Example output:
```json
{
  "status": "success",
  "result": {
    "summary": "A dashboard with navigation bar, sidebar, and content area",
    "observations": [
      {"type": "component", "text": "Top navigation bar with 3 menu items", "bbox": [0.0, 0.0, 1.0, 0.08]},
      {"type": "issue", "text": "Menu items too close together", "bbox": [0.15, 0.02, 0.6, 0.07]}
    ],
    "suggested_regions": [
      {"description": "Menu overlap area", "bbox": [0.15, 0.02, 0.6, 0.07], "reason": "Text may overlap, zoom in"}
    ]
  }
}
```

### `compare_ui` — Screenshot Diff

```python
compare_ui(
    image_before="before.png",
    image_after="after.png",
    mode="viewport",     # viewport/fullpage/component
    focus="changes"      # changes/layout/text/colors/general
)
```

Returns structured differences (added/removed/modified/moved) with coordinates. Perfect for verifying code changes.

### `extract_text` — OCR

```python
extract_text(
    image_source="screenshot.png",
    language="auto",        # auto/chinese/english/japanese/korean
    preserve_layout=True
)
```

Returns text blocks with type, confidence, and bounding box coordinates. Preserves original language.

### `crop_analyze` — Progressive Refinement

```python
crop_analyze(
    image_source="screenshot.png",
    x=0.15, y=0.02, width=0.6, height=0.07,  # normalized 0.0-1.0
    depth=1,       # agent-managed, max 3
    task="ui"
)
```

Use coordinates from `analyze_ui`'s `suggested_regions`. Max depth 3 prevents infinite recursion.

### `capture_page` — Screenshot Fallback

```python
capture_page(
    url="http://localhost:3000",
    viewport_width=1280,
    viewport_height=720,
    full_page=False,
    wait_for="networkidle"  # networkidle/domcontentload/load/none
)
```

**Note:** If your agent has built-in browser capability (Computer Use, Trae browser, Playwright MCP), use that instead and pass the screenshot path to `analyze_ui`.

## Response Format

All tools return a structured JSON envelope:

```json
{
  "status": "success",
  "tool": "analyze_ui",
  "result": { ... },
  "warnings": []
}
```

Status can be: `success` | `partial` | `error` | `unavailable` | `unsupported` | `completed`

## Configuration

| Variable | Default | Description |
|----------|---------|-------------|
| `VISION_API_KEY` | - | VLM API key (required) |
| `VISION_BASE_URL` | `https://api.openai.com/v1` | VLM API endpoint |
| `VISION_MODEL_ID` | `gpt-4o` | Model to use |
| `VISION_TIMEOUT` | `60` | Request timeout (seconds) |
| `VISION_MAX_RETRIES` | `3` | Max retry attempts |
| `VISION_ALLOW_LOCAL_FILES` | `true` | Allow local file paths |
| `VISION_ALLOWED_PATHS` | - | Allowed directories (comma-separated) |
| `VISION_BLOCK_PRIVATE_IPS` | `true` | Block SSRF to private IPs |
| `VISION_MAX_IMAGE_SIZE_MB` | `10` | Max image size |
| `VISION_MAX_IMAGE_PIXELS` | `40000000` | Max pixel count |

## Security

- **SSRF Protection**: Private IP addresses are blocked by default (`VISION_BLOCK_PRIVATE_IPS`)
- **Path Restriction**: Local file access can be restricted to allowed directories (`VISION_ALLOWED_PATHS`)
- **Image Limits**: Max image size and pixel count prevent resource exhaustion

## Development

```bash
pip install -e ".[dev]"
pytest tests/
```

## License

[MIT](LICENSE)