Skip to main content
Glama

Page Vision MCP

Structured visual analysis pipeline for coding agents — give your AI agent the ability to see.

PyPI Python License: MIT

中文文档

What It Does

Page Vision MCP turns visual analysis from a prompt into a tool. Instead of asking an LLM to "look at this image," your coding agent calls a structured API that returns consistent, machine-readable results with coordinates.

The Problem: AI coding agents modify frontend code but can't verify the result. They guess effects from source code alone.

The Solution: Screenshot → VLM Analysis → Structured JSON with coordinates → Agent makes informed decisions.

5 Tools:

Tool

What It Does

VLM

Playwright

capture_page

Screenshot a webpage (fallback)

-

Yes

analyze_ui

Structured UI analysis with region-level localization

Yes

-

compare_ui

Screenshot diff with mode constraints

Yes

-

extract_text

Block-level OCR with coordinates

Yes

-

crop_analyze

Progressive crop refinement with depth guards

Yes

-

3 Resources: vision://health | vision://version | vision://models

Related MCP server: Vision MCP Server

Pipeline

Screenshot ──> analyze_ui ──> Structured JSON
                  │
                  ├── Need detail? ──> crop_analyze (depth +1)
                  ├── Need text?   ──> extract_text
                  └── Need diff?   ──> compare_ui

Typical workflow:

"Check this page" ──> capture_page ──> analyze_ui ──> Agent finds issue
                                                          │
                                                    Agent fixes code
                                                          │
                               capture_page ──> compare_ui ──> Verified ✓

Quick Start

Install

pip install page-vision-mcp

# Optional: Playwright for screenshot fallback
pip install "page-vision-mcp[playwright]"
playwright install chromium

Configure

Set environment variables in your MCP client config (recommended) or create a .env file (see .env.example):

VISION_API_KEY=your-api-key
VISION_BASE_URL=https://api.openai.com/v1
VISION_MODEL_ID=gpt-4o

Works with any OpenAI-compatible VLM: GPT-4o, GLM-4V, Qwen-VL, etc.

MCP Client Config

Add to your MCP client configuration (Claude Code, Trae, Cursor, etc.):

{
  "mcpServers": {
    "page-vision-mcp": {
      "command": "page-vision-mcp",
      "env": {
        "VISION_API_KEY": "your-api-key",
        "VISION_BASE_URL": "https://api.openai.com/v1",
        "VISION_MODEL_ID": "gpt-4o"
      }
    }
  }
}

See examples/mcp_config.json for the full configuration with all options.

Tool Details

analyze_ui — UI Analysis

analyze_ui(
    image_source="screenshot.png",  # URL/path/base64/data_url
    prompt="",                       # optional focus
    task="ui",                       # ui/screenshot/qa/general
    detail="high"                    # low/auto/high
)

Returns structured observations with region-level localization (page → section → component → text), inferences, uncertainties, and suggested regions for further analysis.

Example output:

{
  "status": "success",
  "result": {
    "summary": "A dashboard with navigation bar, sidebar, and content area",
    "observations": [
      {"type": "component", "text": "Top navigation bar with 3 menu items", "bbox": [0.0, 0.0, 1.0, 0.08]},
      {"type": "issue", "text": "Menu items too close together", "bbox": [0.15, 0.02, 0.6, 0.07]}
    ],
    "suggested_regions": [
      {"description": "Menu overlap area", "bbox": [0.15, 0.02, 0.6, 0.07], "reason": "Text may overlap, zoom in"}
    ]
  }
}

compare_ui — Screenshot Diff

compare_ui(
    image_before="before.png",
    image_after="after.png",
    mode="viewport",     # viewport/fullpage/component
    focus="changes"      # changes/layout/text/colors/general
)

Returns structured differences (added/removed/modified/moved) with coordinates. Perfect for verifying code changes.

extract_text — OCR

extract_text(
    image_source="screenshot.png",
    language="auto",        # auto/chinese/english/japanese/korean
    preserve_layout=True
)

Returns text blocks with type, confidence, and bounding box coordinates. Preserves original language.

crop_analyze — Progressive Refinement

crop_analyze(
    image_source="screenshot.png",
    x=0.15, y=0.02, width=0.6, height=0.07,  # normalized 0.0-1.0
    depth=1,       # agent-managed, max 3
    task="ui"
)

Use coordinates from analyze_ui's suggested_regions. Max depth 3 prevents infinite recursion.

capture_page — Screenshot Fallback

capture_page(
    url="http://localhost:3000",
    viewport_width=1280,
    viewport_height=720,
    full_page=False,
    wait_for="networkidle"  # networkidle/domcontentload/load/none
)

Note: If your agent has built-in browser capability (Computer Use, Trae browser, Playwright MCP), use that instead and pass the screenshot path to analyze_ui.

Response Format

All tools return a structured JSON envelope:

{
  "status": "success",
  "tool": "analyze_ui",
  "result": { ... },
  "warnings": []
}

Status can be: success | partial | error | unavailable | unsupported | completed

Configuration

Variable

Default

Description

VISION_API_KEY

-

VLM API key (required)

VISION_BASE_URL

https://api.openai.com/v1

VLM API endpoint

VISION_MODEL_ID

gpt-4o

Model to use

VISION_TIMEOUT

60

Request timeout (seconds)

VISION_MAX_RETRIES

3

Max retry attempts

VISION_ALLOW_LOCAL_FILES

true

Allow local file paths

VISION_ALLOWED_PATHS

-

Allowed directories (comma-separated)

VISION_BLOCK_PRIVATE_IPS

true

Block SSRF to private IPs

VISION_MAX_IMAGE_SIZE_MB

10

Max image size

VISION_MAX_IMAGE_PIXELS

40000000

Max pixel count

Security

  • SSRF Protection: Private IP addresses are blocked by default (VISION_BLOCK_PRIVATE_IPS)

  • Path Restriction: Local file access can be restricted to allowed directories (VISION_ALLOWED_PATHS)

  • Image Limits: Max image size and pixel count prevent resource exhaustion

Development

pip install -e ".[dev]"
pytest tests/

License

MIT

Related MCP Connectors

Related MCP Servers