# Layout Detector MCP

An MCP (Model Context Protocol) server that analyzes webpage screenshots to extract layout information. Given a screenshot and image assets, it finds where each asset appears and calculates spatial relationships - enabling AI assistants to rebuild layouts with proper semantic structure.

## Quick Start

### Install from GitHub

```bash
pip install git+https://github.com/katlis/layout-detector-mcp.git
```

### Or clone and install locally

```bash
git clone https://github.com/katlis/layout-detector-mcp.git
cd layout-detector-mcp
pip install .
```

Verify installation:

```bash
python3 -c "from layout_detector import server; print('OK')"
```

## Configuration

Add to your Claude Code MCP settings (`~/.claude.json` or project `.claude/settings.json`):

```json
{
  "mcpServers": {
    "layout-detector": {
      "command": "layout-detector-mcp"
    }
  }
}
```

After adding the configuration, restart Claude Code and run `/mcp` to verify the server is connected.

## The Problem

When an AI assistant looks at a screenshot, it can describe what it sees but cannot extract precise pixel measurements. This makes it difficult to accurately recreate layouts without human intervention or extensive trial-and-error.

## The Solution

This MCP server uses computer vision (OpenCV template matching) to:

1. **Find known assets** - Locate images within a screenshot with pixel-perfect coordinates
2. **Analyze relationships** - Calculate angles, distances, and relative positions
3. **Detect patterns** - Identify radial, grid, stacked, sidebar, or freeform layouts
4. **Enable semantic rebuilds** - Provide structured data for modern CSS implementation

## Tools

### `analyze_layout`

Performs full layout analysis including pattern detection. This is the main tool you'll use.

**Parameters:**
- `screenshot_path` (string, required): Absolute path to the screenshot image
- `asset_paths` (array of strings, required): Absolute paths to asset images to find
- `threshold` (number, optional): Match confidence 0-1, default 0.8

**Returns:**
```json
{
  "viewport": { "width": 900, "height": 650 },
  "pattern": {
    "type": "radial",
    "confidence": 0.90
  },
  "radial": {
    "center_x": 450,
    "center_y": 250,
    "center_element": "logo.gif",
    "average_radius": 196
  },
  "elements": [
    {
      "asset_name": "planet1.gif",
      "x": 628,
      "y": 89,
      "width": 62,
      "height": 62,
      "angle_degrees": 45.0,
      "distance_from_center": 240
    }
  ]
}
```

### `find_assets_in_screenshot`

Locates image assets within a screenshot without layout analysis.

**Parameters:**
- `screenshot_path` (string, required): Path to the screenshot image
- `asset_paths` (array of strings, required): Paths to asset images to find
- `threshold` (number, optional): Match confidence 0-1, default 0.8

**Returns:**
```json
{
  "found": 5,
  "total_assets": 6,
  "matches": [
    {
      "asset_path": "/path/to/logo.png",
      "asset_name": "logo.png",
      "x": 350,
      "y": 200,
      "width": 200,
      "height": 100,
      "center_x": 450,
      "center_y": 250,
      "confidence": 0.95
    }
  ]
}
```

### `get_screenshot_info`

Get basic screenshot dimensions.

**Parameters:**
- `screenshot_path` (string, required): Path to the screenshot image

**Returns:**
```json
{
  "path": "/path/to/screenshot.png",
  "width": 900,
  "height": 650
}
```

## Supported Layout Patterns

| Pattern | Description | Key Data Returned |
|---------|-------------|-------------------|
| **Radial** | Elements arranged around a center point | Center element, angles, distances |
| **Grid** | Elements in rows and columns | Row/column positions, gaps |
| **Stacked** | Vertical sections (header/main/footer) | Section names, Y positions |
| **Sidebar** | Two-column with narrow sidebar | Sidebar side, widths |
| **Freeform** | No clear pattern | Raw X/Y coordinates |

## Example Usage

Once configured, Claude Code can use these tools:

```
User: Rebuild this webpage screenshot using the images in /assets

Claude: I'll analyze the layout first using the layout detector.

[Calls analyze_layout tool]

The analysis shows:
- Viewport: 900x650px
- Pattern: Radial (90% confidence)
- Center element: logo.gif at (450, 250)
- 8 elements arranged around the center
- Average distance from center: 196px

I'll implement this using CSS with the logo centered and
other elements positioned using absolute positioning...
```

## Supported Image Formats

- PNG
- JPEG
- GIF (including animated - uses first frame)
- WebP
- BMP

## Troubleshooting

### "No module named 'cv2'"

OpenCV isn't installed. Run:
```bash
pip install opencv-python-headless
```

### MCP server not showing in `/mcp`

1. Check your settings file path is correct
2. Ensure the command path is absolute (for source installs)
3. Restart Claude Code after changing settings
4. Run `python3 test_install.py` to verify the package works

### Low confidence matches

Try lowering the threshold parameter (default 0.8). Values between 0.6-0.7 may help with compressed or scaled images.

## Development

```bash
# Install in editable mode with dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Test installation
python3 test_install.py
```

## Requirements

- Python 3.11+
- OpenCV (opencv-python-headless)
- NumPy
- Pillow
- MCP SDK

## License

MIT