Layout Detector MCP

# Layout Detector MCP An MCP (Model Context Protocol) server that analyzes webpage screenshots to extract layout information. Given a screenshot and image assets, it finds where each asset appears and calculates spatial relationships - enabling AI assistants to rebuild layouts with proper semantic structure. ## Quick Start ### Install from GitHub ```bash pip install git+https://github.com/katlis/layout-detector-mcp.git ``` ### Or clone and install locally ```bash git clone https://github.com/katlis/layout-detector-mcp.git cd layout-detector-mcp pip install . ``` Verify installation: ```bash python3 -c "from layout_detector import server; print('OK')" ``` ## Configuration Add to your Claude Code MCP settings (`~/.claude.json` or project `.claude/settings.json`): ```json { "mcpServers": { "layout-detector": { "command": "layout-detector-mcp" } } } ``` After adding the configuration, restart Claude Code and run `/mcp` to verify the server is connected. ## The Problem When an AI assistant looks at a screenshot, it can describe what it sees but cannot extract precise pixel measurements. This makes it difficult to accurately recreate layouts without human intervention or extensive trial-and-error. ## The Solution This MCP server uses computer vision (OpenCV template matching) to: 1. **Find known assets** - Locate images within a screenshot with pixel-perfect coordinates 2. **Analyze relationships** - Calculate angles, distances, and relative positions 3. **Detect patterns** - Identify radial, grid, stacked, sidebar, or freeform layouts 4. **Enable semantic rebuilds** - Provide structured data for modern CSS implementation ## Tools ### `analyze_layout` Performs full layout analysis including pattern detection. This is the main tool you'll use. **Parameters:** - `screenshot_path` (string, required): Absolute path to the screenshot image - `asset_paths` (array of strings, required): Absolute paths to asset images to find - `threshold` (number, optional): Match confidence 0-1, default 0.8 **Returns:** ```json { "viewport": { "width": 900, "height": 650 }, "pattern": { "type": "radial", "confidence": 0.90 }, "radial": { "center_x": 450, "center_y": 250, "center_element": "logo.gif", "average_radius": 196 }, "elements": [ { "asset_name": "planet1.gif", "x": 628, "y": 89, "width": 62, "height": 62, "angle_degrees": 45.0, "distance_from_center": 240 } ] } ``` ### `find_assets_in_screenshot` Locates image assets within a screenshot without layout analysis. **Parameters:** - `screenshot_path` (string, required): Path to the screenshot image - `asset_paths` (array of strings, required): Paths to asset images to find - `threshold` (number, optional): Match confidence 0-1, default 0.8 **Returns:** ```json { "found": 5, "total_assets": 6, "matches": [ { "asset_path": "/path/to/logo.png", "asset_name": "logo.png", "x": 350, "y": 200, "width": 200, "height": 100, "center_x": 450, "center_y": 250, "confidence": 0.95 } ] } ``` ### `get_screenshot_info` Get basic screenshot dimensions. **Parameters:** - `screenshot_path` (string, required): Path to the screenshot image **Returns:** ```json { "path": "/path/to/screenshot.png", "width": 900, "height": 650 } ``` ## Supported Layout Patterns | Pattern | Description | Key Data Returned | |---------|-------------|-------------------| | **Radial** | Elements arranged around a center point | Center element, angles, distances | | **Grid** | Elements in rows and columns | Row/column positions, gaps | | **Stacked** | Vertical sections (header/main/footer) | Section names, Y positions | | **Sidebar** | Two-column with narrow sidebar | Sidebar side, widths | | **Freeform** | No clear pattern | Raw X/Y coordinates | ## Example Usage Once configured, Claude Code can use these tools: ``` User: Rebuild this webpage screenshot using the images in /assets Claude: I'll analyze the layout first using the layout detector. [Calls analyze_layout tool] The analysis shows: - Viewport: 900x650px - Pattern: Radial (90% confidence) - Center element: logo.gif at (450, 250) - 8 elements arranged around the center - Average distance from center: 196px I'll implement this using CSS with the logo centered and other elements positioned using absolute positioning... ``` ## Supported Image Formats - PNG - JPEG - GIF (including animated - uses first frame) - WebP - BMP ## Troubleshooting ### "No module named 'cv2'" OpenCV isn't installed. Run: ```bash pip install opencv-python-headless ``` ### MCP server not showing in `/mcp` 1. Check your settings file path is correct 2. Ensure the command path is absolute (for source installs) 3. Restart Claude Code after changing settings 4. Run `python3 test_install.py` to verify the package works ### Low confidence matches Try lowering the threshold parameter (default 0.8). Values between 0.6-0.7 may help with compressed or scaled images. ## Development ```bash # Install in editable mode with dev dependencies pip install -e ".[dev]" # Run tests pytest # Test installation python3 test_install.py ``` ## Requirements - Python 3.11+ - OpenCV (opencv-python-headless) - NumPy - Pillow - MCP SDK ## License MIT