# kayamcp 🎨🤖

**LibreSprite Visual MCP Server** — Real-time screen capture, visual UI interaction, and self-healing pixel art automation for [LibreSprite](https://libresprite.github.io/).

> An AI agent that can **see** your screen, **detect** LibreSprite's UI, **draw** alongside you in real-time, and **self-correct** any pixel errors automatically.

[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://python.org)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![Tests: 71 passed](https://img.shields.io/badge/tests-71%20passed-brightgreen.svg)](#testing)

---

## ✨ Features

| Feature | Module | Description |
|---------|--------|-------------|
| 🖥️ **Ultra-Fast Screen Streaming** | `capture.py` | 10-60 FPS screen capture via MSS with pre-allocated buffers and zero-copy reads |
| 🎯 **Dynamic Calibration** | `calibration.py` | Auto-detects LibreSprite window and canvas boundaries using Win32 + OpenCV contour analysis |
| 🔧 **Visual UI Interaction** | `ui.py` | Selects tools (pencil, eraser, fill…), colours from the palette, and manages layers |
| ✏️ **Cooperative Drawing** | `draw.py` | Moves cursor, executes stabilised strokes, and draws alongside the user in real-time |
| 🔄 **Self-Healing Loop** | `feedback.py` | Verifies drawn pixels post-stroke and auto-corrects mismatches with retry logic |
| 🌐 **MCP Server** | `mcp_server.py` | JSON-RPC 2.0 server (stdio/HTTP) implementing the Model Context Protocol |

---

## 🚀 Quick Start

### Prerequisites

- **Windows 10/11** (uses Win32 API for desktop switching and window detection)
- **Python 3.10+**
- **LibreSprite** installed and running

### Installation

```bash
# Clone the repository
git clone https://github.com/YOUR_USERNAME/kayamcp.git
cd kayamcp

# Install with pip
pip install -e .

# Or with uv (recommended)
uv pip install -e .
```

### Run the MCP Server

```bash
# stdio mode (for MCP clients like Claude Desktop, Cursor, etc.)
python -m kayamcp

# HTTP mode (for development/testing)
python -m kayamcp --http --port 7799
```

---

## 🔌 MCP Client Configuration

### Claude Desktop / Cursor

Add this to your MCP client configuration:

```json
{
  "mcpServers": {
    "kayamcp": {
      "command": "python",
      "args": ["-m", "kayamcp"],
      "cwd": "C:/path/to/kayamcp"
    }
  }
}
```

### Available MCP Tools

| Tool | Description |
|------|-------------|
| `kayamcp_capture_frame` | Capture current screen frame and return dimensions |
| `kayamcp_calibrate` | Re-calibrate LibreSprite window and canvas boundaries |
| `kayamcp_get_canvas_rect` | Get current canvas rectangle `{left, top, width, height}` |
| `kayamcp_select_tool` | Select a drawing tool (`pencil`, `eraser`, `fill`, `eyedropper`, etc.) |
| `kayamcp_select_color` | Select an RGB colour for drawing |
| `kayamcp_select_layer` | Select an existing layer by name |
| `kayamcp_add_layer` | Add a new layer |
| `kayamcp_draw_stroke` | Draw a stroke through canvas coordinate points |
| `kayamcp_stabilize_stroke` | Smooth points for jitter-free drawing |
| `kayamcp_verify_pixels` | Verify canvas pixels match expected colours |
| `kayamcp_heal_canvas` | Self-heal: verify + redraw mismatched pixels with retries |

---

## 🏗️ Architecture

```
kayamcp/
├── __init__.py          # Package exports
├── __main__.py          # python -m kayamcp entry point
├── desktop.py           # Win32 desktop switching (Default desktop context)
├── capture.py           # High-FPS screen streaming (MSS, threaded)
├── calibration.py       # LibreSprite window & canvas detection (Win32 + OpenCV)
├── ui.py                # Visual UI interaction (tools, palette, layers)
├── draw.py              # Cooperative drawing & cursor control
├── feedback.py          # Closed-loop pixel verification & self-healing
└── mcp_server.py        # JSON-RPC 2.0 MCP server (stdio + HTTP)
```

### Data Flow

```
┌─────────────┐    ┌──────────────┐    ┌──────────────┐
│ MCP Client  │◄──►│  mcp_server  │◄──►│   capture    │
│ (AI Agent)  │    │  (JSON-RPC)  │    │ (Screen FPS) │
└─────────────┘    └──────┬───────┘    └──────┬───────┘
                          │                    │
                   ┌──────▼───────┐    ┌──────▼───────┐
                   │  calibration │◄──►│   desktop    │
                   │ (Win32+CV2)  │    │ (Win32 API)  │
                   └──────┬───────┘    └──────────────┘
                          │
              ┌───────────┼───────────┐
              │           │           │
       ┌──────▼───┐ ┌─────▼────┐ ┌───▼──────┐
       │   ui     │ │  draw    │ │ feedback  │
       │ (tools,  │ │ (cursor, │ │ (verify,  │
       │  colors) │ │ strokes) │ │  heal)    │
       └──────────┘ └──────────┘ └──────────┘
```

---

## 🧪 Testing

The project includes a comprehensive test suite with **71 tests** across 4 tiers:

```bash
# Run all tests
uv run pytest -v

# Run only the adversarial stress tests
uv run pytest tests/test_adversarial_streamer.py -v

# Run only E2E integration tests
uv run pytest tests/test_e2e.py -v
```

### Test Tiers

| Tier | Focus | Tests |
|------|-------|-------|
| **Tier 1** | Feature coverage (streaming, calibration, UI, drawing, healing) | 25 |
| **Tier 2** | Boundary & corner cases (FPS limits, invalid inputs, occlusion) | 25 |
| **Tier 3** | Cross-feature combinations (calibrate+draw, select+draw+heal) | 5 |
| **Tier 4** | Real-world scenarios (interactive painting, colour-by-number, zoom/pan) | 5 |
| **Tier 5** | Adversarial stress (handle leaks, memory leaks, CPU efficiency, thread cleanup) | 4 |
| | | **71 total** |

---

## 📦 Packaging & Distribution

### Build a distributable package

```bash
# Build wheel and sdist
pip install build
python -m build

# The built files will be in dist/
#   dist/kayamcp-0.1.0-py3-none-any.whl
#   dist/kayamcp-0.1.0.tar.gz
```

### Publish to PyPI

```bash
pip install twine
twine upload dist/*
```

---

## 🔧 Dependencies

| Package | Version | Purpose |
|---------|---------|---------|
| `mss` | ≥9.0.0 | High-performance screen capture |
| `opencv-python` | ≥4.8.0 | Canvas detection via contour analysis |
| `pywin32` | ≥306 | Win32 API (desktop switching, window enumeration) |
| `pydirectinput` | ≥0.1.7 | Direct mouse/keyboard input |
| `numpy` | ≥1.24.0 | Frame buffer operations |
| `pillow` | ≥9.5.0 | Image format support |

---

## 🤝 Contributing

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Run the test suite (`uv run pytest -v`)
4. Commit your changes (`git commit -m 'Add amazing feature'`)
5. Push to the branch (`git push origin feature/amazing-feature`)
6. Open a Pull Request

---

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

---

## 🙏 Acknowledgements

- [LibreSprite](https://libresprite.github.io/) — Free and open-source pixel art editor
- [MSS](https://python-mss.readthedocs.io/) — Ultra-fast cross-platform screen capture
- [Model Context Protocol](https://modelcontextprotocol.io/) — Open protocol for AI tool integration