# RokuHarness MCP Server

A Model Context Protocol (MCP) server for comprehensive Roku automated testing. RokuHarness combines **Roku WebDriver** and **ECP (External Control Protocol)** to provide both UI state verification and device control - enabling true acceptance criteria validation, not just remote control simulation.

## Why WebDriver vs ECP?

| Capability | ECP | WebDriver |
|-----------|----------------------------------|-----------|
| Press remote buttons | ✅ | ✅ |
| Launch apps | ✅ | ✅ |
| **Query UI elements** | ❌ | ✅ |
| **Verify text is on screen** | ❌ | ✅ |
| **Check element attributes** | ❌ | ✅ |
| **Get SceneGraph XML** | ❌ | ✅ |
| **Take screenshots** | ❌ | ✅ |
| **Validate acceptance criteria** | ❌ | ✅ |

**Bottom line**: ECP can press buttons, WebDriver can verify what happens.

## Architecture

```
Your Tests (via any MCP client)
    ↓
RokuHarness MCP Server (this project)
    ↓ (WebDriver HTTP API)
Roku WebDriver Server (from Roku's repo)
    ↓ (ECP + Debug APIs)
Roku Device (your sideloaded channel)
```

**Key Point**: The Roku WebDriver Server uses BOTH ECP (for control) and Roku's debug APIs (for UI verification). This MCP server provides a unified interface to both capabilities.

## Prerequisites

### 1. Roku WebDriver Server

You need to download and run Roku's official WebDriver server:

```bash
# Clone Roku's automated testing repo
git clone https://github.com/rokudev/automated-channel-testing.git
cd automated-channel-testing

# Build the WebDriver server (requires Go)
cd src
go build

# Run the server
./RokuWebDriver  # Linux/Mac
# or
RokuWebDriver.exe  # Windows
```

The server will start on `http://localhost:9000` by default.

**Download pre-built binaries**: Check the `automated-channel-testing/bin` folder for pre-compiled executables.

### 2. Sideloaded Channel

WebDriver requires your channel to be sideloaded in developer mode:

1. Enable developer mode on your Roku: Settings → System → About → Press Home 5x, Up, Rewind 2x, Fast Forward 2x
2. Package your channel as a `.zip` file
3. Visit `http://YOUR_ROKU_IP` in a browser
4. Upload and install your channel

**Important**: WebDriver only works with:
- Sideloaded developer channels (app ID: `dev`)
- Channels packaged with your developer account on that specific device
- SceneGraph-based channels (not legacy BrightScript)

### 3. This MCP Server

```bash
npm install
npm run build
```

## Installation & Setup

### 1. Build this MCP server

```bash
npm install
npm run build
```

### 2. Configure Your MCP Client

**For Claude Desktop:**

Add to `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "rokuharness": {
      "command": "node",
      "args": ["/absolute/path/to/rokuharness-mcp/build/index.js"]
    }
  }
}
```

**For Custom MCP Clients:**

RokuHarness is MCP-agnostic and works with any MCP client. See "Using Without Claude Desktop" section below for integration examples.

### 3. Start the Roku WebDriver Server

In a separate terminal:

```bash
cd /path/to/automated-channel-testing/bin
./RokuWebDriver  # or RokuWebDriver.exe on Windows
```

Keep this running while testing.

### 4. Restart Claude Desktop

The MCP server will now be available in Claude.

## Usage Guide

### Step 1: Create a Session

Every test session starts by creating a WebDriver session:

```
Create a Roku WebDriver session:
- WebDriver URL: http://localhost:9000
- Roku IP: 192.168.1.100
- App: dev
```

This connects to your sideloaded channel.

### Step 2: Get UI Source (Critical!)

Before writing any verification queries, inspect the UI:

```
Get the UI source to see what elements are on screen
```

You'll get back SceneGraph XML like:

```xml
<Scene>
  <Label id="titleLabel" text="Welcome to My App" focused="false" />
  <Button id="loginButton" text="Log In" focused="true" />
  <Poster id="heroImage" uri="https://..." />
</Scene>
```

This XML shows you:
- **Tags**: Component types (Label, Button, Poster, etc.)
- **Attributes**: Properties like `id`, `text`, `focused`, `visible`
- **Hierarchy**: Nested structure

### Step 3: Verify Elements

Now you can write acceptance criteria tests:

```
Verify that a Label with text "Welcome to My App" is present on screen
```

```
Verify that a Button with id "loginButton" and focused=true is present
```

```
Verify that the login button text says "Log In"
```

### Step 4: Navigate and Test

```
Navigate using these keys: ["Down", "Down", "Select"]
Then verify that a Label with text "Account Settings" appears
```

### Step 5: Run Complete Test Scenarios

```
Run this acceptance test:

Test: Login Flow
Steps:
1. Press Select to click login button
2. Verify keyboard screen appears (look for Label with text "Enter Email")
3. Send text "test@example.com"
4. Press Down to go to password field
5. Send text "password123"
6. Press Select to submit
7. Verify success message appears
8. Take screenshot
```

## Available Tools

### Session Management

#### `create_webdriver_session`
Creates a new WebDriver session.

**Parameters:**
- `webdriver_url` (optional): URL of WebDriver server (default: http://localhost:9000)
- `roku_ip` (required): IP address of Roku device
- `app` (optional): App ID or "dev" for sideloaded (default: "dev")

**Example:**
```
Create a WebDriver session for Roku at 192.168.1.100
```

#### `end_webdriver_session`
Ends the current session and cleans up.

---

### UI State Verification (The Key Features!)

#### `get_ui_source`
Get the current UI hierarchy as XML or JSON.

**Parameters:**
- `parsed` (optional): Return JSON instead of XML

**Example:**
```
Get the UI source to see current screen structure
```

**Returns:**
```xml
<Scene>
  <LayoutGroup id="mainLayout">
    <Label id="title" text="Home Screen" />
    <Button id="playButton" text="Play" focused="true" />
  </LayoutGroup>
</Scene>
```

This is **essential** for understanding what elements exist and how to query them.

#### `find_element`
Search for a specific element on screen.

**Parameters:**
- `text` (optional): Text content to match
- `tag` (optional): SceneGraph component type
- `attributes` (optional): Attribute key-value pairs

**Examples:**
```
Find a Label with text "Home Screen"
```

```
Find a Button with id "playButton"
```

```
Find an element with tag "Poster" and attribute uri="https://example.com/image.jpg"
```

#### `verify_element_present`
Check if an element exists (returns true/false).

**Parameters:**
- Same as `find_element`
- `timeout_ms` (optional): How long to wait (default: 10000)

**Examples:**
```
Verify a Label with text "Loading..." is present
```

```
Check if login button is focused: Button with focused=true
```

#### `verify_screen_loaded`
Wait for a specific screen to fully load.

**Parameters:**
- `screen_marker`: Element query that identifies the screen
- `timeout_ms` (optional): Maximum wait time

**Example:**
```
Verify the home screen loaded by checking for Label with text "Featured Content"
```

---

### Navigation & Input

#### `press_key`
Press a single remote button.

**Parameters:**
- `key`: Button name (Home, Back, Up, Down, Left, Right, Select, Play, Pause, etc.)

#### `navigate`
Execute a sequence of button presses.

**Parameters:**
- `keys`: Array of keys to press
- `delay_ms` (optional): Delay between presses

**Example:**
```
Navigate: Down, Down, Right, Select with 750ms delays
```

#### `send_text`
Send text input (for keyboards/forms).

**Parameters:**
- `text`: Text to type

---

### Media & Apps

#### `launch_app`
Launch an app with optional deep linking.

**Parameters:**
- `app_id`: App ID ("dev" for sideloaded)
- `content_id` (optional): Deep link content ID
- `media_type` (optional): Type (movie, series, etc.)

#### `get_player_state`
Get current playback state.

**Returns:** Position, duration, state, buffering info

#### `get_installed_apps`
List all installed apps.

---

### Screenshots

#### `take_screenshot`
Capture current screen.

**Parameters:**
- `save_path` (optional): Where to save the image

**Example:**
```
Take a screenshot and save to /tmp/login_screen.png
```

---

### Acceptance Testing

#### `run_acceptance_test`
Execute a complete test case with multiple steps.

**Parameters:**
- `test_name`: Name of the test
- `steps`: Array of test steps

**Step types:**
- `navigate`: Execute key sequence
- `verify_element`: Check element is present
- `press_key`: Press single key
- `send_text`: Type text
- `wait`: Pause for duration
- `screenshot`: Capture screen

**Example:**
```
Run this acceptance test:

Name: "Video Playback Test"

Steps:
1. Action: navigate, Keys: ["Down", "Down", "Select"], Description: "Select first video"
2. Action: verify_element, Query: {tag: "Video", attributes: {state: "playing"}}, Description: "Verify video is playing"
3. Action: wait, Duration: 5000, Description: "Let video play for 5 seconds"
4. Action: press_key, Key: "Pause", Description: "Pause playback"
5. Action: verify_element, Query: {tag: "Video", attributes: {state: "paused"}}, Description: "Verify video paused"
6. Action: screenshot, Description: "Capture paused state"
```

---

## Element Query Syntax

Elements are queried using combinations of:

### By Text
```json
{ "text": "Log In" }
```
Finds elements containing this exact text.

### By Tag
```json
{ "tag": "Button" }
```
Finds elements of this SceneGraph type.

Common tags:
- `Label` - Text display
- `Button` - Interactive button
- `Poster` - Image
- `Video` - Video player
- `LayoutGroup` - Container
- `RowList` - Scrollable list
- `Grid` - Grid layout

### By Attributes
```json
{
  "attributes": {
    "id": "loginButton",
    "focused": "true"
  }
}
```

Common attributes:
- `id` - Unique identifier
- `focused` - Has focus (true/false)
- `visible` - Is visible (true/false)
- `text` - Text content
- `uri` - Image/video URI

### Combined Queries
```json
{
  "tag": "Button",
  "text": "Log In",
  "attributes": {
    "focused": "true"
  }
}
```

Finds a Button with text "Log In" that currently has focus.

---

## Real-World Examples

### Example 1: Validate Login Screen

```
1. Create WebDriver session for Roku at 192.168.1.100

2. Get UI source to inspect elements

3. Verify these elements are present:
   - Label with text "Sign In"
   - Button with text "Email Login"
   - Button with text "Guest Mode"

4. Take screenshot for documentation
```

### Example 2: Test Video Playback

```
Run this acceptance test:

Name: "Video Playback Verification"

Steps:
1. Navigate to content: ["Down", "Down", "Select"]
2. Verify video player loaded: tag=Video
3. Wait 3 seconds for playback to start
4. Verify video is playing: tag=Video, attributes={state: "playing"}
5. Press "Info" to show controls
6. Verify play/pause button visible: tag=Button, text="Pause"
7. Take screenshot of player controls
```

### Example 3: Search Functionality

```
Test search feature:

1. Press "Search" key
2. Verify keyboard screen: Label with text "Search"
3. Send text "Breaking Bad"
4. Press "Select" to submit
5. Verify results loaded: Label with text "Results for: Breaking Bad"
6. Verify at least one result: tag=Poster (poster images indicate results)
```

### Example 4: Settings Navigation

```
Navigate to settings and verify:

1. Press Home
2. Navigate: ["Down", "Down", "Down", "Right", "Right", "Select"]
3. Verify settings screen: Label with text "Settings"
4. Navigate: ["Down", "Select"]
5. Verify account screen: Label with text "Account Information"
6. Get UI source to document screen structure
```

---

## Troubleshooting

### "No active session"
You must call `create_webdriver_session` before any other commands.

### "WebDriver server not responding"
Ensure the Roku WebDriver server is running:
```bash
./RokuWebDriver
```

### "Element not found"
1. Get the UI source first: `get_ui_source`
2. Inspect the actual XML structure
3. Adjust your query to match actual elements
4. Check spelling and capitalization (XML is case-sensitive)

### "Cannot get source from channel"
- Only works with sideloaded channels or channels packaged on that device
- Production channels block source access (security feature)
- Make sure your channel is running (not on Home screen)

### Screenshots not working
Screenshots only work when:
- Your sideloaded channel is active
- Developer mode is enabled
- WebDriver has proper access

### Slow queries
- WebDriver queries can take 500ms-2s depending on complexity
- Use specific queries (tag + attributes) for faster results
- Avoid overly broad queries

---

## Roku WebDriver Limitations

1. **Sideloaded channels only** - Production channels block UI introspection
2. **SceneGraph only** - Legacy BrightScript channels not supported
3. **No visual comparisons** - You get XML structure, not rendered pixels
4. **Element bounds are relative** - Absolute position checking is complex
5. **No direct element interaction** - You still navigate with D-pad, not clicks

---

## Best Practices

### 1. Always Inspect First
```
Get UI source → Understand structure → Write queries
```

### 2. Use Specific Queries
```
❌ { "text": "Play" }  // Might match multiple elements
✅ { "tag": "Button", "id": "mainPlayButton", "text": "Play" }
```

### 3. Wait for Screens to Load
```
verify_screen_loaded with appropriate timeout
```

### 4. Test One Thing at a Time
Break complex flows into individual test cases.

### 5. Take Screenshots
Document state before/after critical steps.

### 6. Use Acceptance Test Tool
For multi-step scenarios, use `run_acceptance_test` to get structured results.

---

## Integration with CI/CD

This MCP server can be integrated into CI/CD pipelines:

1. Provision Roku devices in your test lab
2. Start WebDriver server on each
3. Run tests via MCP server
4. Collect results and screenshots
5. Publish test reports

---

## Using Without Claude Desktop

RokuHarness is built on the open MCP standard and works with **any MCP client**. Here are integration examples:

### Python CI/CD Integration

```python
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client

# Connect to RokuHarness MCP server
server_params = StdioServerParameters(
    command="node",
    args=["/path/to/rokuharness-mcp/build/index.js"]
)

async with stdio_client(server_params) as (read, write):
    async with ClientSession(read, write) as session:
        await session.initialize()
        
        # Create WebDriver session
        result = await session.call_tool(
            "create_webdriver_session",
            arguments={
                "roku_ip": "192.168.1.100",
                "app": "dev"
            }
        )
        
        # Get UI source
        ui_source = await session.call_tool("get_ui_source", {})
        
        # Verify element
        verify_result = await session.call_tool(
            "verify_element_present",
            arguments={
                "text": "Welcome",
                "tag": "Label"
            }
        )
        
        # Assert in your test framework
        assert verify_result["present"] == True
        
        # Run acceptance test
        test_result = await session.call_tool(
            "run_acceptance_test",
            arguments={
                "test_name": "Login Flow",
                "steps": [
                    {
                        "action": "navigate",
                        "description": "Go to login",
                        "keys": ["Down", "Down", "Select"]
                    },
                    {
                        "action": "verify_element",
                        "description": "Check login screen",
                        "element_query": {"text": "Sign In"}
                    }
                ]
            }
        )
        
        print(f"Test Status: {test_result['summary']['status']}")
```

### Node.js Integration

```javascript
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';

const transport = new StdioClientTransport({
  command: 'node',
  args: ['/path/to/rokuharness-mcp/build/index.js']
});

const client = new Client({
  name: 'roku-test-runner',
  version: '1.0.0'
}, {
  capabilities: {}
});

await client.connect(transport);

// Create session
const session = await client.request({
  method: 'tools/call',
  params: {
    name: 'create_webdriver_session',
    arguments: {
      roku_ip: '192.168.1.100',
      app: 'dev'
    }
  }
});

// Verify element
const result = await client.request({
  method: 'tools/call',
  params: {
    name: 'verify_element_present',
    arguments: {
      tag: 'Button',
      text: 'Play'
    }
  }
});

console.log('Element present:', result.present);
```

### Integration Points

✅ **CI/CD Pipelines** - Jenkins, GitHub Actions, GitLab CI, CircleCI  
✅ **Test Frameworks** - Jest, Mocha, Pytest, JUnit  
✅ **QA Platforms** - TestRail, Zephyr, qTest, Xray  
✅ **Custom Dashboards** - Build your own test runner UI  
✅ **Scheduled Testing** - Cron jobs, AWS Lambda, Azure Functions  
✅ **Any MCP-compatible tool** - The protocol is completely open

### Example GitHub Actions Workflow

```yaml
name: Roku UI Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '20'
      
      - name: Install dependencies
        run: |
          cd rokuharness-mcp
          npm install
          npm run build
      
      - name: Start Roku WebDriver Server
        run: |
          wget https://github.com/rokudev/automated-channel-testing/releases/download/v1.0/RokuWebDriver
          chmod +x RokuWebDriver
          ./RokuWebDriver &
          
      - name: Run tests
        run: python tests/run_roku_tests.py
        env:
          ROKU_IP: ${{ secrets.ROKU_IP }}
          
      - name: Upload screenshots
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: test-screenshots
          path: screenshots/
```

---

## Comparison with Other Tools

### vs Roku Robot Framework
- **This**: Natural language via Claude, MCP protocol
- **Robot**: Keyword-driven, separate test files

### vs Appium Roku Driver
- **This**: Direct WebDriver access, simpler setup
- **Appium**: Appium ecosystem integration, more tooling

### vs Manual Testing
- **This**: Automated, repeatable, fast
- **Manual**: Comprehensive but slow, expensive

---

## Resources

- [Roku WebDriver Documentation](https://developer.roku.com/docs/developer-program/dev-tools/automated-channel-testing/web-driver.md)
- [Automated Channel Testing Repo](https://github.com/rokudev/automated-channel-testing)
- [SceneGraph XML Reference](https://developer.roku.com/docs/references/scenegraph/xml-elements.md)
- [Roku Developer Portal](https://developer.roku.com/)

---

## Support

For issues or questions:
- Roku WebDriver: [Roku Community Forums](https://community.roku.com/t5/Developers/ct-p/channel-developers)
- This MCP Server: Open an issue in this repository
- Claude/MCP: [Anthropic Documentation](https://modelcontextprotocol.io/)

---

## License

MIT License - Free to use for your Roku testing needs.