MCP TUI Test

Like Playwright, but for Terminal User Interfaces

An MCP (Model Context Protocol) server that enables AI assistants to test Terminal User Interface (TUI) applications. This server provides tools to launch, interact with, and verify TUI applications programmatically.

v1.0.0 — 22 tools; buffer-mode reads are column-exact and fail loudly when the child has exited (see the changelog for the 1.0 breaking changes).

📚 View Documentation

Features

• Dual Testing Modes: Choose between stream mode (for CLI tools) or buffer mode (for full TUIs with position awareness)

• Launch TUI Applications: Start any terminal-based application with configurable dimensions

• Send Keyboard Input: Simulate user typing, special keys, and control combinations

• Capture Screen Output: Read and analyze the current terminal display

• Screenshot & GIF: Render the screen to a PNG (colour, cursor, emoji) and stitch frames into an animated GIF

• Colour assertions: get_cell_style reads per-cell fg/bg and bold/reverse/italic/underline/strike, so colour-only states are assertable in text

• Live Resize: Resize the terminal mid-session to exercise responsive layouts

• Dead-child detection: Buffer-mode reads and every write fail loudly once the child has exited, instead of replaying a frozen frame (stream-mode reads stay, since a finished command's output is a record)

• Position-Based Testing: Verify text at specific screen coordinates (buffer mode)

• Cursor Tracking: Monitor cursor position in real-time (buffer mode)

• Wait for Text: Asynchronously wait for specific content to appear

• Assertions: Verify that expected content is present in the output

• Session Management: Run multiple TUI applications simultaneously

Related MCP server: HT MCP Server

Testing Modes

Stream Mode (Default)

• Best for: CLI tools, command-line applications, simple interactive programs

• Uses: pexpect for stream-based testing

• Features: Text matching, pattern waiting, output capture

• Example use cases: git, npm, grep, interactive shell scripts

Buffer Mode

• Best for: Full TUI applications, ncurses apps, dialog boxes, menus

• Uses: pexpect + pyte for screen buffer emulation

• Features: All stream features PLUS position-based assertions, cursor tracking, region extraction

• Example use cases: htop, vim, dialog, interactive menus

When to use which mode:

• Use stream mode for applications that output text sequentially

• Use buffer mode for applications that draw complex UIs with cursor movement

Installation

Prerequisites

• Python 3.10 or higher

• uv

Install Dependencies

uv venv
source .venv/bin/activate
uv pip install -r requirements.txt

Or install the package:

uv venv
source .venv/bin/activate
uv pip install -e .

Usage

Running the MCP Server

python server.py

Or if installed as a package:

mcp-tui-test

Configure in Claude Desktop

Add this to your claude_desktop_config.json:

{
  "mcpServers": {
    "tui-test": {
      "command": "python",
      "args": ["/path/to/mcp-tui-test/server.py"]
    }
  }
}

Or if installed as a package:

{
  "mcpServers": {
    "tui-test": {
      "command": "mcp-tui-test"
    }
  }
}

Agent Skill: TUI Scenario Walkthroughs

skills/tui-test/ ships an agent skill (oh-my-pi / Claude Code layout: SKILL.md + references/) that turns the tools above into a repeatable walkthrough workflow: pre-registered expectations, a machine-replayable scenario script linted and run by run_scenario, a transcript as evidence, a visual polish rubric, and a findings review.

A scenario is a markdown document with a name/description header and exactly one python block of bare step calls, parsed with ast and never executed. It may end with a choice() epilogue that reports which of several possible end states the app reached:

launch("env TERM=xterm-256color LC_ALL=C nano notes.txt", mode="buffer", dimensions="120x40")
expect("GNU nano", timeout=5)
send("hello", delay=0.5)
assert_at("hello", row=1, col=0)
choice({
    "saved cleanly": expect("Wrote 1 line", timeout=5),
    "still dirty": assert_text(" *"),
})

run_scenario(script=<document>, dry_run=True) returns lint diagnostics (with carets and did-you-mean hints) plus the resolved pipeline without launching anything; drop dry_run to run it and get the transcript. While iterating, pass the bare step list with ad_hoc=<block> instead of a whole document.

This repository also mounts itself for oh-my-pi sessions via .omp/mcp.json, so the tools appear as mcp__tui_test_launch_tui, mcp__tui_test_send_keys, … See skills/tui-test/references/mounting.md for user-level mounts and the headless fallback.

Available Tools

launch_tui

Launch a TUI application for testing.

Parameters:

• command (required): The command to launch the TUI application

• session_id (optional): Unique identifier for this session (default: "default")

• timeout (optional): Command timeout in seconds (default: 30)

• dimensions (optional): Terminal dimensions as WIDTHxHEIGHT (default: "80x24")

• mode (optional): Testing mode - "stream" or "buffer" (default: "buffer")

Examples:

# Stream mode for CLI tools
launch_tui(command="python example_tui_app.py", session_id="test1")

# Buffer mode for full TUI applications
launch_tui(command="htop", session_id="test2", mode="buffer", dimensions="120x40")

send_keys

Send keyboard input to a TUI application.

Parameters:

• keys (required): Keys to send. Use \n for Enter, \t for Tab, \x1b for Escape

• session_id (optional): Session identifier (default: "default")

• delay (optional): Delay in seconds after sending keys (default: 0.1)

Example:

send_keys(keys="1\n", session_id="test1")

send_ctrl

Send a Ctrl+Key combination to the TUI application.

Parameters:

• key (required): The key to combine with Ctrl (e.g., 'c', 'd', 'z')

• session_id (optional): Session identifier (default: "default")

Example:

send_ctrl(key="c", session_id="test1")

capture_screen

Capture the current screen output of a TUI application.

Parameters:

• session_id (optional): Session identifier (default: "default")

• include_ansi (optional): Whether to include ANSI escape codes in stream mode (default: False)

Examples:

capture_screen(session_id="test1")

get_cell_style (Buffer Mode Only)

Read one cell's colour and text attributes — the text-form witness of a colour-only state (a red failure marker, a dimmed row, a selection highlight).

Parameters:

• row (required): 0-indexed row number

• col (required): 0-indexed column number

• session_id (optional): Session identifier (default: "default")

Example:

get_cell_style(row=4, col=2)   # -> char='X' fg=red bg=default bold=no reverse=no ...

screenshot (Buffer Mode Only)

Render the current screen to a PNG — the only witness of colour, contrast, a colour-only state (a dimmed row, a selection highlight) and the cursor (focus). The picture is drawn from the same pyte cell grid the text tools read, with a per-glyph font fallback (a Nerd Font first) so emoji and status symbols render instead of .notdef boxes.

Parameters:

• path (optional): Where to write the PNG (default: a timestamped file under the system temp dir)

• session_id (optional): Session identifier (default: "default")

• font_size (optional): Font size in points, 4-96 (default: 14)

• cursor (optional): Paint the cursor as an inverted block (default: True)

Example:

screenshot(path="/tmp/state-after-import.png")

gif

Stitch PNGs from screenshot into an animated GIF (a "movie" of a session).

Parameters:

• frames (required): Two or more PNG paths, in play order

• path (optional): Where to write the GIF (default: a timestamped temp file)

• duration_ms (optional): Milliseconds each frame is shown (default: 500)

• loop (optional): 0 loops forever, N plays N+1 times (default: 0)

Example:

gif(frames=["/tmp/1.png", "/tmp/2.png", "/tmp/3.png"], path="/tmp/movie.gif")

resize

Resize a live session's terminal (the pty, so the child gets SIGWINCH) and let the app re-layout — the only way to drive a responsive layout in a headless run.

Parameters:

• dimensions (required): New size as WIDTHxHEIGHT, e.g. "80x20"

• session_id (optional): Session identifier (default: "default")

• delay (optional): Seconds to wait for the repaint (default: 0.3)

Example:

resize(dimensions="80x20")

expect_text

Wait for specific text to appear in the TUI output.

Parameters:

• pattern (required): Text or regex pattern to wait for

• session_id (optional): Session identifier (default: "default")

• timeout (optional): Maximum time to wait in seconds (default: 10)

Example:

expect_text(pattern="Welcome", session_id="test1", timeout=5)

assert_contains

Assert that the current screen contains specific text.

Parameters:

• text (required): Text to search for in the current screen

• session_id (optional): Session identifier (default: "default")

Example:

assert_contains(text="Counter value: 1", session_id="test1")

assert_at_position (Buffer Mode Only)

Assert that specific text appears at a screen position.

Parameters:

• text (required): Text to verify at the position

• row (required): Row number (0-indexed)

• col (required): Column number (0-indexed)

• session_id (optional): Session identifier (default: "default")

Example:

# Verify "Error" appears at row 5, column 10
assert_at_position(text="Error", row=5, col=10, session_id="test1")

get_cursor_position (Buffer Mode Only)

Get the current cursor position.

Parameters:

• session_id (optional): Session identifier (default: "default")

Example:

get_cursor_position(session_id="test1")
# Returns: "Cursor position (session: test1): row 10, column 25"

get_screen_region (Buffer Mode Only)

Extract a rectangular region of the screen.

Parameters:

• row_start (required): Starting row (0-indexed, inclusive)

• row_end (required): Ending row (0-indexed, exclusive)

• col_start (optional): Starting column (0-indexed, inclusive, default: 0)

• col_end (optional): Ending column (0-indexed, exclusive, default: end of line)

• session_id (optional): Session identifier (default: "default")

Example:

# Extract rows 5-10, full width
get_screen_region(row_start=5, row_end=10, session_id="test1")

# Extract rows 5-10, columns 20-60
get_screen_region(row_start=5, row_end=10, col_start=20, col_end=60, session_id="test1")

get_line (Buffer Mode Only)

Get a specific line from the screen buffer.

Parameters:

• row (required): Row number (0-indexed)

• session_id (optional): Session identifier (default: "default")

Example:

get_line(row=3, session_id="test1")
# Returns: "Line 3 (session: test1): [line content]"

close_session

Close a TUI testing session.

Parameters:

• session_id (optional): Session identifier (default: "default")

Example:

close_session(session_id="test1")

list_sessions

List all active TUI testing sessions.

Example:

list_sessions()

Example Test Scenario

Here's how you might use this MCP to test a TUI application:

1. Launch the application:

   launch_tui(command="python example_tui_app.py")

2. Wait for it to load:

   expect_text(pattern="Welcome to the Example TUI Application")

3. Interact with it:

   send_keys(keys="1\n")

4. Verify output:

   assert_contains(text="Hello, TUI Tester!")

5. Clean up:

   close_session()

Testing the Example App

This repository includes an example TUI application (example_tui_app.py) that you can use to test the MCP server.

Run it directly:

python example_tui_app.py

Or test it through the MCP:

launch_tui(command="python example_tui_app.py")
send_keys(keys="1\n")
capture_screen()

Use Cases

• Automated Testing: Verify TUI applications behave correctly

• Integration Testing: Test command-line tools and interactive CLIs

• Documentation: Generate screenshots and examples from TUI apps

• Debugging: Inspect the state of TUI applications during development

• CI/CD: Add TUI testing to your continuous integration pipeline

Technical Details

This MCP server uses:

• FastMCP: For the MCP server implementation

• pexpect: For spawning and controlling terminal applications

• pyte: For terminal emulation and screen buffer management (buffer mode)

• ScreenSession wrapper: Combines pexpect and pyte for hybrid testing

Architecture

• Stream Mode: pexpect directly captures output stream

• Buffer Mode: pexpect output → pyte terminal emulator → screen buffer

• Auto-detection: Tools automatically use appropriate mode based on session

Limitations

• Currently designed for Unix-like systems (Linux, macOS)

• Windows support may require modifications (consider using winpty or similar)

• Mouse support in TUIs is not currently available

• Buffer mode requires slightly more memory for screen emulation

• Position-based assertions only work in buffer mode

Contributing

Contributions are welcome! Please feel free to submit issues or pull requests.

License

MIT License - see LICENSE file for details

Related Projects

• Playwright - Browser automation (inspiration for this project)

• pexpect - Python module for spawning child applications

• pyte - Python terminal emulator

• MCP - Model Context Protocol specification

Author

Created for testing TUI applications with AI assistance.