SmartCLI

Read this in: English · 简体中文 · 繁體中文 · 日本語 · 한국어

A local Python toolkit for driving, perceiving, and rendering the terminal — three agent skills over one pluggable PTY + pyte core.

Let an AI drive, perceive, and render real terminal programs. SmartCLI reads the actual screen with a pyte cell model — not a byte pipe — so it knows which menu row is highlighted, presses the right keys, and waits for the screen to settle. Below: it drives the real lazygit TUI end-to-end (arrow-key navigation, opening a commit diff, highlighting a branch) — no script, no mock.

pip install smartcli-toolkit

Requires Python 3.10 or newer. The install includes the shared Python library, the persistent TUI driver, and the stdio MCP server.

Drive something in 30 seconds

Copy-paste this. It starts a real Python REPL under a PTY, waits for the prompt (never a blind sleep), types into it, and reads the screen back:

pip install smartcli-toolkit
SID=$(smartcli-tui start --cmd "python3 -i -q" --cols 80 --rows 24 --json | python3 -c "import json,sys;print(json.load(sys.stdin)['sid'])")
smartcli-tui wait-regex --id $SID ">>> " --timeout-ms 15000
smartcli-tui send-line --id $SID "print(6*7)"
smartcli-tui wait-regex --id $SID "42"          # prints the cell grid it sees
smartcli-tui close --id $SID

On Windows use --cmd "py -i -q". Swap the command for vim, htop or lazygit and the same five verbs drive those too — that is the whole point: wait-regex and friends react to what the screen actually shows, so an agent never guesses whether its keystroke landed.

Want the same thing against a real editor, end to end and verifiable? examples/drive_vim.py drives the actual vim binary — opens a file, appends a line, saves, and then checks the filesystem, not the screen:

python examples/drive_vim.py
#   [OK ] vim painted its screen
#   [OK ] file contents visible on screen
#   [OK ] alternate screen is active
#   [OK ] typed text appears on screen
#   [OK ] vim restored the main screen on exit
#   [OK ] file on disk really changed

Run the same file against smartcli-toolkit==0.1.8 and two steps fail — and the file is never saved, because a driver that cannot see the alternate screen mistimes the :wq. That is why the emulation work below matters: a wrong screen model does not error, it silently succeeds at nothing.

Already running an MCP client (Claude Code, Cursor, VS Code)? The same verbs are MCP tools, with the per-session token attached for you:

smartcli-mcp        # stdio MCP server; or `uvx --from smartcli-toolkit smartcli-mcp`

What & why

SmartCLI is a workspace for terminal work that agents and humans both do: driving interactive terminal programs, perceiving what a screen actually shows, and rendering visuals and layouts back out. It is built on one shared, pluggable PTY backend plus a pyte screen model — chosen over screenshot/vision so a single structured screen model feeds both perception (read the screen) and rendering (draw the screen). The PTY layer is intentionally not tmux-bound: local dev runs on Windows via ConPTY (pywinpty), while target programs can run under POSIX ptys or tmux elsewhere. Three skills sit on that core, each a self-contained tool you run in place from the checkout.

Related MCP server: Forge

Driving a real TUI

The demo above is SmartCLI driving lazygit — a real full-screen curses app — through its perceive → act → confirm loop: it reads the pyte cell grid (which row is selected, the alt-screen diff), moves with arrow keys, opens a commit's diff, and highlights a branch. Captured by driving the actual program in a Linux container, not scripted or mocked. A byte-stream matcher like pexpect can't perceive "which row is highlighted"; a screen model can.

How we know the perception is right. A screen model is only useful if it matches what a real terminal shows, so we measure that instead of asserting it: identical bytes go to a real tmux pane and to our model, and the two cell grids are diffed. Three suites do it — 35 curated cases, a three-way check that only trusts a behaviour when tmux and GNU screen agree, and a generative fuzz over random VT sequences. That campaign found and fixed 12 emulation bugs, including the alternate screen buffer (pyte implements none of modes 1049/1047/47, so a full-screen program's output used to be painted over the main screen and never restored). Scope and remaining edges: LIMITATIONS.md.

Live effects

Real captures of the cmd-art fx engine — each GIF is the actual effect rendered frame-by-frame through the project's own pipeline (no screen recorder). Reproduce any with python -m fx play <name> (see Quickstart).

🌐 Explore the live showcase → — play with the effect engine, drive a menu with arrow keys, and poke the widgets, right in your browser.

Install

Primary — from PyPI:

pip install smartcli-toolkit

Distribution vs import name: the PyPI distribution is smartcli-toolkit (the names smartcli / smart-cli were taken or blocked), but the importable package is smartcli_core. So after pip install smartcli-toolkit you still write from smartcli_core import PtySession.

Alternative — reproduce the full dev environment from a source checkout:

git clone https://github.com/dwgx/SmartCLI SmartCLI
cd SmartCLI
python -m pip install -r requirements.txt

requirements.txt installs pyte, the MCP SDK, and pywinpty on Windows only (POSIX uses the stdlib pty backend). pip install . installs smartcli_core plus the smartcli-tui, smartcli-mcp, and smartcli-toolkit commands. The visual cmd-art and tui-ui skills still run in place from a checkout via python -m fx and python -m ui.

Optional extras (real FIGlet fonts, raster images, authoritative cell widths — all degrade gracefully to stdlib fallbacks when absent):

python -m pip install -r requirements-optional.txt
# or, from the checkout, via pyproject extras:
pip install ".[all]"        # pyfiglet + Pillow + wcwidth
pip install ".[art]"        # pyfiglet only
pip install ".[image]"      # Pillow only  (also: the PNG screenshot harness needs it)
pip install ".[width]"      # wcwidth only

Windows note: set UTF-8 output before running any skill so box-drawing and CJK glyphs encode cleanly (the CLIs also auto-reconfigure stdout, but set this to be safe):

set PYTHONIOENCODING=utf-8

Verified dep versions on the dev box (Windows 11, CPython 3.14.6): pyte 0.8.2, pywinpty 3.0.5, pyfiglet 1.0.4, Pillow 12.2.0, wcwidth 0.8.1.

Diagnostics. python -m smartcli_core prints your OS, Python, terminal, PTY backend, and dependency versions. smartcli-tui doctor reports where the core was loaded from and whether drive dependencies are present. Include both outputs when filing a terminal-sensitive bug.

Quickstart

cmd-art — terminal visual effects

cd skills/cmd-art
python -m fx list                          # list all 30 effects
python -m fx play donut --seconds 5        # play one effect (bounded)
python -m fx gallery                       # one frame of each effect
python -m fx show --seq "donut:fire:3,plasma::3"

tui-ui — cell-accurate terminal UI

cd skills/tui-ui
python -m ui widgets                       # list all 17 widgets
python -m ui gallery --width 100 --height 30
python -m ui demo table --width 80 --height 12 --theme dashboard

drive-tui — perceive & drive interactive programs

Persistent-session CLI (state survives across shell calls):

smartcli-tui start --cmd "python3 -i -q" --cols 80 --rows 24
smartcli-tui wait-regex --id <SID> ">>> " --timeout-ms 15000
smartcli-tui send-line --id <SID> "print(6*7)"
smartcli-tui snapshot --id <SID>
smartcli-tui close --id <SID>

On Windows use py -i -q as the child command. From a source checkout, replace smartcli-tui with python skills/drive-tui/scripts/tui.py.

Or drive from any MCP client — the same verbs as MCP tools, with the per-session token attached automatically:

pip install smartcli-toolkit
smartcli-mcp     # stdio MCP server; smartcli-toolkit is an equivalent alias

As a library

The shared core is importable directly:

import sys
from smartcli_core import PtySession

s = PtySession()
s.start([sys.executable, "-q"])
s.wait_for(r">>> ")            # readiness sync, never a blind sleep
print(s.snapshot().to_text())  # pyte-backed structured screen
s.close()

For the full command reference, the screenshot/AGENTCLI harnesses, and the regression suite, see README-USAGE.md.

Features

cmd-art (skills/cmd-art) — a "living-template" effect engine: an Effect ABC + @register decorator + auto-discovery. 30 effects (donut, solarsystem, fire, plasma, rain, starfield, tunnel, text3d, cube, sphere, boids, life, fireworks, sparkle, decrypt, gradient_text, banner_scroll, image2ascii, typewriter, julia, mandelbrot, perlin, flames, water, nebula, text_flyin, text_converge, text_decrypt, spectrum_bars, cbonsai) across 8 themes (mono, fire, ocean, synthwave, viridis, pastel, matrix-green, rainbow). Effects are pure frame producers; play is bounded by default and always restores the terminal.

tui-ui (skills/tui-ui) — a web-like terminal layout engine emitting tmux-safe ANSI frames (SGR color runs + newlines only; no cursor moves, no alt-screen). 17 widgets (badge, banner, braille_chart, card, fuzzy_filter_list, gradient_rule, kv, meter, panel, preview_pane, progress, radial_glow, rule, slider_track, table, tabs, tree) over a real engine: field.py (shader compositors), raster.py (sub-cell half/quad/braille pixels), box_junction.py (edge-algebra box joins), color_model.py (honest truecolor → 256 → 16 → mono degrade). Display-cell accurate for CJK/emoji/ZWJ so columns never desync.

drive-tui (skills/drive-tui) — drives interactive terminal programs (REPLs, menus, pagers, y/N prompts, wizards) through a PTY via a perceive → decide → act → wait → confirm loop, never a blind sleep. A thin CLI (scripts/tui.py) offers a persistent detached session and a one-shot run mode, with an importable pattern library of 8 recipes (repl, menu_select, pager, search_filter, confirm, form, progress, wizard) that classify() a screen and drive() it.

Shared core (smartcli_core) — the pluggable PTY backend + pyte screen model + semantic snapshot + readiness sync (pty_backend / screen_model / snapshot / readiness / session). The reusable, importable foundation under all three skills.

Knowledge graph (knowledge/) — a wiki-link graph (140+ .md files) of exact rendering formulas, ANSI sequences, and measured constants, each note carrying a source and cross-links. See knowledge/INDEX.md.

Project layout

SmartCLI/
  smartcli_core/           shared PTY + pyte engine (importable package)
  skills/cmd-art/          fx effect package and CLI (30 effects, 8 themes)
  skills/drive-tui/        TUI pattern library and PTY driver CLI (8 recipes)
  skills/tui-ui/           terminal UI layout engine and widgets (17 widgets)
  tools/screenshot/        pyte -> PNG smoke-test harness
  tools/agentcli/          agent-CLI control validation harness
  knowledge/               wiki-link knowledge graph, 140+ .md files (see knowledge/INDEX.md)
  showcase/                rendered effect PNGs + demo GIFs (shown above)
  tests/                   direct script-style regressions
  research/                archived first-pass research notes

Documentation

• README-USAGE.md — the full usage cheat-sheet: every skill, the screenshot and AGENTCLI harnesses, and the regression commands.

• knowledge/INDEX.md — the knowledge graph (140+ .md files).

• AGENTCLI-VALIDATION.md — agent-CLI control test matrix.

• CHANGELOG.md — release history.

License

MIT — see LICENSE.