```
████████╗██████╗ ██████╗ ██╗██╗      ██████╗ ████████╗
╚══██╔══╝██╔══██╗██╔══██╗██║██║     ██╔═══██╗╚══██╔══╝
   ██║   ██║  ██║██████╔╝██║██║     ██║   ██║   ██║
   ██║   ██║  ██║██╔═══╝ ██║██║     ██║   ██║   ██║
   ██║   ██████╔╝██║     ██║███████╗╚██████╔╝   ██║
   ╚═╝   ╚═════╝ ╚═╝     ╚═╝╚══════╝ ╚═════╝    ╚═╝
```

# TDPilot Runtime v2.4.1

[![CI](https://github.com/dreamrec/TDPilot/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/dreamrec/TDPilot/actions/workflows/ci.yml)
[![npm](https://img.shields.io/npm/v/tdpilot?label=npm)](https://www.npmjs.com/package/tdpilot)
[![downloads](https://img.shields.io/npm/dm/tdpilot?label=downloads)](https://www.npmjs.com/package/tdpilot)
[![license](https://img.shields.io/badge/license-MIT-blue)](./LICENSE)
[![python](https://img.shields.io/badge/python-3.10%2B-blue)](./pyproject.toml)
[![MCP tools](https://img.shields.io/badge/MCP%20tools-114-blueviolet)](./docs/API_REFERENCE.md)
[![TouchDesigner](https://img.shields.io/badge/TouchDesigner-2025.30000%2B-ff6200)](https://derivative.ca)

**Build, debug, and verify TouchDesigner networks by talking to your AI.**

TDPilot gives Claude (or Cursor, Codex, any MCP client) live control of a
running TouchDesigner session. The agent creates and wires operators, sets
real parameter values, **screenshots the render to check its own work**, and
can **roll back anything it did** — transactionally, with snapshots and a
full activity log. It knows TouchDesigner: every one of TD's operators is
covered by a reviewed knowledge atlas grounded in official Derivative docs.

<!-- demo-gif-slot: assets/demo-feedback-loop.gif — generated by the product
     itself via /td-first-wow + td_screenshot. See task: launch prep. -->

Try these, verbatim, once installed:

```
/td-first-wow
```
> Builds a moving feedback visual in your project, verifies it error-free, and shows you the screenshot — in about two minutes.

```
/td-concept A luminous particle tunnel with deep fog and slow camera drift
```
> Grounds the complete artistic request, reviews a concrete TD architecture, and executes it only when every required facet is covered.

```
Make my project audio-reactive: drive the main TOP chain from the kick drum of audio_in.
```
> Plans the CHOP analysis chain, wires it to your visuals, and proves the binding works before calling it done.

```
Why is my render black? Fix it.
```
> Reads errors recursively, inspects the render chain state, finds the broken link, fixes it, and screenshots the repaired output.

**Why artists trust it on real projects:** every mutation can be snapshot,
validated, and rolled back; risky parameter writes are gated by an
operator-semantics contract; and the whole session is auditable. No signup,
no API key, no hosted service — your work never leaves your machine.
On TouchDesigner 2025.33070+, fresh components also bind their Web Server DAT
to loopback by default instead of exposing the bridge on every interface.

## Install — Claude Code plugin (recommended)

**Easiest:** paste these two slash commands into any Claude Code session.

```
/plugin marketplace add dreamrec/TDPilot
/plugin install tdpilot@dreamrec-TDPilot
```

That installs the full MCP tool surface, the TDPilot skills (`tdpilot-core`, `tdpilot-production`, `popx-touchdesigner`, plus the public brain skills), the brain agents, the guided slash commands (`/td-concept`, `/td-first-wow`, `/td-audio-reactive`, `/td-explain-patch`, `/td-check`, `/td-snapshot`), and the TD-side `.tox` component — one command, no Python setup required.

**Shell one-liner alternative:**

```bash
curl -fsSL https://raw.githubusercontent.com/dreamrec/TDPilot/main/scripts/install_claude_plugin.sh | bash
```

**Or via npx:**

```bash
npx tdpilot plugin-install
```

### TouchDesigner side (once, after install)

Drag `~/.claude/plugins/cache/dreamrec-TDPilot/tdpilot/<version>/td_component/tdpilot.tox` into your TD `/local` container. Or paste the auto-setup Python block from [`docs/INSTALL_CLAUDE_PLUGIN.md`](docs/INSTALL_CLAUDE_PLUGIN.md) into the Textport (auto-detects the latest installed version).

Using Claude Desktop, Cursor, Codex, or another MCP client? See [`docs/INSTALL_OTHER_CLIENTS.md`](docs/INSTALL_OTHER_CLIENTS.md) — one page per client, including the one-click `.mcpb` bundle for Claude Desktop. (Don't mix the Desktop and Claude Code flows on one machine.)

## Documentation

**Which doc do I read?** Installing → `INSTALL_CLAUDE_PLUGIN.md`. First hour
→ `GETTING_STARTED.md`. Daily driving → `USER_GUIDE.md`. Running a show →
`MANUAL.md`. Looking up a tool → `API_REFERENCE.md`. Something broke →
`TROUBLESHOOTING.md`.

- Install (Claude Code plugin): `docs/INSTALL_CLAUDE_PLUGIN.md`
- Install (Claude Desktop / Cursor / Codex / generic MCP): `docs/INSTALL_OTHER_CLIENTS.md`
- Getting started: `docs/GETTING_STARTED.md`
- User guide: `docs/USER_GUIDE.md`
- Memory guide: `docs/MEMORY_GUIDE.md`
- Production manual: `docs/MANUAL.md`
- Practical Intelligence v2.4 architecture and guarantees: `docs/PRACTICAL_INTELLIGENCE.md`
- API reference: `docs/API_REFERENCE.md`
- Security model: `docs/SECURITY.md`
- Troubleshooting: `docs/TROUBLESHOOTING.md`
- Contributing: `CONTRIBUTING.md`
- Release notes: `CHANGELOG.md`
- Architecture deep-dives: `docs/TDPILOT_CONCEPT_TO_NODE_MASTER_PLAN.md`, `docs/TDPILOT_EFFECTIVENESS_ROADMAP.md`, `docs/MCP_1_1_SURFACE.md` (historical)

## What This Is

- A practical control layer between AI agents and TouchDesigner — built for iterative patch development, not one-shot guessing.
- A planner that grounds ideas in real operators: every TD operator family (CHOP, COMP, DAT, MAT, POP, SOP, TOP) is covered by a reviewed knowledge atlas sourced from official Derivative docs, with key parameters and gotchas per operator.
- A safety layer professionals can rely on: transactional apply, preflight checks, snapshots, verified rollback, parameter-semantics gating on risky writes, and an activity log of everything the agent touched.
- A technique memory that learns validated patterns from your projects and reuses them — exportable, importable, shareable.
- Diagnostics that close the loop: error scans, cook-time profiling, and screenshot verification so the agent proves its work instead of declaring it.

## Packaged Add-ons

- **Reviewed operator atlas** — The core local add-on for translating abstract ideas into real TD operators. Agents can use the 659-card reviewed operator atlas, Official Derivative source URLs, params, concepts, and gotchas to choose smaller, safer operator chains.
- **Brain skills and agents** — Codex and Claude Code ship explorer, builder, validator, and recovery workflows. Exact validated patterns use deterministic planning; artistic and multi-domain work uses ground -> author -> propose before transactional execution.
- **Scoped transaction hook** — The shipped hook runs only after brain/transaction mutations to reinforce final validation. Source-repository release guards check mirror drift, personal paths, and stale artifacts but are not shipped to user projects.
- **Optional local knowledge packs** — POPX and future specialty packs stay local and user-owned; they extend planning context without adding hosted-service dependencies.

## Start Here: Core Workflow

You don't need all 114 tools. Start with these and expand as needed:

| Step | Tools | What You're Doing |
|------|-------|-------------------|
| **Route a pattern** | `td_brain_plan` | Compile an exact validated topology, then require complete intent coverage |
| **Author a concept** | `td_brain_ground`, `td_brain_propose` | Ground artistic, multi-domain, spatial, or implicit architecture and review the authored graph |
| **Execute** | `td_brain_execute`, `td_transaction_apply` | Apply only a valid BrainPlan or PatchPlan with preflight, snapshots, rollback, and validation |
| **Inspect** | `td_get_info`, `td_get_nodes`, `td_get_params`, `td_get_errors` | Understand current state before touching anything |
| **Check memory** | `td_memory_recall` | See if a reusable technique already exists |
| **Build** | `td_create_node`, `td_connect_nodes`, `td_set_params` | Make changes in small, reversible steps |
| **Verify** | `td_get_errors`, `td_cooking_info`, `td_screenshot` | Prove the change worked |
| **Protect** | `td_snapshot_scene`, `td_restore_snapshot` | Save milestones, roll back if needed |
| **Remember** | `td_memory_learn`, `td_memory_save` | Save successful patterns for reuse |

**The loop:** Inspect -> Route -> Plan or Ground/Propose -> Execute transactionally -> Validate -> Learn only if validated.

Everything else (vision, streaming, optimization, planning, TD2025 inspection) builds on top of this core.

## Latest Release

**v2.4.1** — TouchDesigner 2025.33070 compatibility: exact release knowledge, reviewed Script POP and glTF In/Out COMP cards, native `.pop` round-trip guidance, release-driven live operator sampling, and loopback-by-default Web Server DAT builds. The 114-tool practical-intelligence surface remains local-first and backward compatible. Read the [Practical Intelligence guide](docs/PRACTICAL_INTELLIGENCE.md) or the full [`CHANGELOG.md`](CHANGELOG.md).

## Core Thinking Model (How To Think With This MCP)

Use this loop for every non-trivial task:

1. **Inspect the relevant scope** — Read focus, target nodes, direct connections, and parameters before touching anything. Batch independent reads when that saves a roundtrip.

2. **Check memory** — Before building from scratch, use `td_memory_recall` to check if a similar technique already exists in the library.

3. **Choose the route** — Use `td_brain_plan` for an exact validated pattern. Use `td_brain_ground` -> author -> `td_brain_propose` for artistic, multi-domain, spatial, or implicit architecture.

4. **Execute safely** — Execute only complete accepted plans, or use a small typed direct edit for one proven change. Validate at logical checkpoints rather than after every primitive operation.

5. **Validate the request** — Check graph, runtime, bindings/references, and the requested visual/temporal behavior. Zero TD errors alone is not visual proof.

6. **Learn and control cost** — Save only validated reusable work. Prefer metadata and one low-quality proof thumbnail; ask before repeated images or streaming.

## Tool Map (114 Tools)

### 0) Brain Planning + Transactions
Use for non-trivial visual programming tasks where complete intent coverage, rollback, and validation matter.

- `td_brain_plan`, `td_brain_ground`, `td_brain_propose`, `td_brain_execute`, `td_transaction_apply`, `td_cockpit_render`

### 1) Scene + Timeline + Project Lifecycle
Use for global context, playback control, save/load, and undo operations.

- `td_get_info`, `td_list_families`, `td_timeline`, `td_timeline_set`, `td_project_lifecycle`

### 2) Network Build + Wiring
Use for creating, moving, renaming, connecting, and pruning structure.

- `td_get_nodes`, `td_get_node_detail`, `td_search_nodes`
- `td_create_node`, `td_delete_node`, `td_copy_node`, `td_rename_node`
- `td_connect_nodes`, `td_disconnect`, `td_get_connections`

### 3) Parameters + DAT Content
Use for patch logic, expressions, config tables, scripts, and trigger pulses.

- `td_get_params`, `td_set_params`, `td_pulse_param`
- `td_get_content`, `td_set_content`, `td_custom_parameters`

### 4) Diagnostics + Capture
Use for proving behavior instead of assuming behavior.

- `td_screenshot`, `td_chop_data`, `td_geometry_data`, `td_pop_inspect`
- `td_cooking_info`, `td_get_errors`
- `td_exec_python`, `td_python_help`, `td_python_classes`

Structured exec note: `td_exec_python` now returns JSON-safe `result`, `result_type`, and `result_is_structured` fields. Use it for lightweight structured probes before reaching for stdout parsing.

### 5) Events + Streaming
Use for reactive and continuous workflows.

- `td_subscribe`, `td_unsubscribe`, `td_get_events`
- `td_capture_and_analyze`
- `td_monitor_visual`, `td_stop_monitor_visual`
- `td_stream_top`, `td_stop_stream_top`

Token guidance: start with `include_image=false` for monitors/streams. Use image payloads only when visual detail is explicitly required. Prefer `td_screenshot` for single checks.

### 6) Optimization + Dynamics
Use for quality passes and temporal behavior analysis.

- `td_optimize_visual` — now accepts direct `objective_weights` (e.g. `{"stability": 0.8, "complexity": 0.2}`)
- `td_describe_dynamics`

### 7) Safety + Recovery
Use for guardrails, emergency control, and rollback confidence.

- `td_set_param_bounds`, `td_clear_param_bounds`
- `td_detect_instability`, `td_emergency_stabilize`
- `td_snapshot_scene`, `td_list_snapshots`, `td_diff_snapshots`, `td_restore_snapshot`
- `td_get_state_vector`, `td_get_timescale_state`

### 8) Technique Memory & User Knowledge Store
Two parallel persistence surfaces — **technique memory** for replayable network recipes, **knowledge store** (new in v1.5.3) for free-form markdown reference essays (prose + math).

**Technique memory** — learning, saving, and replaying reusable network patterns:

- `td_memory_learn` — Analyze a live network subtree and extract a portable recipe. Auto-detects complexity: small/medium networks get full recipes with all params and expressions; large networks get structure summaries + key params.
- `td_memory_save` — Persist a technique to the project or global library.
- `td_memory_recall` — Search the library by text query and/or tags. Returns summaries.
- `td_memory_replay` — Rebuild a saved technique in a new location. Creates nodes, sets parameters and expressions, wires connections.
- `td_memory_list` — List all saved techniques with optional filtering.
- `td_memory_favorite` — Mark techniques as favorites and rate them (0-5).
- `td_memory_promote` — Copy a project-level technique to the global library for use across all projects.
- `td_memory_export` — Export the technique library as a portable JSON object for sharing or backup.
- `td_memory_import` — Import techniques from an exported library (from `td_memory_export`).
- `td_memory_preferences` — Get/set user preferences (color palettes, default resolutions, naming conventions, etc.)

**User knowledge store** *(new in v1.5.3)* — free-form markdown reference essays for prose-with-math reference content (BZ reaction equations, feedback recipes, "why this approach works" essays):

- `td_knowledge_save` — Persist a markdown body with name/description/tags/source/notes. Project- or global-scoped. Body capped at 200 KB; split larger writeups into linked entries.
- `td_knowledge_recall` — Search by free-text query and/or tags across name/description/tags/source/notes. Optional `full_text=true` also reads bodies (slower but more thorough).
- `td_knowledge_get` — Fetch full markdown body + metadata for one entry by id.
- `td_knowledge_list` — List entry summaries newest-first with optional filtering.

Storage lives at `~/.tdpilot/{memory,knowledge}/` with per-project and global scopes:
```
~/.tdpilot/
  memory/
    global/
      techniques.json
      preferences.json
    projects/{project_name}/
      techniques.json
      preferences.json
  knowledge/
    global/
      index.json
      entries/<uuid>.md
    projects/{project_name}/
      index.json
      entries/<uuid>.md
```

**Starter technique recipes** — the repo ships canonical live-visual recipes under `data/techniques_starter/` (one `td_memory_import`-format JSON file per technique: plain feedback loop, feedback displacement bloom, audio-reactive level pulse, beat-detected flash, kaleidoscope mirror, noise-displace flow, particle GPU sprite trail, camera post chain, LFO param wobble, edge-glow composite). Load one with a single call — pass the file's parsed JSON as the `data` argument:

```
td_memory_import(data=<contents of data/techniques_starter/plain_feedback_loop.json>, scope="global")
```

Parameter names are verified against the operator atlas cards (a test enforces this), but every entry ships `state: "candidate"` with `verified_on: null` — they have not yet been replayed against a live TD build. Replay one with `td_memory_replay`, and promote it once it validates.

### 9. Macros & Planning (7)
| Tool | Purpose |
|------|---------|
| `td_create_macro` | Create a reusable macro from a template |
| `td_list_macros` | List available macros |
| `td_get_macro_params` | Get macro parameter schema |
| `td_plan_patch` | Plan a multi-step network patch |
| `td_preflight_patch` | Pre-validate a patch plan |
| `td_validate_recipe` | Validate a technique recipe |
| `td_audit_project` | Audit project subtree |

### 10. Vision & Streaming (7)
| Tool | Purpose |
|------|---------|
| `td_capture_frame` | Capture a single frame from a TOP |
| `td_analyze_frame` | Analyze frame content (colors, regions) |
| `td_monitor_visual` | Start continuous visual monitoring |
| `td_stop_monitor_visual` | Stop visual monitoring |
| `td_stream_top` | Stream TOP output via WebSocket |
| `td_stop_stream_top` | Stop TOP streaming |
| `td_optimize_visual` | Get optimization suggestions for visuals |

### 11. Knowledge Corpus (7)
| Tool | Purpose |
|------|---------|
| `td_search_official_docs` | Search official TD documentation |
| `td_get_operator_doc` | Get detailed operator documentation |
| `td_get_param_help` | Get parameter-level help |
| `td_lookup_snippets` | Find code snippets by topic |
| `td_lookup_palette_component` | Look up Palette component info |
| `td_get_release_delta` | Get changes between TD builds |
| `td_get_build_compatibility` | Check operator build compatibility |

### 12. Server Introspection (3)
| Tool | Purpose |
|------|---------|
| `td_get_capabilities` | Report server capabilities |
| `td_get_server_metrics` | Get server performance metrics |
| `td_describe_surface` | Describe the full tool surface |

### 13. Recommendations (3)
| Tool | Purpose |
|------|---------|
| `td_recommend_official_component` | Suggest official components |
| `td_find_official_example` | Find relevant official examples |
| `td_explain_better_way` | Suggest better approaches |

### 14. TD 2025 Native (6)
| Tool | Purpose |
|------|---------|
| `td_python_env_status` | Inspect Python environment in TD |
| `td_threading_status` | Check threading configuration |
| `td_logger_status` | Inspect TD logger state |
| `td_tdresources_inspect` | Inspect TDResources categories |
| `td_component_standardize` | Audit/fix COMP standards |
| `td_color_pipeline` | Inspect color management pipeline |

## How To Use It (Practical Workflow)

1. Connect MCP client to TDPilot.
2. Ask for current project state.
3. Request a scoped patch goal.
4. Let agent apply changes in batches.
5. Require end-of-task `td_get_errors` check.
6. Save snapshot at stable milestone.
7. When you find something worth keeping: learn it, save it, rate it.

## What It Is Good At

- Building and refactoring operator networks quickly.
- Inspecting modern POP systems with attribute-aware reads.
- Converting high-level creative goals into concrete TD graph operations.
- Audio-reactive/control-system patch scaffolding.
- Automated cleanup, relayout, and consistency passes.
- Diagnosing wiring/parameter/runtime errors with direct evidence.
- Remembering what works and reusing it across projects.

## What It Is Not Good At

- Replacing artistic direction by itself.
- High-level show design without iterative user feedback.
- Unlimited always-on image streaming without token impact.
- Ignoring TD-specific context (operator families, cook behavior, timing model).
- "One shot perfect patch" generation in complex scenes.

## Network Design Protocol (Default Aesthetic Rules)

When generating or reorganizing networks: use color coding by role, keep clean spacing and avoid overlaps, group nodes into functional clusters, preserve clear flow direction, name nodes by purpose, and run `td_get_errors` after edits.

## Quick Setup

Recommended runtime (no manual Python setup in client config):

```bash
npx -y tdpilot
```

Local development runtime:

```bash
git clone https://github.com/dreamrec/TDPilot.git
cd TDPilot
uv sync
uv run tdpilot
```

### TouchDesigner Side

Run the setup script once inside the TD Textport:

```python
exec(open("/path/to/TDPilot/setup_mcp_in_td.py").read(), globals(), globals())
```

This installs the MCP component into `/local/mcp_server` by default, which means it **persists across project opens** within the same TD session. You only need to run this once — every project you open afterward will already have TDPilot available.

To install into a specific project instead: `os.environ["TD_MCP_PARENT_PATH"] = "/project1"` before running.

Alternatively, drag-and-drop `td_component/tdpilot.tox` into `/local` manually.

One-command setup helpers: macOS `./install.sh`, Windows `./install.ps1`

## MCP Bundle (Standardized)

TDPilot ships a standard bundle in-repo:

- `mcp/manifest.json`
- `mcp/profiles/claude-desktop.json`, `cursor.json`, `generic.json`

Auto-generate client config:

```bash
tdpilot init --client claude-desktop
tdpilot init --client cursor --output ./cursor_mcp_config.json
tdpilot init --client generic --print-only
```

## Doctor Command

Run a final environment/runtime check:

```bash
tdpilot doctor
tdpilot doctor --json
```

## Environment Variables

- `TD_MCP_HOST` (default `127.0.0.1` — supports hostnames like `desktop-3lurf0p.tail88651a.ts.net`)
- `TD_MCP_PORT` (default `9981`)
- `TD_MCP_SCHEME` (default `http` — set to `https` for Tailscale HTTPS or TLS-enabled setups)
- `TD_MCP_WS_PORT` (default `9982`)
- `TD_MCP_TRANSPORT` (`stdio` or `streamable_http`)
- `TD_MCP_HTTP_PORT` (default `8765`)
- `TD_MCP_CAPTURE_QUALITY` (default `0.3`)
- `TD_MCP_STREAM_MAX_FPS` (default `15.0`)
- `TD_MCP_EXEC_MODE` (`off`, `restricted`, `standard`, `full`)
- `TDPILOT_PROJECT_NAME` (set to enable per-project technique memory)
- `TDPILOT_MEMORY_DIR` (override default `~/.tdpilot/memory/` path)

## Test Suite

Run the test suite:

```bash
uv run --extra dev pytest tests/ -v
```

Planning quality is measured, not asserted: the brain is gated by a 50+ case concept-to-node golden eval corpus (`scripts/eval_brain_golden.py`) plus schema-snapshot, docs-truth, and packaging-mirror checks — all in CI.

## Reliability Habit

Treat this as mandatory for every meaningful task: before edits inspect, during edits take small reversible steps, after edits run `td_get_errors`, before risky changes snapshot.

## Community

- **Show what you built / ask anything** — [GitHub Discussions](https://github.com/dreamrec/TDPilot/discussions)
- **Share a working technique** — the [technique-share issue template](https://github.com/dreamrec/TDPilot/issues/new/choose) turns your patch (with real parameter values + a screenshot) into bundled TDPilot knowledge, with credit
- **Report a bug** — [issue templates](https://github.com/dreamrec/TDPilot/issues/new/choose) (the bug form asks for `tdpilot doctor --json`)
- **Contribute** — [CONTRIBUTING.md](CONTRIBUTING.md); techniques and hint packs land fastest
- **Security** — private disclosure via [GHSA](https://github.com/dreamrec/TDPilot/security/advisories/new), see [docs/SECURITY.md](docs/SECURITY.md)

## License

MIT

```
┌─────────────────────────────────────────────────────────────────────┐
│ dreamrec // TDPilot // live laugh love                             │
└─────────────────────────────────────────────────────────────────────┘
```