# TouchDesigner AI Companion

A local desktop tool that captures your TouchDesigner node network via hotkey, sends the screenshot to Claude (with vision + function calling), and returns targeted, context-aware answers about your patches — with full session memory and Langfuse observability.

## Stack

| Layer | Tool |
|---|---|
| Language | Python 3.11+ |
| Screenshot | `mss` + `Pillow` |
| Hotkey listener | `pynput` |
| MCP server | `mcp` Python SDK (FastMCP) |
| LLM | Claude Sonnet 4 (`claude-sonnet-4-20250514`) |
| Observability | Langfuse |
| Session storage | SQLite |

## How to run

```bash
# 1. Clone and enter the project
git clone <repo-url> td-companion
cd td-companion

# 2. Install dependencies
pip install -r requirements.txt

# 3. Set your API keys
cp .env.example .env
# Edit .env with your ANTHROPIC_API_KEY and Langfuse keys

# 4. Launch
python main.py

# 5. Focus your TouchDesigner window, press Ctrl+Shift+T,
#    type your question, and get an answer.
```

> **macOS**: Grant Accessibility permissions to your terminal app when prompted by `pynput`.

## How it works

1. **Hotkey → Screenshot** — Pressing `Ctrl+Shift+T` triggers `mss` to capture your full screen. The PNG bytes are held in memory and sent to Claude as a base64 image.
2. **Claude with function calling** — The image + question + full session history are sent to Claude Sonnet 4. Claude can invoke four MCP-defined tools (`analyze_network`, `suggest_next_node`, `diagnose_problem`, `explain_node`) by extracting what it sees in the screenshot and routing the question to the right analytical frame.
3. **Session persistence + observability** — Every turn (user question + assistant answer) is saved to a local SQLite database and logged as a Langfuse trace with model, token usage, and I/O metadata for debugging and cost tracking.

## Project structure

```
td-companion/
├── main.py               # Entry point, hotkey listener, main loop
├── screenshot.py          # mss screen capture → PNG bytes
├── agent.py               # Claude API with vision + function calling
├── session.py             # SQLite session read/write
├── observability.py       # Langfuse trace logging
├── mcp_server/
│   ├── __init__.py
│   └── tools.py           # 4 FastMCP tools + Anthropic tool schemas
├── .env.example
├── .gitignore
└── requirements.txt
```