# Pixeltable MCP Server (Developer Edition)

Multimodal AI data infrastructure as an MCP server. **35 tools · 13 resources · 11 prompts** for catalog management, AI/ML pipelines, dependency management, project scaffolding, and an interactive REPL.

Uses **sync endpoints** + **uvloop** for best performance with Pixeltable **≥ 0.6.3** (see `pyproject.toml`). Aligned with the [pixeltable-skill](https://github.com/pixeltable/pixeltable-skill) and [pixeltable-starter-kit](https://github.com/pixeltable/pixeltable-starter-kit) conventions (`pxt.mcp_udfs`, `FastAPIRouter`, `return_rows=True`, `pxt.Required[T]`, `pxt.Array[T]`).

---

## Quick Start

Requires [`uv`](https://docs.astral.sh/uv/getting-started/installation/):

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

**Claude Code (easiest)** — just say:
> "Install https://github.com/pixeltable/mcp-server-pixeltable-developer as a uv tool and add it to your MCPs"

**One-shot (no install):**

```bash
uvx --from git+https://github.com/pixeltable/mcp-server-pixeltable-developer mcp-server-pixeltable-developer
```

**Manual install:**

```bash
uv tool install --from git+https://github.com/pixeltable/mcp-server-pixeltable-developer.git mcp-server-pixeltable-developer
claude mcp add pixeltable mcp-server-pixeltable-developer   # Claude Code
```

If `uv` warns that `~/.local/bin` is not on your `PATH`, run `uv tool update-shell` (or add that directory to `PATH`) so `mcp-server-pixeltable-developer` is found. Check with `mcp-server-pixeltable-developer --version`.

**From source:**

```bash
git clone https://github.com/pixeltable/mcp-server-pixeltable-developer && cd mcp-server-pixeltable-developer
uv sync
```

### Client Configuration

<details><summary><strong>Claude Desktop</strong></summary>

```json
{
  "mcpServers": {
    "pixeltable": {
      "command": "mcp-server-pixeltable-developer",
      "env": {
        "PIXELTABLE_HOME": "/Users/{you}/.pixeltable",
        "PIXELTABLE_FILE_CACHE_SIZE_G": "10"
      }
    }
  }
}
```

From source — use `"command": "uv"` with `"args": ["run", "--directory", "{repo}", "python", "-m", "mcp_server_pixeltable_stio"]`.

</details>

<details><summary><strong>Cursor</strong></summary>

**User config** — `~/.cursor/mcp.json` (applies to all workspaces):

```json
{
  "mcpServers": {
    "pixeltable-developer": {
      "command": "mcp-server-pixeltable-developer",
      "env": {
        "PIXELTABLE_HOME": "/Users/you/.pixeltable"
      }
    }
  }
}
```

If Cursor reports **command not found**, use the full path from `uv tool update-shell` / `which mcp-server-pixeltable-developer`, e.g. `"command": "/Users/you/.local/bin/mcp-server-pixeltable-developer"`.

**Develop this repo from source** — optional project `.cursor/mcp.json` so the server runs from your clone (replace the path):

```json
{
  "mcpServers": {
    "pixeltable-developer": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/mcp-server-pixeltable-developer", "python", "-m", "mcp_server_pixeltable_stio"],
      "env": {
        "PIXELTABLE_HOME": "/Users/you/.pixeltable"
      }
    }
  }
}
```

Do not define the same server name twice (user + project) unless you intend to run two entries.

</details>

Restart your client after configuration changes.

---

## Testing

Use a **dedicated** `PIXELTABLE_HOME` for testing so you do not touch other catalogs.

### MCP Inspector (interactive)

From a clone, after `uv sync`:

```bash
export PIXELTABLE_HOME="$HOME/.pixeltable-mcp-test"
uv run mcp dev src/mcp_server_pixeltable_stio/server.py:mcp
```

This starts the server and opens the **MCP Inspector** in your browser so you can invoke tools, read resources, and try prompts without an IDE.

**Quick checks:** tool `pixeltable_check_dependencies` with expression `openai.chat_completions`; tool `execute_python` with `print(pxt.__version__)`; resource `pixeltable://version`.

### Cursor

Add `PIXELTABLE_HOME` under `env` in `.cursor/mcp.json` (see **Client Configuration** above). Restart Cursor, confirm the server connects, then run a simple tool from the MCP panel.

### CLI sanity (no JSON-RPC)

```bash
mcp-server-pixeltable-developer --version
uv run python list_tools.py
```

`--version` / `--help` exit immediately. `list_tools.py` only prints registered tools, resources, and prompts (import check, not a full MCP session).

---

## Tools (35)

| Category | Tools |
|---|---|
| **Init** | `init` (recovery for circular env init) |
| **Tables** | `create_table` · `drop_table` · `create_view` · `create_snapshot` |
| **Data** | `create_replica` · `query_table` · `insert_data` · `query` · `add_computed_column` |
| **Directories** | `create_dir` · `drop_dir` · `move` |
| **Config** | `configure_logging` · `set_datastore` |
| **AI/ML** | `create_udf` · `create_array` · `create_tools` · `connect_mcp` (wraps `pxt.mcp_udfs`) |
| **Deps** | `check_dependencies` · `install_dependency` |
| **Types** | `create_type` (Image, Video, Audio, Array[Float], `Required[T]`, …) |
| **Docs** | `search_docs` |
| **Scaffolding** | `scaffold_project` · `list_project_templates` (wraps `pixeltable-new`) |
| **REPL** | `execute_python` · `introspect_function` · `list_available_functions` · `install_package` |
| **Logging** | `log_bug` · `log_missing_feature` · `log_success` · `generate_bug_report` · `get_session_summary` |
| **Display** | `display_in_browser` (canvas-extra + `PIXELTABLE_MCP_CANVAS=1`) |

All tools are prefixed `pixeltable_` (except REPL/logging helpers). Full docstrings available via `introspect_function`. `pixeltable://tools` always reflects the live FastMCP registration.

`create_view` accepts `iterator` (`frame_iterator` / `document_splitter` / `audio_splitter` / `string_splitter`) + `iterator_kwargs` so frame and chunk views don't require dropping into `execute_python`.

## Resources (13)

| URI | What it returns |
|---|---|
| `pixeltable://tables` | All tables with count |
| `pixeltable://tables/{path}` | Info about a table / view / snapshot |
| `pixeltable://tables/{path}/schema` | Column schema |
| `pixeltable://directories` | All directories |
| `pixeltable://ls` / `pixeltable://ls/{path}` | Directory listing |
| `pixeltable://version` | Pixeltable version |
| `pixeltable://config/datastore` | Datastore config |
| `pixeltable://types` | Available data types |
| `pixeltable://functions` | Registered Pixeltable functions |
| `pixeltable://tools` | MCP tool list (introspects the live FastMCP server) |
| `pixeltable://help` | Workflow guidance + pitfalls |
| `pixeltable://diagnostics` | System & dependency diagnostics |

## Prompts (11)

Core: `pixeltable_usage_guide` (now leads with pitfalls) · `getting_started` · `computer_vision_pipeline` · `rag_pipeline` · `video_analysis_pipeline` · `audio_processing_pipeline`

Agent / data (from [pixeltable-skill](https://github.com/pixeltable/pixeltable-skill)): `tool_calling_agent_pipeline` · `agent_with_memory_pipeline` · `video_rag_agent_pipeline` · `agentic_patterns_guide` · `ml_data_pipeline`

---

## Examples

```text
Create a table called movies with title, year, and rating columns → insert sample data → query ratings above 8.5

Add a computed column that runs YOLOX object detection on every image

Check what deps I need for openai.chat_completions(...) → install them

execute_python("print(pxt.list_tables())")
```

---

## Documentation

- [Pixeltable docs](https://docs.pixeltable.com/)
- [pixeltable-skill](https://github.com/pixeltable/pixeltable-skill/blob/main/skills/pixeltable-skill/SKILL.md) — task router, [critical anti-patterns](https://github.com/pixeltable/pixeltable-skill/blob/main/skills/pixeltable-skill/references/anti-patterns.md) (`openai.vision`, `FrameIterator`, positional `.similarity`, etc.), and workflow examples aligned with current Pixeltable
- [pixeltable-starter-kit](https://github.com/pixeltable/pixeltable-starter-kit) — `FastAPIRouter` / `pxt serve` patterns and seven app templates; use `pixeltable_scaffold_project` to bootstrap them
- Consumer side of MCP: `pxt.mcp_udfs(url)` (Pixeltable 0.6.x+) pulls this server's tools into a Pixeltable agent; expose them with `pxt.tools(local_udf, *mcp_tools)` and `invoke_tools`

---

## Architecture

```
src/mcp_server_pixeltable_stio/
  server.py            FastMCP server, tool/resource/prompt registration, uvloop, canvas gate
  core/
    tables.py          Table CRUD, views (iterator support), snapshots, replicas, queries,
                       computed columns (eval context exposes all skill providers),
                       _resolve_pxt_type (Required[T] / Array[T] / dict form)
    directories.py     Directory CRUD, listing, moving
    dependencies.py    Dependency checking, unified installer, diagnostics
    udf.py             UDF creation, type system, pxt.tools / pxt.mcp_udfs wrappers
    helpers.py         Config, version, docs search, live pixeltable://tools introspection
    resources.py       Read-only MCP resource handlers
    scaffold.py        pixeltable-new wrappers (module first, uvx fallback)
    canvas_server.py   Optional browser canvas (lazy-imported; opt-in via env)
  prompt.py            Prompt templates (11 prompts covering core flows + 5 skill agent flows)
  repl_functions.py    Persistent Python REPL, introspection, package management
tests/                 Fast pytest suite + slow smoke (--run-slow uses tmp PIXELTABLE_HOME)
.github/workflows/     CI: matrix py3.10/3.11/3.12, list_tools.py + pytest
```

## Optional canvas display

The `display_in_browser` tool requires:

1. Install the canvas extra: `uv pip install 'mcp-server-pixeltable-developer[canvas]'` (adds FastAPI + uvicorn).
2. Start the MCP server with `PIXELTABLE_MCP_CANVAS=1` (override the port with `PIXELTABLE_MCP_CANVAS_PORT`).

Without those, the tool returns a clear error and core MCP traffic is unaffected.

## Project scaffolding

`pixeltable_list_project_templates` and `pixeltable_scaffold_project` wrap [`pixeltable-new`](https://github.com/pixeltable/pixeltable-new). The MCP imports `pixeltable_new.new` if installed, otherwise shells out to `uvx pixeltable-new --json`. Patterns: `serving` / `backend` / `batch`. Templates include `multimodal-rag`, `agent`, `video-intel`, `audio-intel`, `data-lab`, `content-pipeline`, `full-stack-showcase`.

---

## Troubleshooting

- **Restart your client** after any config change
- **Python 3.10+** and **`uv`** are required
- **`command not found` after `uv tool install`:** ensure `~/.local/bin` is on `PATH` (`uv tool update-shell`) or invoke via full path; confirm with `mcp-server-pixeltable-developer --version`
- Check that `PIXELTABLE_HOME` points to a valid directory
- **Circular env initialization detected:** call the `pixeltable_init` tool to clear the stale state, or re-create your `PIXELTABLE_HOME`
- **Canvas error from `display_in_browser`:** install the `canvas` extra and set `PIXELTABLE_MCP_CANVAS=1` (see above)
- Use `log_bug(...)` / `generate_bug_report()` for structured issue tracking
- File issues at [github.com/pixeltable/mcp-server-pixeltable-developer](https://github.com/pixeltable/mcp-server-pixeltable-developer/issues)