# Kaggle-MCP-Agent

> **v1.0** — Drive Kaggle notebooks end-to-end from an AI agent (or your CLI)
> through a **local stdio MCP (Model Context Protocol) server**. No Kaggle
> OAuth dance required — just a `KGAT_...` API token from your Kaggle settings.

```
open  ->  edit  ->  save (v2)  ->  run  ->  monitor  ->  fetch logs  ->  summarise
                                          + attach datasets, toggle GPU T4, internet
```

**v1.0** adds first-class **dataset attachment**, **GPU (T4 / P100) via
`machine_shape`**, and **internet toggle**, plus a defensive mount-wait cell
helper for large dataset mounts. All verified live against a private notebook
(`shubhojitnaskar/mcpnotebook`, pushed through v1 -> v32). Multi-agent
orchestration (a separate "Kaggle Grandmaster Agent" repo) comes later.

---

## Why this exists

Kaggle publishes an MCP server (`kaggle-mcp-server`), but the **official
remote** endpoint (`https://www.kaggle.com/mcp`) requires OAuth 2.0 for any
tool execution — `initialize` / `tools/list` work with a static token, but
`tools/call` returns `isError: true`. The OAuth flow is awkward to set up
locally (no stable Python OAuth client, browser-redirect setup breaks on
Windows, etc.).

This repo uses the **community `kaggle-mcp-server`** package as a **local stdio
process** instead. The same static `KGAT_...` token you generate at
`https://www.kaggle.com/settings` works there because the server talks to
Kaggle's private REST API on your behalf — no OAuth needed. This is free,
scriptable, and works on Windows / macOS / Linux.

### What v1.0 can do today

- ✅ Read a private notebook's source (`kernel_pull`)
- ✅ Edit code cells (replace / insert / text-replace) and serialise back to `.ipynb` JSON
- ✅ **Update an existing notebook** as a new version (`v1` -> `v2` -> ...) — via a
  small `kagglesdk`-direct helper that works around the community MCP server's
  `kernel_push` slug bug (see below)
- ✅ **Attach one or more datasets** (`dataset_data_sources=["owner/dataset-slug",
  ...]`) — multiple datasets in one push, verified up to a 12GB ASR parquet set
- ✅ **Toggle GPU** via `machine_shape` (`NvidiaTeslaT4` -> Tesla T4;
  `NvidiaTeslaP100` -> normalised to generic `Gpu`, scheduler picks the device);
  CPU when empty. `enable_gpu` is deprecated, kept for compatibility.
- ✅ **Toggle internet** (`enable_internet=True|False`) — verified both directions
  with a live `urllib` reach probe
- ✅ `mount_wait_cell_source()` — a defensive first cell that polls
  `os.path.exists()` so large dataset mounts don't race the kernel start
- ✅ Poll the run to `COMPLETE` / `ERROR` (`kernel_session_status`)
- ✅ Fetch run output files (`kernel_session_output`)
- ✅ Run end-to-end from the CLI (`python -m kaggle_mcp_agent.run full ...`) OR
  natively from an **opencode** AI agent session (all `kaggle_*` tools auto-loaded)

### Known limitations (v1.0)

- **The community MCP `kernel_push` tool does not send the `slug` field** when
  saving a notebook. As a result, calling it to **update** an existing
  notebook returns `409 Conflict` (Kaggle thinks you're creating a new notebook
  with a duplicate title). `Kaggle-MCP-Agent` ships a `kagglesdk`-direct
  `save_notebook_version()` helper that sets `req.slug`, cleanly creating a new
  version instead. This is a real upstream bug worth fixing in
  `kaggle-mcp-server` (still present in 0.2.0 as of this release).
- **Dataset version pinning is NOT supported by the Save API.**
  `dataset_data_sources` accepts only the bare slug `owner/dataset-slug`; any
  version suffix (`/v2`, `/versions/2`, `@2`, `/2`) is silently stripped and the
  notebook always mounts the **LATEST** version. A manual UI pin of a specific
  version does **not** survive a Save API re-push that re-specifies the dataset
  (and omitting the dataset field DETACHES it, so you must re-specify on every
  save). Only `model_data_sources` supports a `/{version-number}` suffix.
  Workaround: keep a single version of the dataset you depend on, or copy the
  pinned version into a private "snapshot" dataset.
- **GPU `machine_shape` is partly advisory.** Only `NvidiaTeslaT4` round-trips
  exactly (`machineShape` -> `NvidiaTeslaT4`, runs on a Tesla T4). The other
  GPU values (`NvidiaTeslaP100`, and bare strings like `gpu_t4x2`) are
  normalised by Kaggle to the generic `Gpu`, and the physical device (T4 vs
  P100) is chosen by the scheduler. If you need a guaranteed T4, use
  `MACHINE_SHAPES.GPU_T4`.
- **Cell outputs / error tracebacks are not retrievable via API** for batch
  runs. `kernel_session_output` returns an empty ZIP for print-only notebooks
  (print output goes to cell outputs, not the output-file ZIP). To see an
  error traceback you must open the version on the Kaggle website, OR write
  your diagnostics to `/kaggle/working/<file>` and download that file.
- **No Kaggle Secrets or pip Dependency Manager via the Save API.**
  `ApiSaveKernelRequest` has no field for either. Secrets are a Notebook
  Editor UI feature (Add-ons -> Secrets); the pip Dependency Manager is an
  editor + Package-export feature that writes a `.sh` script run pre-Save.
  Workarounds: set secrets as env vars in code, or run `!pip install <pkg>` as
  the first cell (requires internet ON, unless the package is cached on the
  Docker image).
- Only notebook (`ipynb`) kernels are exercised end-to-end in v1.0; script
  kernels parse correctly but the save path is untested.
- No multi-agent orchestration, memory, or experiment tracking — by design.

---

## Quick start

### 1. Install

```bash
# Clone this repo:
git clone https://github.com/Shubhojit2000/Kaggle-MCP-Agent.git
cd Kaggle-MCP-Agent

# Install runtime + dev deps (Python >= 3.10):
pip install -r requirements.txt
pip install -r requirements-dev.txt   # tests only

# Install the community kaggle-mcp-server + kagglesdk (one time):
pip install --user kaggle-mcp-server kagglesdk "mcp==1.29.0"
```

### 2. Get a Kaggle API token

1. Open `https://www.kaggle.com/settings`.
2. Scroll to **API** -> **Generate New Token**. It begins with `KGAT`.
3. Save it locally — never commit it.

### 3. Configure

Copy `.env.example` to `.env` and fill in your token:

```bash
cp .env.example .env
# edit .env:
#   KAGGLE_API_TOKEN=KGAT_your_real_token_here
#   KAGGLE_USERNAME=your_kaggle_username
```

If `kaggle-mcp-server` is NOT on your PATH (or you want to point at a specific
binary), also set in `.env`:

```
KAGGLE_MCP_SERVER_EXE=/full/path/to/kaggle-mcp-server   # or .exe on Windows
```

### 4. Use it

#### Option A — From the CLI

```bash
# Read a notebook's source:
python -m kaggle_mcp_agent.run pull shubhojitnaskar/mcpnotebook

# List its code cells:
python -m kaggle_mcp_agent.run list-cells shubhojitnaskar/mcpnotebook

# Get the latest run status:
python -m kaggle_mcp_agent.run status shubhojitnaskar/mcpnotebook

# Full workflow: read -> apply edits -> save as v2 -> monitor -> fetch logs:
python -m kaggle_mcp_agent.run full shubhojitnaskar/mcpnotebook --edits examples/edits.json

# v1.0: full workflow WITH a dataset attached, GPU T4, and internet ON:
python -m kaggle_mcp_agent.run full shubhojitnaskar/mcpnotebook \
    --dataset shubhojitnaskar/googlewaxal \
    --dataset shubhojitnaskar/waxalasr \
    --gpu NvidiaTeslaT4 --internet

# v1.0: programmatic equivalent (see examples/attach_dataset_gpu.py):
#   from kaggle_mcp_agent import save_notebook_version, MACHINE_SHAPES, mount_wait_cell_source
#   save_notebook_version(
#       slug="shubhojitnaskar/mcpnotebook", title="MCPNOTEBOOK", text=ipynb_json,
#       dataset_data_sources=["shubhojitnaskar/googlewaxal", "shubhojitnaskar/waxalasr"],
#       enable_internet=True, machine_shape=MACHINE_SHAPES.GPU_T4,
#   )
# Defensive first cell for a 12GB mount that races the kernel start:
#   cell0 = mount_wait_cell_source(
#       "/kaggle/input/datasets/owner/waxalasr/.../lug-train-00000.parquet",
#       seconds=300, poll=5, label="asr",
#   )
```

#### Edits JSON format

```json
[
  {"op": "replace_code_cell", "index": 0, "source": "print('edited')"},
  {"op": "insert_code_cell",  "index": 1, "source": "print(2)"}
]
```

Operations:
- `replace_code_cell` — replace the source of code cell at `index`
- `insert_code_cell`   — insert a new code cell BEFORE `index`

#### Option B — From an opencode AI agent

With `opencode.json` (see `examples/opencode.example.json`) in your project
directory, all `kaggle_*` tools become natively callable from your opencode
prompts. After putting `opencode.json` in place and starting opencode:

> "Read `shubhojitnaskar/mcpnotebook`, add `print(1)` below the existing HF
> token line in the first code cell and a new `print(2)` cell after it, save
> as v2, monitor the run, and fetch the logs."

OpenCode will itself call `kaggle_kernel_pull`, build the new `.ipynb` JSON,
push (using the v0.1 slug-bug workaround), poll `kaggle_kernel_session_status`,
and `kaggle_kernel_session_output` for you.

> ⚠️ Within opencode, the `kaggle_kernel_push` tool only **creates** notebooks
> (it can't push v2 of an existing one due to the slug bug). For updating an
> existing notebook from an agent prompt, ask it to run the v0.1 helper:
> ```python
> from kaggle_mcp_agent.push import save_notebook_version
> save_notebook_version(slug="owner/slug", title="...", text=ipynb_json, ...)
> ```

---

## Project layout

```
Kaggle-MCP-Agent/
├── src/kaggle_mcp_agent/
│   ├── __init__.py         # public API exports
│   ├── config.py           # env-driven Settings + token resolution
│   ├── local_mcp.py        # minimal stdio MCP JSON-RPC client (cross-platform)
│   ├── tools.py            # per-tool wrappers over the local kaggle-mcp-server
│   ├── push.py             # kagglesdk-direct save_notebook_version (slug workaround)
│   ├── notebook_utils.py   # parse / edit / re-serialise .ipynb + script sources
│   └── run.py              # CLI: python -m kaggle_mcp_agent.run ...
├── tests/
│   ├── conftest.py
│   ├── test_config.py       # token resolution priority (offline)
│   ├── test_local_mcp.py    # MCP isError handling (offline, faked stdio)
│   ├── test_notebook_utils.py  # .ipynb parse/edit/round-trip (offline)
│   └── test_push.py         # save_notebook_version sets slug (offline, faked kagglesdk)
├── examples/
│   ├── edits.json          # sample edits file
│   ├── attach_dataset_gpu.py  # v1.0: dataset + GPU T4 + internet push recipe
│   └── opencode.example.json  # opencode.json template (no secrets)
├── docs/
│   └── AUTH.md             # full auth history (what works, what doesn't)
├── .env.example
├── .gitignore
├── pyproject.toml
├── requirements.txt
├── requirements-dev.txt
├── LICENSE                 # MIT
└── README.md
```

---

## Tests

```bash
# Offline tests run anywhere, no Kaggle token needed:
python -m pytest tests/ -v
```

All tests are offline and use fakes / mocks — there is no live Kaggle network
access in CI. To exercise the live flow against your own Kaggle account, see
the `examples/` snippets and `docs/AUTH.md`.

---

## Security

- **Never commit your `KGAT_...` token.** `.env` is in `.gitignore`. The
  committed `opencode.example.json` / `.env.example` deliberately contain
  placeholders only.
- If you leak a token (paste in chat, push to a public repo), regenerate at
  `https://www.kaggle.com/settings` -> API -> **Expire** / Generate New Token,
  and update `.env`.
- Treat any Kaggle token like a password — it carries your account's Kaggle
  permissions (read private notebooks, push new versions, submit to
  competitions, etc.).

---

## Contributing

Bug reports and PRs welcome at `https://github.com/Shubhojit2000/Kaggle-MCP-Agent/issues`.
For the most impactful contribution, the project would benefit from:

1. **Upstream-fix the `kernel_push` slug bug** in the community
   `kaggle-mcp-server` so the MCP tool itself supports pushing new versions of
   existing notebooks (still present in 0.2.0; then we can retire
   `push.save_notebook_version`).
2. **Find a Save-API path for dataset version pinning** (currently impossible:
   bare slugs only, always floats to latest). If a kagglesdk upgrade exposes a
   versioned dataset ref, plumb it through `dataset_data_sources`.
3. **First-class `script` kernel support** in the full workflow (currently
   untested end-to-end).
4. **Output/traceback retrieval** for batch runs (write diagnostics to
   `/kaggle/working/` and add a helper to parse them from the output ZIP).

Run tests before submitting:

```bash
python -m pytest tests/ -v
```

---

## License

MIT — see [LICENSE](LICENSE).

---

## Acknowledgements

- The community [`kaggle-mcp-server`](https://pypi.org/project/kaggle-mcp-server/)
  package, which provides the MCP server we drive.
- [opencode](https://opencode.ai/) for being a great agent host that natively
  supports stdio MCP servers.