Onde Inference MCP Server

Manage your Onde Inference account, fine-tune local models, and export them to GGUF, all from the terminal.

Install

Install onde-cli with your favorite tool. For package docs and the full install matrix, see https://ondeinference.com/cli.

npm

npm install -g @ondeinference/cli

Homebrew

brew tap ondeinference/homebrew-tap && brew trust --tap ondeinference/homebrew-tap
brew install onde

pip / uv / uvx

pip install onde-cli
# or
uv tool install onde-cli
uv run onde
# or with
uvx --from onde-cli onde

.NET tool

dotnet tool install --global Onde.Cli

Dart pub global

dart pub global activate onde_cli

The Dart package is a thin launcher. On first run it downloads the right native binary into ~/.onde/cli, then reuses the local copy.

Pre-built binary

Download a release from GitHub Releases:

# macOS Apple Silicon
curl -Lo onde https://github.com/ondeinference/onde-cli/releases/latest/download/onde-macos-arm64
chmod +x onde && mv onde /usr/local/bin/onde

Related MCP server: AgentVault MCP Server

Usage

onde

This opens the TUI. You can sign up or sign in right there.

MCP server

Run onde as a Model Context Protocol server over stdio instead of the TUI:

onde --mcp

This exposes Onde account and model-catalog operations as MCP tools — login, me, apps_list, app_create, app_rename, models_list, model_register, model_assign, hf_search — returning structured JSON. stdout is the JSON-RPC channel; tools run non-interactively and reuse the token from a TUI sign-in (or the login tool). Point any MCP client at the command onde --mcp.

Fine-tuning

onde includes a LoRA fine-tuning pipeline for Qwen2, Qwen2.5, and Qwen3 models. It runs locally: Metal on Apple Silicon, CPU elsewhere. No cloud setup. No Python environment.

The flow is straightforward: download a safetensors base model, fine-tune it with LoRA, merge the adapter back into the base weights, then export to GGUF for use in the Onde SDK.

If you want a quick refresher on what the model is actually doing at inference time, Onde has a short note on the forward pass.

Training data format

Each line should be one complete conversation in Qwen's chat template:

{"text": "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nWhat is LoRA?<|im_end|>\n<|im_start|>assistant\nLoRA adds small trainable matrices to frozen layers, letting you fine-tune large models without updating all the weights.<|im_end|>"}

Save the file wherever you want. The TUI lets you point to it directly.

Running it

onde
  → Models tab (Tab from Apps)
  → Select a safetensors model (↑↓, Enter)
  → Press f

Only safetensors models can be fine-tuned. GGUF models are already quantized, so their weights are not differentiable.

Configure the run:

Press Enter to start. In a healthy run, loss usually starts dropping by epoch 2. If it stays flat, try 0.0003.

After training

For rank 8 on a 0.6B model, the adapter is about 1.5 MB. From the fine-tune complete screen:

• m to merge the adapter into the base model

• g to export the merged model to GGUF

The resulting GGUF loads directly in the Onde SDK for on-device AI inference.

Supported base models

You can search for any of these from the Models tab with /.

Debug

Logs are written to ~/.cache/onde/debug.log.

If you installed through pub.dev, the launcher cache lives under ~/.onde/cli.

License

Dual-licensed under MIT and Apache 2.0.

Copyright

© 2026 Splitfire AB (Onde Inference).