# @hallaxius/nvidia-nim-mcp

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![npm version](https://img.shields.io/npm/v/@hallaxius/nvidia-nim-mcp.svg)](https://www.npmjs.com/package/@hallaxius/nvidia-nim-mcp)
[![Bun](https://img.shields.io/badge/runtime-Bun-%23000000.svg)](https://bun.com)
[![Tests](https://img.shields.io/badge/tests-23%20passing-brightgreen.svg)](https://bun.sh/docs/cli/test)
[![code style](https://img.shields.io/badge/code%20style-Biome-60a5fa.svg)](https://biomejs.dev)

**A Model Context Protocol (MCP) server that exposes 140+ [NVIDIA NIM](https://build.nvidia.com/models) models to AI agents via stdio.**

Supports: chat, embeddings, reranking, vision, image generation (FLUX), OCR, and content safety.

## Features

- Thin wrapper around `integrate.api.nvidia.com/v1` and `ai.api.nvidia.com/v1`
- Zero external dependencies (uses global `fetch`)
- Bun-native, Node 20+ compatible
- Offline model capability snapshot for quick lookups
- Automatic fallback heuristics for unknown models
- Streaming disabled by default (safe for stdio)
- TypeScript strict mode with BiomeJS linting
- Test suite with 23+ tests (`bun test`)

## Installation

```bash
bun add @hallaxius/nvidia-nim-mcp
# or
bunx @hallaxius/nvidia-nim-mcp
```

## Quick Setup

The fastest way to get started — configure your API key once and forget it:

```bash
# Configure your key (one-time)
bunx @hallaxius/nvidia-nim-mcp setup "nvapi-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

# Or run interactively (it will prompt for the key)
bunx @hallaxius/nvidia-nim-mcp setup

# Start the MCP server
bunx @hallaxius/nvidia-nim-mcp
```

The key is saved to `~/.config/nvidia-nim-mcp/config.json`. Environment variables still take priority when set.

## Usage

### 1. Set your NVIDIA API Key

You can also configure the key via environment variable:

```bash
export NVIDIA_API_KEY=nvapi-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
```

Supported aliases: `NVIDIA_API_KEY`, `NIM_API_KEY`, `NVAPI_KEY`.

Get your key: [https://build.nvidia.com/explore/discover](https://build.nvidia.com/explore/discover)

### 2. Run the MCP Server

```bash
bunx @hallaxius/nvidia-nim-mcp
```

The server starts over stdio and registers 9 tools.

### 3. Configure Your AI Agent

Integration with:
- [Codex CLI](https://github.com/microsoft/codex-cli)
- [Claude Desktop](https://claude.ai/desktop)
- [OpenCode](https://github.com/anomalyco/opencode)
- [Cline](https://cline.sh)
- [Continue.dev](https://continue.dev)

**OpenCode:** A global `@nvidia` subagent is available at `~/.config/opencode/agents/nvidia.md` for querying NIM models and capabilities.

## Tools

| Tool | Description | Default Model |
|------|-------------|---------------|
| `nim_list_models` | List all available NIM model IDs | — |
| `nim_get_model_capabilities` | Get model type, vision, tools, context | — |
| `nim_chat_completion` | Chat completion with LLM | `z-ai/glm-5.2` |
| `nim_create_embeddings` | Generate text embeddings | `nvidia/nv-embed-v1` |
| `nim_rerank_passages` | Rank passages by query relevance | `nvidia/llama-3.2-nemoretriever-500m-rerank-v2` |
| `nim_vision_inference` | Multi-modal inference (text + images) | `meta/llama-3.2-90b-vision-instruct` |
| `nim_flux_generate_image` | Generate images via FLUX | `black-forest-labs/flux.1-schnell` |
| `nim_ocr_extract` | Extract text from images (OCR) | `nvidia/nemotron-ocr-v1` |
| `nim_safety_classify` | Classify text for safety | `meta/llama-guard-4-12b` |

## Examples

### List Available Models

```typescript
// Agent code / MCP client usage
const models = await callTool("nim_list_models", {});
// Returns: "z-ai/glm-5.2\nmeta/llama-3.3-70b-instruct\n..."
```

### Chat Completion

```typescript
const result = await callTool("nim_chat_completion", {
  messages: [{ role: "user", content: "Explain quantum computing simply" }],
  model: "z-ai/glm-5.2",
  temperature: 0.7,
  max_tokens: 2048,
});
// Returns: { content: [{ type: "text", text: "..." }] }
```

### Vision (VLM)

```typescript
const result = await callTool("nim_vision_inference", {
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "What is in this image?" },
        { type: "image_url", image_url: { url: "data:image/jpeg;base64,/9j/..." } },
      ],
    },
  ],
  model: "meta/llama-3.2-90b-vision-instruct",
});
```

### Embeddings

```typescript
const result = await callTool("nim_create_embeddings", {
  inputs: ["Hello world", "AI is amazing"],
  model: "nvidia/nv-embed-v1",
});
// Returns: { embeddings: [[...], [...]], model: "nvidia/nv-embed-v1", usage: {...} }
```

### Reranking

```typescript
const result = await callTool("nim_rerank_passages", {
  query: "What is the capital of France?",
  passages: ["Paris is the capital of France.", "Berlin is the capital of Germany.", "London is the capital of the UK."],
  model: "nvidia/llama-3.2-nemoretriever-500m-rerank-v2",
});
// Returns: { results: [{ index: 0, score: 0.95 }, ...] }
```

### Image Generation (FLUX)

```typescript
const result = await callTool("nim_flux_generate_image", {
  prompt: "A serene mountain landscape at sunset, digital art",
  width: 1024,
  height: 1024,
  model: "black-forest-labs/flux.1-schnell",
});
// Returns: { content: [{ type: "image", data: "<base64>", mimeType: "image/png" }] }
```

### OCR

```typescript
const result = await callTool("nim_ocr_extract", {
  image_path: "/absolute/path/to/screenshot.png",
  // or: image_base64: "<base64 string>"
  model: "nvidia/nemotron-ocr-v1",
});
// Returns: { text: "Extracted text...", detections: [{ text, confidence, bounding_box }] }
```

### Safety Classification

```typescript
const result = await callTool("nim_safety_classify", {
  text: "This is a harmless text.",
  model: "meta/llama-guard-4-12b",
});
// Returns: { safe: true, categories: {...}, explanation: "..." }
```

## Development

### Setup

```bash
bun install
```

### Build

```bash
bun run build
```

### Type-check

```bash
bun run typecheck
```

### Lint & Format

```bash
bun run lint
```

Uses [BiomeJS](https://biomejs.dev) for linting and formatting. Runs automatically before publish via `prepare`.

### Test

```bash
bun test
```

23+ tests covering capabilities, config, HTTP client, and server structure.

### Dev mode (stdio server)

```bash
bun run dev
```

### Model Capabilities

Capabilities are inferred in real-time using model ID heuristics. No static snapshot needed — the model list is fetched live from the NVIDIA API via `nim_list_models`.

## Supported Models

140+ models across families (sourced from [build.nvidia.com/models](https://build.nvidia.com/models)):

| Category | Families |
|---|---|
| **LLM** | Llama, Nemotron, DeepSeek, Qwen, GLM, Mistral, Mixtral, Gemma, Phi, GPT-OSS, MiniMax, Step, DiffusionGemma, SEED-OSS, Solar, Sarvam M, Stockmark |
| **VLM** | Llama Vision, Nemotron VL, Phi-4 Multimodal, Cosmos, PaliGemma, MiniMax-M3, Ising Calibration |
| **Embeddings** | NV-Embed, BGE-M3, Llama-Nemotron-Embed, ESM-2 (protein) |
| **Rerank** | Llama-Nemotron-Rerank, Rerank-QA-Mistral |
| **Image Gen** | FLUX.1 (dev, schnell), FLUX.2 Klein, Stable Diffusion 3.5, Qwen-Image |
| **OCR & Document** | Nemotron OCR, Nemoretriever Parse/PAGE, PaddleOCR, NV-YOLOX |
| **Safety** | Llama-Guard, Nemotron Safety, Nemoguard, GLiNER PII |
| **Audio & Speech** | Whisper, Parakeet, Canary, Magpie TTS, Chatterbox TTS, Riva Translate, Studio Voice, Nemotron Voicechat |
| **Biology & Science** | AlphaFold2, ESMFold, OpenFold2/3, Boltz-2, MolMIM, DiffDock, ProteinMPNN, RFDiffusion, Evo2, GenMol, VISTA-3D |
| **Physics & Simulation** | CuOpt, FourCastNet, Simcenter STAR-CCM+, Fidelity, Fluent, Spectre-X |
| **Autonomous Driving** | BEVFormer, SparseDrive, StreamPETR |
| **3D & Vision** | TRELLIS, Relighting, EyeContact, Synthetic Video Detector |

> The catalog changes — call `nim_list_models` at runtime for the latest list.

See [https://build.nvidia.com/models](https://build.nvidia.com/models) for the full catalog.

## License

MIT © 2026 @hallaxius