# multi-model-mcp

An MCP server that exposes tools for **sub-agent style reasoning across multiple LLM providers**. From Claude Code (or any MCP client), you can delegate prompts to OpenAI, Anthropic, Gemini, Groq, Ollama, OpenRouter, and any LiteLLM-supported provider — then run critique loops, debates, red-teaming, and answer ranking without leaving your conversation.

## Tools

| Tool | Description |
|---|---|
| `ask_model` | Send a prompt to one configured model |
| `ask_many` | Send the same prompt to multiple models in parallel |
| `reason_together` | Multi-step reasoning: independent → critique, debate, or red-team |
| `critique_answer` | Ask models to critique a draft answer |
| `pick_best_answer` | Have a judge model rank candidate answers |
| `list_models` | List all configured model aliases |

### `reason_together` strategies

- **`independent_then_critique`** (default): All models answer independently → critic synthesizes
- **`debate`**: Models see each other's answers and refine over N rounds → critic synthesizes
- **`red_team`**: Proposer answers → red teamers attack → proposer revises (N rounds) → critic finalizes

## Setup

### 1. Install

Requires Python ≥ 3.11 and [uv](https://docs.astral.sh/uv/).

```bash
git clone https://github.com/YOUR_USERNAME/multi-model-mcp
cd multi-model-mcp
uv sync
```

### 2. Configure models

Copy and edit `models.yaml` — it ships with common models pre-configured. Each entry is a **model alias** pointing to a LiteLLM model string:

```yaml
models:
  gpt:
    litellm_model: gpt-4.1
    api_key_env: OPENAI_API_KEY

  claude:
    litellm_model: claude-sonnet-4-5
    api_key_env: ANTHROPIC_API_KEY

  local:
    litellm_model: ollama/qwen3:latest
    api_base: http://localhost:11434   # no key needed
```

Add any provider LiteLLM supports: Groq (`groq/llama-3.3-70b-versatile`), Mistral, Together AI, DeepSeek, OpenRouter (`openrouter/...`), etc.

### 3. Set API keys

```bash
cp .env.example .env
# edit .env with your keys
```

Only keys for providers you actually use are required.

### 4. Register with Claude Code

Add to your project's `.mcp.json` (or `~/.claude.json` for global):

```json
{
  "mcpServers": {
    "multi-model": {
      "command": "uv",
      "args": [
        "run",
        "--project", "/path/to/multi-model-mcp",
        "multi-model-mcp"
      ],
      "env": {
        "OPENAI_API_KEY": "sk-...",
        "ANTHROPIC_API_KEY": "sk-ant-...",
        "GEMINI_API_KEY": "...",
        "MODELS_CONFIG_PATH": "/path/to/multi-model-mcp/models.yaml"
      }
    }
  }
}
```

Or if you install it:
```bash
uv tool install .
```
Then use `"command": "multi-model-mcp"` without `args`.

## Example Claude Code usage

```
# Simple query
Use ask_model with alias "gpt" to explain backpressure in streaming systems.

# Parallel comparison
Use ask_many with aliases ["gpt", "claude", "gemini"] to explain the CAP theorem.
Compare their answers.

# Multi-model reasoning
Use reason_together with task "Should we use event sourcing for this service?"
model_aliases ["gpt", "gemini"], critic_model_alias "claude", strategy "independent_then_critique"

# Debate
Use reason_together with task "Is GraphQL worth the complexity over REST?"
model_aliases ["gpt", "claude"], critic_model_alias "gemini", strategy "debate", rounds 2

# Red-team a decision
Use reason_together with task "Our plan is to use a single Postgres instance for all tenants"
model_aliases ["gpt", "gemini", "groq"], strategy "red_team", rounds 2

# Critique a draft
Use critique_answer with question "What is eventual consistency?"
draft_answer "It means data will eventually be the same across nodes."
model_aliases ["claude", "gpt"]

# Pick the best
Use pick_best_answer with question "What is the best way to handle auth tokens?"
candidate_answers ["Store in localStorage", "Store in httpOnly cookies", "Store in memory only"]
judge_model_alias "claude"
```

## Configuration reference

### `models.yaml` fields

| Field | Required | Description |
|---|---|---|
| `litellm_model` | Yes | LiteLLM model string (e.g. `gpt-4.1`, `gemini/gemini-2.5-pro`, `ollama/qwen3:latest`) |
| `description` | No | Human-readable label |
| `api_key_env` | No | Env var name holding the API key |
| `api_base` | No | Override base URL (needed for Ollama, proxies) |
| `timeout` | No | Per-call timeout in seconds (default: 60) |
| `max_retries` | No | Retry attempts on rate limit / timeout (default: 2) |

### LiteLLM model strings by provider

| Provider | Example model string |
|---|---|
| OpenAI | `gpt-4.1`, `gpt-4o`, `o4-mini` |
| Anthropic | `claude-sonnet-4-5`, `claude-opus-4-8` |
| Google Gemini | `gemini/gemini-2.5-pro`, `gemini/gemini-2.5-flash` |
| Groq | `groq/llama-3.3-70b-versatile` |
| Ollama | `ollama/qwen3:latest`, `ollama/llama3.3` |
| OpenRouter | `openrouter/anthropic/claude-sonnet-4-5` |
| Mistral | `mistral/mistral-large-latest` |
| DeepSeek | `deepseek/deepseek-chat` |
| Together AI | `together_ai/meta-llama/Llama-3-70b-chat-hf` |

See [LiteLLM providers docs](https://docs.litellm.ai/docs/providers) for the full list.

## Design notes

- **No key leakage**: API keys are never logged; errors are sanitized before returning.
- **Failure isolation**: one model failing in `ask_many` / `reason_together` does not crash the call.
- **Synthesis ≠ truth**: `reason_together` presents the critic's output as a synthesized answer, not ground truth.
- **No hidden reasoning exposed**: traces summarize what happened (which model, which step) without exposing chain-of-thought internals.
- **Easy to extend**: add any LiteLLM-supported model in `models.yaml` with no code changes.