Allows using Google Gemini models as part of multi-model reasoning, debate, and red-teaming workflows.
Allows running local models via Ollama for on-premise inference in multi-model setups.
Allows using OpenAI models (e.g., GPT-4) for sub-agent style reasoning and parallel comparisons.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@multi-model-mcp Explain the CAP theorem using gpt, claude, and gemini."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An MCP server that exposes tools for sub-agent style reasoning across multiple LLM providers. From Claude Code (or any MCP client), you can delegate prompts to OpenAI, Anthropic, Gemini, Groq, Ollama, OpenRouter, and any LiteLLM-supported provider — then run critique loops, debates, red-teaming, and answer ranking without leaving your conversation.
Tool | Description |
| Send a prompt to one configured model |
| Send the same prompt to multiple models in parallel |
| Multi-step reasoning: independent → critique, debate, or red-team |
| Ask models to critique a draft answer |
| Have a judge model rank candidate answers |
| List all configured model aliases |
reason_together strategiesindependent_then_critique (default): All models answer independently → critic synthesizes
debate: Models see each other's answers and refine over N rounds → critic synthesizes
red_team: Proposer answers → red teamers attack → proposer revises (N rounds) → critic finalizes
Related MCP server: delegations-mcp
Requires Python ≥ 3.11 and uv.
git clone https://github.com/YOUR_USERNAME/multi-model-mcp
cd multi-model-mcp
uv syncCopy and edit models.yaml — it ships with common models pre-configured. Each entry is a model alias pointing to a LiteLLM model string:
models:
gpt:
litellm_model: gpt-4.1
api_key_env: OPENAI_API_KEY
claude:
litellm_model: claude-sonnet-4-5
api_key_env: ANTHROPIC_API_KEY
local:
litellm_model: ollama/qwen3:latest
api_base: http://localhost:11434 # no key neededAdd any provider LiteLLM supports: Groq (groq/llama-3.3-70b-versatile), Mistral, Together AI, DeepSeek, OpenRouter (openrouter/...), etc.
cp .env.example .env
# edit .env with your keysOnly keys for providers you actually use are required.
Add to your project's .mcp.json (or ~/.claude.json for global):
{
"mcpServers": {
"multi-model": {
"command": "uv",
"args": [
"run",
"--project", "/path/to/multi-model-mcp",
"multi-model-mcp"
],
"env": {
"OPENAI_API_KEY": "sk-...",
"ANTHROPIC_API_KEY": "sk-ant-...",
"GEMINI_API_KEY": "...",
"MODELS_CONFIG_PATH": "/path/to/multi-model-mcp/models.yaml"
}
}
}
}Or if you install it:
uv tool install .Then use "command": "multi-model-mcp" without args.
# Simple query
Use ask_model with alias "gpt" to explain backpressure in streaming systems.
# Parallel comparison
Use ask_many with aliases ["gpt", "claude", "gemini"] to explain the CAP theorem.
Compare their answers.
# Multi-model reasoning
Use reason_together with task "Should we use event sourcing for this service?"
model_aliases ["gpt", "gemini"], critic_model_alias "claude", strategy "independent_then_critique"
# Debate
Use reason_together with task "Is GraphQL worth the complexity over REST?"
model_aliases ["gpt", "claude"], critic_model_alias "gemini", strategy "debate", rounds 2
# Red-team a decision
Use reason_together with task "Our plan is to use a single Postgres instance for all tenants"
model_aliases ["gpt", "gemini", "groq"], strategy "red_team", rounds 2
# Critique a draft
Use critique_answer with question "What is eventual consistency?"
draft_answer "It means data will eventually be the same across nodes."
model_aliases ["claude", "gpt"]
# Pick the best
Use pick_best_answer with question "What is the best way to handle auth tokens?"
candidate_answers ["Store in localStorage", "Store in httpOnly cookies", "Store in memory only"]
judge_model_alias "claude"models.yaml fieldsField | Required | Description |
| Yes | LiteLLM model string (e.g. |
| No | Human-readable label |
| No | Env var name holding the API key |
| No | Override base URL (needed for Ollama, proxies) |
| No | Per-call timeout in seconds (default: 60) |
| No | Retry attempts on rate limit / timeout (default: 2) |
Provider | Example model string |
OpenAI |
|
Anthropic |
|
Google Gemini |
|
Groq |
|
Ollama |
|
OpenRouter |
|
Mistral |
|
DeepSeek |
|
Together AI |
|
See LiteLLM providers docs for the full list.
No key leakage: API keys are never logged; errors are sanitized before returning.
Failure isolation: one model failing in ask_many / reason_together does not crash the call.
Synthesis ≠ truth: reason_together presents the critic's output as a synthesized answer, not ground truth.
No hidden reasoning exposed: traces summarize what happened (which model, which step) without exposing chain-of-thought internals.
Easy to extend: add any LiteLLM-supported model in models.yaml with no code changes.
Send the same prompt to multiple models in parallel. Returns one response per model. Individual failures are isolated — one model failing does not prevent others from responding.
Inputs: prompt (required), model_aliases (required), system_prompt, temperature, +1 more. Returns: object
Send a prompt to a single configured model and return its response. Use this for a direct query to a specific provider/model.
Inputs: model_alias (required), prompt (required), system_prompt, temperature, +1 more. Returns: object
Ask one or more models to critique a draft answer to a question. Returns per-model critiques with identified weaknesses and suggested improvements. Useful for improving a draft before finalizing it.
Inputs: question (required), draft_answer (required), model_aliases (required), temperature, +1 more. Returns: object
List all model aliases available in the current models.yaml configuration.
Inputs: no named parameters. Returns: object
Given multiple candidate answers to a question, ask a judge model to rank them and identify the best one. Returns winner, ranking, and explanation.
Inputs: question (required), candidate_answers (required), judge_model_alias, rubric, +2 more. Returns: object
Multi-step reasoning workflow across multiple models. Strategies: • independent_then_critique (default): models answer independently, then a critic synthesizes. • debate: models see each other's answers and refine over N rounds, then a critic synthesizes. • red_team: proposer answers, others attack it, proposer revises — repeated for N rounds. Returns a trace, individual responses, and a final synthesized answer. The final answer is presented as synthesis, not ground truth.
Inputs: task (required), model_aliases (required), critic_model_alias, rounds, +4 more. Returns: object
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Scored across 6 tools
Each tool has a distinct purpose: ask_many for parallel queries, ask_model for single query, list_models for enumeration, critique_answer for critiquing drafts, pick_best_answer for ranking candidates, and reason_together for multi-step reasoning. No two tools overlap in function.
All tool names follow a consistent verb_noun pattern: ask_many, ask_model, critique_answer, list_models, pick_best_answer, reason_together. The verbs are descriptive and the naming style is uniform.
With 6 tools, the surface is well-scoped for a multi-model interaction server. Each tool adds clear value without redundancy or bloat, covering essential operations for querying, critiquing, ranking, and reasoning.
The tool set covers all core workflows: single/parallel queries, critique, ranking, and multi-step reasoning. A minor gap might be a tool for managing model configurations or conversation history, but the set is largely complete for the stated purpose.
MCP server for building and testing AI agents with multi-model experimentation and insights.
AI Reasoning Cache & Consensus Layer with 11 MCP tools via Streamable HTTP.
MCP server for your apps' tools and custom tools, plus hosted AI agents and approval-gated workflows
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
