Allows using Google Gemini models as part of multi-model reasoning, debate, and red-teaming workflows.
Allows running local models via Ollama for on-premise inference in multi-model setups.
Allows using OpenAI models (e.g., GPT-4) for sub-agent style reasoning and parallel comparisons.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@multi-model-mcp Explain the CAP theorem using gpt, claude, and gemini."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An MCP server that exposes tools for sub-agent style reasoning across multiple LLM providers. From Claude Code (or any MCP client), you can delegate prompts to OpenAI, Anthropic, Gemini, Groq, Ollama, OpenRouter, and any LiteLLM-supported provider — then run critique loops, debates, red-teaming, and answer ranking without leaving your conversation.
Tool | Description |
| Send a prompt to one configured model |
| Send the same prompt to multiple models in parallel |
| Multi-step reasoning: independent → critique, debate, or red-team |
| Ask models to critique a draft answer |
| Have a judge model rank candidate answers |
| List all configured model aliases |
reason_together strategiesindependent_then_critique (default): All models answer independently → critic synthesizes
debate: Models see each other's answers and refine over N rounds → critic synthesizes
red_team: Proposer answers → red teamers attack → proposer revises (N rounds) → critic finalizes
Related MCP server: delegations-mcp
Requires Python ≥ 3.11 and uv.
git clone https://github.com/YOUR_USERNAME/multi-model-mcp
cd multi-model-mcp
uv syncCopy and edit models.yaml — it ships with common models pre-configured. Each entry is a model alias pointing to a LiteLLM model string:
models:
gpt:
litellm_model: gpt-4.1
api_key_env: OPENAI_API_KEY
claude:
litellm_model: claude-sonnet-4-5
api_key_env: ANTHROPIC_API_KEY
local:
litellm_model: ollama/qwen3:latest
api_base: http://localhost:11434 # no key neededAdd any provider LiteLLM supports: Groq (groq/llama-3.3-70b-versatile), Mistral, Together AI, DeepSeek, OpenRouter (openrouter/...), etc.
cp .env.example .env
# edit .env with your keysOnly keys for providers you actually use are required.
Add to your project's .mcp.json (or ~/.claude.json for global):
{
"mcpServers": {
"multi-model": {
"command": "uv",
"args": [
"run",
"--project", "/path/to/multi-model-mcp",
"multi-model-mcp"
],
"env": {
"OPENAI_API_KEY": "sk-...",
"ANTHROPIC_API_KEY": "sk-ant-...",
"GEMINI_API_KEY": "...",
"MODELS_CONFIG_PATH": "/path/to/multi-model-mcp/models.yaml"
}
}
}
}Or if you install it:
uv tool install .Then use "command": "multi-model-mcp" without args.
# Simple query
Use ask_model with alias "gpt" to explain backpressure in streaming systems.
# Parallel comparison
Use ask_many with aliases ["gpt", "claude", "gemini"] to explain the CAP theorem.
Compare their answers.
# Multi-model reasoning
Use reason_together with task "Should we use event sourcing for this service?"
model_aliases ["gpt", "gemini"], critic_model_alias "claude", strategy "independent_then_critique"
# Debate
Use reason_together with task "Is GraphQL worth the complexity over REST?"
model_aliases ["gpt", "claude"], critic_model_alias "gemini", strategy "debate", rounds 2
# Red-team a decision
Use reason_together with task "Our plan is to use a single Postgres instance for all tenants"
model_aliases ["gpt", "gemini", "groq"], strategy "red_team", rounds 2
# Critique a draft
Use critique_answer with question "What is eventual consistency?"
draft_answer "It means data will eventually be the same across nodes."
model_aliases ["claude", "gpt"]
# Pick the best
Use pick_best_answer with question "What is the best way to handle auth tokens?"
candidate_answers ["Store in localStorage", "Store in httpOnly cookies", "Store in memory only"]
judge_model_alias "claude"models.yaml fieldsField | Required | Description |
| Yes | LiteLLM model string (e.g. |
| No | Human-readable label |
| No | Env var name holding the API key |
| No | Override base URL (needed for Ollama, proxies) |
| No | Per-call timeout in seconds (default: 60) |
| No | Retry attempts on rate limit / timeout (default: 2) |
Provider | Example model string |
OpenAI |
|
Anthropic |
|
Google Gemini |
|
Groq |
|
Ollama |
|
OpenRouter |
|
Mistral |
|
DeepSeek |
|
Together AI |
|
See LiteLLM providers docs for the full list.
No key leakage: API keys are never logged; errors are sanitized before returning.
Failure isolation: one model failing in ask_many / reason_together does not crash the call.
Synthesis ≠ truth: reason_together presents the critic's output as a synthesized answer, not ground truth.
No hidden reasoning exposed: traces summarize what happened (which model, which step) without exposing chain-of-thought internals.
Easy to extend: add any LiteLLM-supported model in models.yaml with no code changes.
Send the same prompt to multiple models in parallel. Returns one response per model. Individual failures are isolated — one model failing does not prevent others from responding.
| Name | Required | Description | Default |
|---|---|---|---|
| prompt | Yes | The prompt to send to all models. | |
| model_aliases | Yes | List of model aliases to query in parallel. | |
| system_prompt | No | Optional system prompt applied to all models. | |
| temperature | No | Sampling temperature (0–2). Default 0.7. | |
| max_tokens | No | Max tokens per response. Default 2048. |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behaviors. It mentions failure isolation, which is good, but omits side effects, authentication, rate limits, or response structure. Decent but incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the primary action and key feature (parallel execution, failure isolation). No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers core functionality but lacks guidance on usage context, error scenarios, or comparison to siblings. Output schema exists, so return values are covered. Still, given no annotations and moderate complexity, it could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds no additional parameter meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sends the same prompt to multiple models in parallel and returns one response per model. It distinguishes from sibling tools like 'ask_model' by emphasizing the multi-model parallel nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like 'ask_model', 'reason_together', or 'pick_best_answer'. The description implies usage for parallel prompting but lacks when-not-to-use or comparisons.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Send a prompt to a single configured model and return its response. Use this for a direct query to a specific provider/model.
| Name | Required | Description | Default |
|---|---|---|---|
| model_alias | Yes | Model alias from models.yaml (e.g. 'gpt', 'claude', 'gemini') | |
| prompt | Yes | The prompt / question to send. | |
| system_prompt | No | Optional system prompt. | |
| temperature | No | Sampling temperature (0–2). Default 0.7. | |
| max_tokens | No | Max tokens in the response. Default 2048. |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Describes the basic operation but omits potential side effects, error handling, or rate limits. For a straightforward query tool, this is minimally adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no wasted words. First sentence defines function, second adds usage guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 5 well-documented parameters and presence of an output schema, the description provides sufficient context for a simple tool. Does not explain return format, but output schema covers that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed parameter descriptions. The description adds no additional meaning beyond the schema, meeting the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'send a prompt' and the resource 'single configured model', and explicitly distinguishes from multi-model siblings like ask_many.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises 'use this for a direct query to a specific provider/model', implying when to use it. Though it does not list alternatives explicitly, the context from sibling names provides differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Ask one or more models to critique a draft answer to a question. Returns per-model critiques with identified weaknesses and suggested improvements. Useful for improving a draft before finalizing it.
| Name | Required | Description | Default |
|---|---|---|---|
| question | Yes | The original question the draft answer is responding to. | |
| draft_answer | Yes | The draft answer to be critiqued. | |
| model_aliases | Yes | Models to use as critics. | |
| temperature | No | Sampling temperature. Default 0.7. | |
| max_tokens | No | Max tokens per critique. Default 2048. |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. It states the tool returns per-model critiques, which is helpful, but does not mention potential costs, rate limits, or any side effects. It lacks depth but is not misleading.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loading the action and then the value. Every word earns its place; no fluff. It is highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is an output schema, return values are covered. However, the tool has 5 parameters and 3 required ones, and the description does not mention anything about token usage, model availability, or possible constraints. It is adequate but could provide more context for a critique tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the description does not need to add much. It does not provide additional semantics beyond the parameter descriptions in the schema, such as clarifying the role of 'model_aliases' or 'temperature.' Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'critique a draft answer to a question' and 'Returns per-model critiques with identified weaknesses and suggested improvements.' It uses a specific verb and resource, and distinguishes from sibling tools like 'ask_model' (which generates answers) and 'pick_best_answer' (which selects).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'Useful for improving a draft before finalizing it,' which provides clear context for when to use the tool. However, it does not explicitly state when not to use it or compare to alternatives, though sibling names imply different purposes.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all model aliases available in the current models.yaml configuration.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions only the action of listing, but with no annotations, the burden falls on the description. It does not explicitly state that the operation is read-only, non-destructive, or has any side effects. However, the simplicity of the tool (no parameters, no mutations) makes this minimally adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, complete sentence that fully explains the tool's functionality without any unnecessary words. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists, the description is mostly complete. However, it could provide more context about why listing model aliases is useful, especially in relation to sibling tools like ask_model. As it stands, it is adequate but not rich.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the schema is fully covered (100%). The description adds no parameter information, but none is needed. The baseline for zero-parameter tools is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states that the tool lists all model aliases from a specific configuration file. The verb 'list' and resource 'model aliases' are precise, and the tool name'list_models' aligns with this purpose. Sibling tools have different actions (ask, critique, pick, reason), making this tool distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. Although sibling tools are conceptually different, the description does not offer any context for decision-making, such as prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Given multiple candidate answers to a question, ask a judge model to rank them and identify the best one. Returns winner, ranking, and explanation.
| Name | Required | Description | Default |
|---|---|---|---|
| question | Yes | The question the candidate answers are responding to. | |
| candidate_answers | Yes | List of candidate answers to rank. | |
| judge_model_alias | No | Model to act as judge. Defaults to the first model in models.yaml. | |
| rubric | No | Optional evaluation rubric (e.g. 'Prioritize accuracy over brevity'). | |
| temperature | No | Sampling temperature for the judge. Default 0.2. | |
| max_tokens | No | Max tokens for the judge response. Default 2048. |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description is the sole source of behavioral info. It discloses the core action (ask a judge model to rank) and return values, but does not mention potential costs, latency, or any prerequisites. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 20 words that efficiently conveys the tool's purpose and outputs. No unnecessary words; front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown), the description does not need to detail return values. It covers the main purpose and key behavior. Missing details about the judge model selection or ranking logic, but acceptable for a tool with a rich schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage, so the baseline is 3. The description adds high-level context about returning winner, ranking, and explanation, but does not elaborate on parameter meanings beyond the schema. No additional semantics provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: given multiple candidate answers, it uses a judge model to rank them and identify the best one. This directly reflects the tool's name and purpose, and implicitly distinguishes it from siblings like ask_many and critique_answer.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when one has multiple candidate answers to compare, but does not explicitly state when not to use it or offer alternative tools. It provides clear context but lacks exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Multi-step reasoning workflow across multiple models. Strategies: • independent_then_critique (default): models answer independently, then a critic synthesizes. • debate: models see each other's answers and refine over N rounds, then a critic synthesizes. • red_team: proposer answers, others attack it, proposer revises — repeated for N rounds. Returns a trace, individual responses, and a final synthesized answer. The final answer is presented as synthesis, not ground truth.
| Name | Required | Description | Default |
|---|---|---|---|
| task | Yes | The question or task to reason about. | |
| model_aliases | Yes | Model aliases to use as reasoners. | |
| critic_model_alias | No | Alias of the model that critiques / synthesizes. Defaults to the first model in model_aliases. | |
| rounds | No | Number of debate/revision rounds (used by 'debate' and 'red_team'). Default 1. | |
| strategy | No | Strategy: 'independent_then_critique' | 'debate' | 'red_team'. Default 'independent_then_critique'. | independent_then_critique |
| system_prompt | No | Optional system prompt for all reasoner calls. | |
| temperature | No | Sampling temperature. Default 0.7. | |
| max_tokens | No | Max tokens per call. Default 2048. |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It thoroughly discloses the workflow steps for each strategy, the number of rounds, the roles of models and critic, and that the output is a synthesis. This equips the agent with key behavioral insights.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded with the essential purpose, followed by a clean list of strategies. No redundant sentences; every part adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (8 parameters, multi-step workflow) and the presence of an output schema, the description covers the workflow, strategies, and return values (trace, individual responses, synthesized answer), providing sufficient context for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds context about strategy behavior but does not significantly enhance understanding beyond the schema's parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is a 'multi-step reasoning workflow across multiple models' and lists three specific strategies (independent_then_critique, debate, red_team), which distinguishes it from simpler sibling tools like ask_model or ask_many.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains each strategy and their behavior (e.g., 'models answer independently, then a critic synthesizes') and notes that the final answer is 'synthesis, not ground truth'. However, it does not explicitly state when to prefer this tool over alternatives like ask_many or pick_best_answer.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0ask_manyask_modelcritique_answerlist_modelspick_best_answerreason_togetherScored across 6 tools
Each tool has a distinct purpose: ask_many for parallel queries, ask_model for single query, list_models for enumeration, critique_answer for critiquing drafts, pick_best_answer for ranking candidates, and reason_together for multi-step reasoning. No two tools overlap in function.
All tool names follow a consistent verb_noun pattern: ask_many, ask_model, critique_answer, list_models, pick_best_answer, reason_together. The verbs are descriptive and the naming style is uniform.
With 6 tools, the surface is well-scoped for a multi-model interaction server. Each tool adds clear value without redundancy or bloat, covering essential operations for querying, critiquing, ranking, and reasoning.
The tool set covers all core workflows: single/parallel queries, critique, ranking, and multi-step reasoning. A minor gap might be a tool for managing model configurations or conversation history, but the set is largely complete for the stated purpose.
MCP server for building and testing AI agents with multi-model experimentation and insights.
AI Reasoning Cache & Consensus Layer with 11 MCP tools via Streamable HTTP.
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
