compare-mcp

Multi-model code review with ranked todos and subagent dispatch, inside Claude Code CLI.

Claude Code is great at code review — but it only talks to one model. Copilot CLI recently shipped multi-model debug, letting you bounce a problem off GPT, Claude, and Gemini in one shot. Claude Code can't do that natively. This MCP server adds it: bring your own API keys, fan out to any combination of models, and get back a diffed, ranked list of what they each found.

Fan out any bug or task to multiple LLMs simultaneously, diff their unique insights, optionally run a debate round where models critique each other, then dispatch parallel subagents to implement the combined best fixes — each with its own git commit.

Demo

https://github.com/user-attachments/assets/8990dabb-bc61-4625-8930-c914cffe75da

/compare models → /compare review config.py for security issues → /compare --debate → /compare status

Architecture

Install

pip install compare-mcp
claude mcp add -s user compare-mcp -- python -m compare_mcp

Then grab the /compare skill and example config:

git clone https://github.com/carolinacherry/compare-mcp.git --depth 1
mkdir -p ~/.claude/skills ~/.compare
cp -r compare-mcp/.claude/skills/compare ~/.claude/skills/
cp compare-mcp/.compare/config.example.json ~/.compare/config.json

Quick start

1. Edit ~/.compare/config.json — enable at least 2 providers by setting "enabled": true and adding your API key (either as a $ENV_VAR reference or paste the key directly)

2. In Claude Code:

   /compare memory leak in the tile rendering loop
   /compare race condition in the connection pool --debate --providers claude,openai
   /compare status
   /compare models

Config reference

Config lives at ~/.compare/config.json. API keys use $ENV_VAR syntax — expanded at load time.

Provider types

Compare settings

Timeout note: Some models (e.g. Kimi's kimi-k2.5) are significantly slower than GPT-4o on large prompts and will time out at 60s. We default to 120s. If a provider consistently times out, try a faster model variant — for Kimi, moonshot-v1-auto is faster than kimi-k2.5 and auto-selects the right context window.

Adding providers

Any OpenAI-compatible endpoint

{
  "my_provider": {
    "enabled": true,
    "type": "openai_compat",
    "api_key": "$MY_API_KEY",
    "model": "model-name",
    "base_url": "https://api.example.com/v1"
  }
}

Works with: OpenAI, Kimi (api.moonshot.ai), Minimax (api.minimax.io), Gemini (generativelanguage.googleapis.com/v1beta/openai/), Ollama API (localhost:11434/v1), OpenRouter, Together AI, Groq, etc.

CLI subprocess model

{
  "ollama_local": {
    "enabled": true,
    "type": "cli",
    "cli_command": "ollama",
    "cli_args": ["run", "codellama"],
    "cli_parser": "text"
  }
}

cli_parser options: "text" (raw stdout), "json" (parse as JSON), "jsonl" (last complete JSON line).

Commands

In Claude Code, type any of these:

After /compare runs, you'll be asked whether to dispatch subagents to fix the findings in parallel. Each subagent gets one todo, implements the fix, and commits.

How it works

1. Dispatch — compare_run fans out the code + issue to all enabled providers via asyncio.gather. Providers that timeout or error are excluded, never crash the whole run.

2. Diff — compare_diff uses rapidfuzz (token sort ratio) to deduplicate findings across providers. Findings seen by 2+ providers are "shared"; the rest are "unique". Agreement rate = shared / total unique groups.

3. Debate (optional) — compare_debate sends each provider's findings to every other provider for critique. A synthesis call merges the results. Capped at 4 providers to limit API calls (N*(N-1)+1).

4. Todos — compare_todos writes ranked findings to SQLite. High severity first, then by provider count.

5. Execute — The /compare skill dispatches parallel Claude Code subagents, one per todo. Each implements the fix and commits.

MCP tools (7)

vs multi_mcp

multi_mcp does parallel dispatch well. compare-mcp builds the workflow layer on top:

vs Copilot CLI multi-model

Copilot CLI routes through GitHub's API proxy — no BYO keys, no Kimi/Minimax/local models. compare-mcp calls provider APIs directly: full context windows, your own rate limits, any model with an HTTP endpoint or CLI binary.

Development

git clone https://github.com/carolinacherry/compare-mcp.git
cd compare-mcp
pip install -e ".[dev]"
pytest
ruff check .