LLM Council MCP

🏛️ Provenance & What This Fork Changes

Fork Lineage:
This project is an advanced, production-hardened fork of Andrej Karpathy's llm-council (expanded from az9713/llm-council).

Karpathy built the original llm-council as a fun "Saturday vibe-hack" — a lightweight web script to compare commercial frontier LLMs side-by-side using OpenRouter.

LLM Council MCP transforms that initial prototype into an enterprise deliberation engine and a headless consensus oracle for autonomous AI agents. Instead of humans manually querying a browser, agents like Claude Code and Google Antigravity call the council programmatically via the Model Context Protocol (MCP) whenever they face irreversible, high-stakes (Type-1) architectural dilemmas.

Related MCP server: AI Consensus Connector

⚡ Original vs. LLM Council MCP: Feature Comparison

📸 Visual Tour of New Capabilities

1. Stage 1: Domain-Specialized Independent Responses

Models are not treated as generic chatbots. Each seat operates with an injected domain skill and provides explicit confidence calibration and structured reasoning.

2. Specialized Council Boards

Switch between dedicated expert boards with a single click or specify council_id in MCP tool calls:

• 🧠 cognitive-strategy: High-stakes architectural & strategic trade-offs (Red Team + First Principles + Deep Research).

• 🛠️ code-craft: Deep refactoring, diff-risk minimization & surgical simplicity.

• 🔬 deep-tech: Protocol RFCs, performance limits, and dependency audits.

• 🛡️ sec-ops: Production security, OWASP audits, and SRE resilience.

• 🎨 frontend-craft: Distinctive design systems, UI/UX DNA, and client workflows.

3. Empirical Performance Dashboard

Track which models and skills provide the most accurate evaluations through peer review. Features historical win rates, peer agreement metrics, and Chairman synthesis quality.

4. Advanced Consensus Modes & Settings

Configure early-exit consensus, Chain-of-Thought reasoning, adversarial reviews, and weighted voting directly from the settings drawer:

🔌 Using as an MCP Oracle (Claude Code & Antigravity)

The repository bundles a standalone FastMCP server in mcp/ that lets AI coding assistants deliberate before committing dangerous or irreversible changes.

The 4-Point Gating Guardrail

To prevent agents from lazily delegating routine tasks, ask_council enforces a strict gating checklist:

1. Type-1 Decision: Must be irreversible or carry a high rollback cost (justified in type1_rationale).

2. Genuine Uncertainty: The agent must have attempted solo reasoning first and encountered a real conflict or unknown.

3. High Cost of Error: The cost of picking the wrong path must exceed ~35s + API token cost.

4. User Has Not Decided: Council informs open choices; it never overrides an explicit user directive.

Trivial or unjustified queries are rejected in milliseconds with ## Verdict: Gating Rejection without triggering backend LLM calls.

Output Schema (Bounded ≤150-Word ADR)

Calls to ask_council return a structured, high-density Markdown Architectural Decision Record:

## Verdict: Use SQLite with WAL mode for local conversation storage
**Confidence:** Consensus — 3 models evaluated (top ranked: local/antigravity@red-team-reasoning)
**Recommendation:** Deploy SQLite with PRAGMA journal_mode=WAL and PRAGMA busy_timeout=5000. It eliminates network daemon failure modes and delivers near-zero operational complexity.
**Dissenting risk:** If write contention exceeds 1% busy timeouts under horizontal multi-process scale, pivot to PostgreSQL.

🚀 Quick Setup

1. Configure Environment

Copy the template and configure your local endpoints or OpenRouter API key:

cp .env.example .env

# Optional: OpenRouter API key (only needed for cloud models)
OPENROUTER_API_KEY=sk-or-v1-...

# Optional: Local vLLM / OpenAI-compatible endpoint (defaults to host gateway)
QWEN_BASE_URL=http://host.docker.internal:8002/v1

2. Run Full Stack via Docker Compose (Recommended)

docker compose -f infra/docker-compose.yml up -d

• Web UI: http://localhost:5173

• Backend API: http://localhost:8001 (Health check: curl http://localhost:8001/)

3. Connect MCP to Your Agents

cd mcp
bash install.sh
cd ..

Claude Code (~/.claude.json):

{
  "mcpServers": {
    "llm-council": {
      "command": "/absolute/path/to/llm-council-mcp/mcp/.venv/bin/python",
      "args": ["/absolute/path/to/llm-council-mcp/mcp/server.py"],
      "timeout": 140000
    }
  }
}

Antigravity (~/.gemini/config/mcp_config.json):

{
  "mcpServers": {
    "llm-council": {
      "command": "/absolute/path/to/llm-council-mcp/mcp/.venv/bin/python",
      "args": ["/absolute/path/to/llm-council-mcp/mcp/server.py"],
      "timeout": 140000
    }
  }
}

🛠️ Tech Stack

• Core Engine: FastAPI, Async HTTPX, Pydantic, uv

• Protocol: FastMCP (Model Context Protocol stdio transport)

• Frontend: React 18, Vite, Custom Design System, React Markdown

• Models: OpenRouter, vLLM (Qwen 2.5/3.6), Local Host Shims (Claude Code CLI, Antigravity CLI)

• Containerization: Docker & Docker Compose

📜 Acknowledgments & License

• Original concept and initial implementation by Andrej Karpathy

• Extended multi-feature baseline by az9713

• Released under the MIT License.