blessthis-llm-council

Blind multi-model councils, and direct seat chat, as a local MCP server.

Ask one question — get N independent answers from N different LLMs, each running as a real agent CLI on your machine, each answering anonymously so no model can bandwagon another's take. A synthesizer merges the answers, the council scores them blind, and only then are the hats lifted. When you don't need a whole council, open a direct 1:1 chat with any seat over the same MCP connection. You bring your own models, gateways, and credentials — the package is pure orchestration.

How it works

 you (via your agent host)
        │  brief
        ▼
  council_start ──► N anonymous seats (hat_1 … hat_N)
        │            each seat = one LLM family, spawned as a
        │            headless CLI subprocess (claude / pi / codex)
        ▼
  council_poll  ◄── watch seats work (live progress)
        │
        ▼
  council_answer / council_ask ──► collect answers, cross-examine
        │
        ▼
  synthesis ──► council_score (blind: hats still on)
        │
        ▼
  council_reveal ──► hats off: which model wrote what
        │
        ▼
  council_close

The conductor (your agent host: Claude Code, pi, Cursor, …) talks to the MCP server. The server spawns the seat CLIs, tracks per-(seat, model) health and cooldowns, and persists councils, scores, and chat sessions in a local SQLite database. No seat ever sees another seat's answer before scoring.

Related MCP server: conclave

Quick start

Requirements: uv, and at least one seat CLI installed (claude, pi, or codex) with credentials. Linux/macOS first-class; Windows untested (best-effort).

uvx blessthis-llm-council

That's it. The interactive wizard will:

1. Build your seats.yaml (seat templates for common setups, per-seat env prompts, optional 1-token probe per seat).

2. Ask about telemetry (opt-in, default off).

3. Wire up one host of your choice (MCP registration + council agent files). Re-run blessthis-llm-council install to add more hosts.

Then restart/reload your host and either ask the installed council agent ("run a council on …") or call the tools directly (e.g. mcp__llm-council__council_start).

Useful CLI commands (same binary):

blessthis-llm-council seats list|add|edit|remove|probe   # manage seats.yaml
blessthis-llm-council doctor                             # full diagnostics
blessthis-llm-council status                             # fast wiring overview
blessthis-llm-council uninstall                          # clean removal

Two console scripts ship in the package: blessthis-llm-council is the installer/management CLI; blessthis-llm-council-server is the MCP server your host launches (you never run it by hand — the wizard registers it).

seats.yaml

All seat topology and secrets live in one file: ~/.blessthis-llm-council/seats.yaml, mode 0600, never committed anywhere.

telemetry:
  enabled: false          # explicit, opt-in

seats:
  fable:
    models: [claude-fable-5, claude-opus-4-8]   # preferred first
    agent:
      bin: claude
      args: ["-p", "{prompt}", "--output-format", "json",
             "--dangerously-skip-permissions",
             "--model", "{model}", "--add-dir", "{workdir}"]
      env:
        ANTHROPIC_API_KEY: __REPLACE_ME__

  glm:                       # pi reads its own models.json — no env needed
    models: [glm-5.2]
    agent:
      bin: pi
      args: ["--mode", "json", "-p", "{prompt}", "--model", "{model}",
             "--tools", "read,write,edit,grep,find,ls,bash",
             "--add-dir", "{workdir}"]
      env: {}

  gpt:                       # codex runner via an OpenAI-wire gateway
    models: [gpt-5]
    agent:
      bin: codex
      args: ["exec", "--json", "--skip-git-repo-check", "-s", "read-only",
             "--color", "never", "-m", "{model}", "-C", "{workdir}", "{prompt}"]
      env:
        OPENAI_BASE_URL: https://your-gateway.example/v1
        OPENAI_API_KEY: __REPLACE_ME__

A seat is an LLM family, not a runner — the same binary can back several seats with different credentials (e.g. native API + gateway fallback as two seats, preferred-first). args is a pure exec-array (one argv token per element, {prompt} / {model} placeholders required). The wizard writes this file for you; edit later with blessthis-llm-council seats edit <name>.

Tools reference (17)

Council (10)

Direct seat chat (6)

Discovery (1)

Host support

Everywhere the server is registered under the canonical name llm-council as { "command": "uvx", "args": ["blessthis-llm-council-server"] } — a non-destructive merge that never touches your other servers or agents (we only ever write/remove entries matching our fingerprint and blessthis-council-* files).

Philosophy

• Why blind? Models bandwagon. Show an LLM another model's answer and you get agreement theater, not independent judgment. Hats stay on until scoring is done, so scores reflect the answer, not the author.

• Why real CLIs instead of API calls? Your seat CLIs already have working auth, tooling, session resume, and model routing. The council spawns the same claude / pi / codex binaries you use interactively — headless, with your config — so there's no second credential store to break and no reimplemented client to drift. This package never makes a direct LLM HTTP call.

• BYO everything. Models, gateways, API keys, base URLs — all yours, all in seats.yaml. No hosted component, no account, no lock-in.

Telemetry

Opt-in (telemetry.enabled: true, asked during install, default off).

When enabled, we send only anonymized scoring events: {model, kind, score, usage, host, tool_version, ts, council_uuid}.

We never send: your code, file paths, briefs, answer content, notes, seat names, credentials, or seat-health data.

Uninstall & privacy

blessthis-llm-council uninstall            # removes MCP entries + agent files
blessthis-llm-council uninstall --purge    # also deletes seats.yaml + state.db

Everything lives under ~/.blessthis-llm-council/ (seats, SQLite state) plus the llm-council entry and blessthis-council-* files in your host configs. Nothing else on your system is touched. seats.yaml contains your API keys — keep it 0600 and never commit it.

License

AGPL-3.0-or-later. Fork it, improve it — but improvements to this package must be shared back under the same license.