Conclave MCP
Conclave MCP
여러 LLM 모델의 '컨클레이브(conclave)'에 액세스할 수 있게 해주는 MCP(Model Context Protocol) 서버입니다. MCP 호환 클라이언트를 통해 다양한 의견, 동료 평가, 종합적인 답변을 얻을 수 있습니다.
이 프로젝트의 목적
AI 어시스턴트와 작업할 때, 보통은 한 모델의 관점만 얻게 됩니다. 때로는 그것으로 충분할 때도 있습니다. 하지만 기술 아키텍처, 비즈니스 전략, 창의적 방향, 복잡한 분석 등 맹점이 중요한 상황에서는 다양한 의견이 숨겨진 대안을 찾아낼 수 있습니다.
Conclave는 민주적인 AI 합의를 모든 워크플로우에 도입합니다.
여러 AI 서비스를 수동으로 쿼리하는 대신, Claude Desktop, Claude Code 또는 모든 MCP 클라이언트를 통해 컨클레이브에 문의할 수 있습니다. 여러 최첨단 모델(GPT, Claude, Gemini, Grok, DeepSeek)로부터 순위가 매겨진 의견을 얻고, 집단적 AI 지혜를 나타내는 종합적인 답변을 받아보세요.
사용 사례:
기술: 아키텍처 결정, 코드 리뷰, 디버깅, API 설계
비즈니스: 전략 분석, 제안서 검토, 시장 조사 종합
창의적 작업: 글쓰기 피드백, 브레인스토밍, 편집 관점
연구: 문헌 검토, 사실 확인, 다각적 분석
의사 결정: 장단점 분석, 위험 평가, 옵션 평가
Andrej Karpathy의 llm-council 개념에서 영감을 받았습니다. 이 프로젝트는 핵심 아이디어를 MCP 서버로 재구현하여 AI 보조 워크플로우와 원활하게 통합되도록 했습니다.
Related MCP server: AI Council MCP Server
작동 방식
컨클레이브는 최대 3단계로 운영됩니다:
┌─────────────────────────────────────────────────────────────────┐
│ Stage 1: OPINIONS │
│ Query multiple LLMs in parallel for independent responses │
│ (GPT, Claude, Gemini, Grok, DeepSeek, etc.) │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ Stage 2: PEER RANKING │
│ Each model anonymously evaluates and ranks all responses │
│ Aggregate scores reveal best performers (lower = better) │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ Stage 3: SYNTHESIS │
│ Chairman model synthesizes final answer from collective wisdom │
│ Consensus level reported (strong/moderate/weak/split) │
│ Tiebreaker vote cast if conclave is split │
└─────────────────────────────────────────────────────────────────┘기능
계층화된 쿼리: 비용/깊이 트레이드오프 선택 (quick | ranked | full)
3단계 위원회 계층: 프리미엄(최첨단), 표준(균형), 예산(빠름/저렴)
합의 프로토콜: 합의 수준을 감지하고 의견이 갈릴 경우 타이브레이커(결정권자)를 트리거
홀수 컨클레이브 규모: 타이브레이커 투표로 교착 상태 해결 보장
순환 의장제: 매주 의장을 교체하여 단일 모델 편향 방지
의장 프리셋: 상황별 의장 선택 (코드, 창의적, 추론)
비용 추정: 쿼리 전 예상 비용 확인
Eval-light: 시간 경과에 따른 성능 추적을 위한 독립형 벤치마크 러너
설치
사전 요구 사항
https://openrouter.ai/keys 에서 OpenRouter API 키를 받으세요.
OpenRouter 계정에 크레딧을 충전하세요 (종량제).
설정
# Clone the repository
git clone https://github.com/stephenpeters/conclave-mcp.git
cd conclave-mcp
# Install dependencies
uv sync
# Optional: Create .env file for running tests locally
# (Not required for MCP usage - API key is passed via client config)
echo "OPENROUTER_API_KEY=sk-or-v1-your-key-here" > .envClaude Desktop 구성
옵션 1: 데스크톱 확장 프로그램 (권장)
Claude Desktop을 엽니다.
**Settings > Extensions > Advanced settings > Install Extension...**으로 이동합니다.
conclave-mcp디렉토리로 이동합니다.안내에 따라
OPENROUTER_API_KEY를 구성합니다.Claude Desktop을 재시작합니다.
옵션 2: 수동 구성
Claude Desktop을 열고 Settings > Developer > Edit Config로 이동한 뒤, claude_desktop_config.json에 다음을 추가합니다:
{
"mcpServers": {
"conclave": {
"command": "uv",
"args": ["run", "--directory", "/path/to/conclave-mcp", "python", "server.py"],
"env": {
"OPENROUTER_API_KEY": "sk-or-v1-your-key-here"
}
}
}
}/path/to/conclave-mcp를 실제 경로로 바꾸고 저장한 뒤 Claude Desktop을 재시작하세요.
Claude Code 구성
CLI를 사용하여 서버를 추가합니다:
claude mcp add --transport stdio conclave -- uv run --directory /path/to/conclave-mcp python server.py --env OPENROUTER_API_KEY=sk-or-v1-your-key-here또는 .mcp.json.example을 .mcp.json으로 복사하고 경로를 업데이트합니다:
cp .mcp.json.example .mcp.json
# Edit .mcp.json with your paths and API keyClaude Code에서 /mcp를 입력하거나 터미널에서 claude mcp list를 실행하여 확인합니다.
사용 가능한 도구
conclave_quick
빠른 병렬 의견 (1단계만). 모든 컨클레이브 모델을 쿼리하고 개별 응답을 반환합니다.
비용: 쿼리당 약 $0.01-0.03
용도: 빠른 브레인스토밍, 다양한 관점을 빠르게 얻을 때
conclave_ranked
동료 순위가 포함된 의견 (1단계 + 2단계). 특정 질문에서 어떤 모델이 가장 뛰어난 성능을 보였는지 보여줍니다.
비용: 쿼리당 약 $0.05-0.10
용도: 코드 리뷰, 접근 방식 비교, 어떤 모델이 '승리'했는지 확인할 때
conclave_full
종합적인 컨클레이브 (3단계 모두). 합의 감지 및 의장 타이브레이커가 포함됩니다.
비용: 쿼리당 약 $0.10-0.20
옵션:
tier: 모델 계층 -"premium","standard"(기본값),"budget"chairman: 의장 모델 재정의 (예:"anthropic/claude-sonnet-4")chairman_preset: 프리셋 사용 ("code","creative","reasoning","concise","balanced")
용도: 중요한 결정, 아키텍처 선택, 복잡한 디버깅
conclave_config
현재 구성 보기: 컨클레이브 멤버, 의장 순환 상태, 합의 임계값.
conclave_estimate
쿼리 실행 전 비용 추정.
conclave_models
선택 번호와 함께 사용 가능한 모든 모델 나열. 계층별로 그룹화된 모델을 안정적인 번호로 표시:
프리미엄 계층: 1-10
표준 계층: 11-20
예산 계층: 21-30
의장 풀: 31-40
conclave_select
모델 번호로 사용자 지정 컨클레이브 생성. 첫 번째 모델이 의장이 됩니다.
conclave_select(models="31,1,11,21")생성 결과:
의장: #31 (deepseek-r1)
멤버: #1 (claude-opus-4.5), #11 (claude-sonnet-4.5), #21 (gemini-2.5-flash)
사용자 지정 선택은 서버 재시작 또는 conclave_reset 전까지 유지됩니다.
conclave_reset
사용자 지정 컨클레이브 선택을 지우고 계층 기반 구성으로 돌아갑니다.
사용자 지정 모델 선택
컨클레이브에 참여할 모델을 완전히 제어하려면:
사용 가능한 모델 나열:
conclave_models를 사용하여 모든 모델과 번호를 확인합니다.라인업 선택:
conclave_select(models="31,1,11,21")을 사용합니다 (첫 번째 번호가 의장).쿼리: 평소처럼
conclave_quick,conclave_ranked또는conclave_full을 사용합니다.재설정:
conclave_reset을 사용하여 계층 기반 구성으로 돌아갑니다.
예시 워크플로우:
> conclave_models
## Available Models
### Premium Tier (1-10)
1. anthropic/claude-opus-4.5
2. google/gemini-3-pro-preview
...
> conclave_select(models="31,1,12,21")
## Custom Conclave Created
Chairman (#31): deepseek/deepseek-r1
Members:
- #1: anthropic/claude-opus-4.5
- #12: google/gemini-2.5-pro
- #21: google/gemini-2.5-flash
> conclave_quick("What is the best approach for...")
[Uses your custom selection]
> conclave_reset
## Custom Conclave Cleared구성
config.py를 편집하여 사용자 지정:
컨클레이브 계층
각 계층은 적절한 가격/성능 차별화를 위해 고유한 모델(중복 없음)을 가집니다:
# Premium: 6 frontier models for complex questions (~$0.30-0.50/query)
COUNCIL_PREMIUM = [
"anthropic/claude-opus-4.5", # Claude Opus 4.5
"google/gemini-3-pro-preview", # Gemini 3 Pro
"x-ai/grok-4", # Grok 4 (full reasoning)
"openai/gpt-5.1", # GPT-5.1 (flagship)
"deepseek/deepseek-v3.2-speciale", # DeepSeek V3.2 Speciale
"moonshotai/kimi-k2-thinking", # Kimi K2 Thinking (1T MoE)
]
# Standard: 4 balanced models (default) (~$0.10-0.20/query)
COUNCIL_STANDARD = [
"anthropic/claude-sonnet-4.5", # Claude Sonnet 4.5
"google/gemini-2.5-pro", # Gemini 2.5 Pro
"openai/o4-mini", # OpenAI o4-mini
"deepseek/deepseek-chat-v3.1", # DeepSeek Chat V3.1
]
# Budget: 4 cheap/fast models (~$0.02-0.05/query)
COUNCIL_BUDGET = [
"google/gemini-2.5-flash", # Gemini 2.5 Flash
"qwen/qwen3-235b-a22b:free", # Qwen 3 235B (free tier)
"openai/gpt-4.1-mini", # GPT-4.1 Mini
"moonshotai/kimi-k2:free", # Kimi K2 (free tier)
]의장 순환
의장 풀은 고품질 종합을 위해 추론 모델만 사용합니다 (채팅 모델 제외):
CHAIRMAN_ROTATION_ENABLED = True
CHAIRMAN_ROTATION_DAYS = 7 # Rotate weekly
CHAIRMAN_POOL = [
"deepseek/deepseek-r1", # DeepSeek R1 reasoning
"openai/o3-mini", # OpenAI o3-mini reasoning
"anthropic/claude-sonnet-4", # Claude Sonnet 4 (strong reasoning)
"qwen/qwq-32b", # Qwen QWQ reasoning model
]합의 임계값
CONSENSUS_STRONG_THRESHOLD = 0.75 # 75%+ agreement
CONSENSUS_MODERATE_THRESHOLD = 0.50 # 50-75% agreement
CHAIRMAN_TIEBREAKER_ENABLED = True # Chairman breaks tiesEval-Light
계층 전반에 걸쳐 컨클레이브 성능을 테스트하고 비교하기 위한 독립형 벤치마크 러너입니다.
테스트 스위트 개요
평가 스위트에는 다양한 모델 기능을 테스트하기 위해 설계된 9개 카테고리의 16개 작업이 포함되어 있습니다:
카테고리 | 작업 수 | 난이도 | 테스트 항목 |
math | 2 | 쉬움-보통 | 산술, 문장제 문제, 단계별 추론 |
code | 2 | 쉬움-보통 | 버그 감지, 개념 설명, 코드 예제 |
reasoning | 2 | 보통-어려움 | 삼단논법, 다단계 논리 퍼즐 |
analysis | 2 | 보통 | 논리적 오류, 트레이드오프 분석 |
summarization | 2 | 보통 | 기술 문서, 비즈니스 보고서 |
writing_business | 2 | 쉬움-보통 | 전문 이메일, 제안서 |
writing_creative | 2 | 쉬움-보통 | 이야기 도입부, 독창적인 은유 |
creative | 1 | 쉬움 | 설명이 포함된 유추 |
factual | 1 | 쉬움 | 일반 대중을 위한 과학 설명 |
평가 실행
# Run all 16 tests at standard tier (default)
python eval.py
# Run at different tiers
python eval.py --tier premium # 6 frontier models (~$0.30-0.50/query)
python eval.py --tier standard # 4 balanced models (~$0.10-0.20/query)
python eval.py --tier budget # 4 cheap/fast models (~$0.02-0.05/query)
# Different modes
python eval.py --mode quick # Stage 1 only (fastest, cheapest)
python eval.py --mode ranked # Stage 1 + 2 (adds peer rankings)
python eval.py --mode full # All 3 stages (default, includes synthesis)
# Filter by category
python eval.py --category math
python eval.py --category code
python eval.py --category reasoning
# Don't save results to disk
python eval.py --no-save
# Combine options
python eval.py --tier premium --mode full --category reasoning출력 형식
결과는 evals/eval_<tier>_<mode>_<timestamp>.json에 저장되며 다음을 포함합니다:
metadata: 타임스탬프, 계층, 모드, 의장 모델
summary: 성공률, 총 시간, 작업당 평균 시간
results: 다음을 포함한 작업별 세부 정보:
개별 모델 응답
동료 순위 (ranked/full 모드)
의장 종합 (full 모드)
합의 수준
출력 예시
🏛️ Conclave Eval-Light
Tier: standard | Mode: full | Tasks: 16
--------------------------------------------------
[1/16] Running: math_arithmetic (math)
✓ Completed in 12.34s
[2/16] Running: math_word_problem (math)
✓ Completed in 15.67s
...
==================================================
📊 EVAL SUMMARY
==================================================
Tier: standard | Mode: full
Chairman: deepseek/deepseek-r1
Tasks: 16/16 successful
Total time: 287.45s
Avg per task: 17.97s
📋 Results by Task:
✓ math_arithmetic (easy) - 12.34s
✓ math_word_problem (medium) - 15.67s
✓ code_debug (easy) - 11.23s
...
💾 Results saved to: evals/eval_standard_full_20251204_143052.json계층 비교
모든 계층에서 동일한 평가를 실행하여 모델 품질 대 비용을 비교하세요:
python eval.py --tier budget --category reasoning
python eval.py --tier standard --category reasoning
python eval.py --tier premium --category reasoning그런 다음 JSON 출력을 비교하여 동일한 작업에서 다른 모델 계층이 어떻게 수행되는지 확인하세요.
사용 사례
시나리오 | 권장 도구 | 이유 |
"이 함수 검토해줘" |
| 어떤 모델이 가장 많은 문제를 찾아내는지 확인 |
"세션용 Redis vs PostgreSQL?" |
| 중요한 결정, 종합 필요 |
"이 기능에 대한 아이디어" |
| 빠르고 다양한 브레인스토밍 |
"이 오류 디버깅" |
| 빠른 병렬 진단 |
"이 문단 다시 쓰기" |
| 창의적 종합 |
"이 아키텍처가 건전한가?" |
| 기술적 종합 |
도구 출력 예시
## Conclave Full Result
**Consensus: ✅ STRONG** (75% agreement)
---
### Chairman's Synthesis
_Chairman: deepseek/deepseek-r1_
[Synthesized answer incorporating best points from all models...]
---
### Model Rankings (lower is better)
1. **claude-sonnet-4.5**: 1.50
2. **o4-mini**: 2.00
3. **gemini-2.5-pro**: 2.75
4. **deepseek-v3.1**: 3.75
_First-place votes:_ claude-sonnet-4.5=3, o4-mini=1프로젝트 구조
conclave-mcp/
├── server.py # MCP server entry point (5 tools)
├── conclave.py # Core 3-stage council logic
├── config.py # Model tiers, chairman rotation, cost estimates
├── eval.py # Standalone benchmark runner
└── evals/ # Saved evaluation results모델 추가
OpenRouter는 200개 이상의 모델을 지원합니다. https://openrouter.ai/models 에서 모델 ID를 찾으세요.
# Add to COUNCIL_* lists in config.py
"x-ai/grok-4" # xAI Grok
"meta-llama/llama-4-maverick" # Meta Llama
"mistralai/mistral-large-2" # Mistral
"deepseek/deepseek-r1" # DeepSeek reasoning중요: 적절한 차별화를 위해 각 계층의 모델을 고유하게 유지하세요(중복 없음).
OpenRouter 작동 방식
OpenRouter는 통합 API 게이트웨이입니다. OpenAI, Google, Anthropic 등에 별도의 계정을 가질 필요가 없습니다. 하나의 API 키, 하나의 크레딧 잔액으로 모든 모델에 액세스할 수 있습니다.
크레딧 충전 (선불 또는 자동 충전 활성화)
모든 모델에 단일 API 키 사용
라이선스
MIT
저작자 표시
Andrej Karpathy의 llm-council에서 영감을 받았습니다. 원본은 LLM 비교를 대화형으로 탐색하기 위한 웹 애플리케이션입니다. 이 프로젝트는 컨실 개념을 AI 보조 편집기와의 통합을 위한 MCP 서버로 재구현하고, 합의 프로토콜 및 타이브레이커 메커니즘을 추가했습니다.
Available Tools
8 toolsconclave_configA
View current conclave configuration.
Shows conclave member models, current chairman with rotation info, available chairman presets, consensus thresholds, and API key status.
Also shows custom conclave selection if active.
Returns: Current configuration as formatted JSON
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that this is a read-only operation ('View') and describes the return format ('formatted JSON'), but lacks details on permissions, rate limits, or error behavior. It adds some context about what data is included, which is helpful but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose, followed by specific details in bullet-like structure, and ends with return information. Every sentence adds value without redundancy, making it efficient and well-organized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (configuration viewing with multiple data points), no annotations, and an output schema present, the description is mostly complete. It lists what data is shown and the return format, but could improve by mentioning sibling differentiation or behavioral constraints like authentication needs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, so no parameter documentation is needed. The description correctly doesn't discuss parameters, earning a high baseline score for not adding unnecessary information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('View current conclave configuration') and lists the exact resources it shows (member models, chairman info, presets, thresholds, API key status, custom selection). It distinguishes from siblings like 'conclave_estimate' or 'conclave_reset' by focusing on configuration viewing rather than estimation or resetting.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'conclave_models' (which might show models only) or 'conclave_full' (unclear purpose). It implies usage for viewing configuration but doesn't specify scenarios, prerequisites, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
conclave_estimateA
Estimate cost for a conclave query before running it.
Provides approximate cost breakdown for quick/ranked/full query types.
Args: question: The question (used to estimate token count) tier: Which tier to estimate - "quick", "ranked", "full" (default: all)
Returns: Cost estimates for each query type
| Name | Required | Description | Default |
|---|---|---|---|
| question | Yes | ||
| tier | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the tool provides 'approximate cost breakdown' and estimates based on token count, which adds useful behavioral context. However, it doesn't mention potential limitations like accuracy, rate limits, or authentication needs, leaving gaps for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by brief elaboration. Every sentence adds value without redundancy, and the structure with 'Args:' and 'Returns:' sections enhances readability without unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 2 parameters with 0% schema coverage and an output schema present, the description is mostly complete. It explains parameters and return values ('Cost estimates for each query type'), but could benefit from more detail on behavioral aspects like error handling or prerequisites, especially since no annotations are provided.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds meaning beyond the schema by explaining that 'question' is 'used to estimate token count' and 'tier' specifies 'quick/ranked/full query types' with a default of 'all'. This clarifies parameter purposes, though it doesn't detail format constraints or examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Estimate cost for a conclave query before running it.' It specifies the verb ('estimate'), resource ('cost'), and scope ('before running it'), distinguishing it from sibling tools like conclave_quick or conclave_full that likely execute queries rather than estimate costs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: 'before running it' implies it's for pre-execution cost estimation. However, it doesn't explicitly state when not to use it or name alternatives among siblings, such as comparing to conclave_config or conclave_select, which might have overlapping or related purposes.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
conclave_fullA
Run the full conclave with synthesis (all 3 stages).
Most comprehensive - collects opinions, peer rankings, then has a Chairman model synthesize the best possible answer from the collective wisdom.
If a custom conclave is active (via conclave_select), it will be used instead of the tier-based config. The custom chairman overrides the chairman and chairman_preset parameters.
Args: question: The question to ask the conclave tier: Model tier - "premium" (complex), "standard" (default), "budget" (simple) Ignored if custom conclave is active. chairman: Override chairman model (e.g., 'anthropic/claude-sonnet-4') Ignored if custom conclave is active. chairman_preset: Use a context-based preset - "code", "creative", "reasoning", "concise", "balanced" Ignored if custom conclave is active.
Returns: Chairman's synthesis, consensus level, rankings, and individual responses
| Name | Required | Description | Default |
|---|---|---|---|
| question | Yes | ||
| tier | No | standard | |
| chairman | No | ||
| chairman_preset | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does well by explaining the multi-stage process ('collects opinions, peer rankings, then has a Chairman model synthesize'), the override behavior with custom conclaves, and what the tool returns. It doesn't mention rate limits, auth needs, or error conditions, but provides substantial behavioral context beyond basic functionality.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loaded with the core purpose first. The Args and Returns sections are well-structured. Some sentences could be slightly more concise, but overall it's efficient with zero wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (multi-stage synthesis with overrides), no annotations, and 0% schema coverage, the description provides complete context. It explains the process, parameter semantics, conditional behavior, and return values. The output schema exists, so the description appropriately doesn't need to detail return structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description fully compensates by explaining all 4 parameters in detail: what 'question' is for, the meaning of 'tier' values, what 'chairman' overrides, and the purpose of 'chairman_preset' options. It also clarifies conditional behavior ('Ignored if custom conclave is active') that isn't in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verbs ('Run the full conclave with synthesis') and resources ('all 3 stages'), and distinguishes it from siblings by emphasizing it's the 'most comprehensive' option that includes synthesis. It explicitly mentions what makes it different from other conclave tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool ('Most comprehensive') and when parameters are ignored ('Ignored if custom conclave is active'). It also implies alternatives through sibling tool names like conclave_quick and conclave_ranked, giving clear context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
conclave_modelsA
List all available models with selection numbers.
Shows all models from all tiers with unique numbers that can be used with conclave_select to create a custom conclave.
Numbers are stable:
Premium tier: 1-10
Standard tier: 11-20
Budget tier: 21-30
Chairman pool: 31-40
Returns: Numbered list of all available models grouped by tier
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does well by disclosing key behavioral traits: it lists models grouped by tier, specifies that numbers are stable with defined ranges per tier, and describes the return format as a numbered list. It doesn't mention aspects like rate limits or authentication needs, but covers essential behavior adequately for a read-only tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose in the first sentence, followed by supporting details in bullet points and a returns section. Every sentence earns its place by adding specific information about tiers, number stability, and usage context without any redundant or vague statements.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no annotations, but with an output schema), the description is complete. It explains the purpose, behavioral context (stable numbers per tier), usage with 'conclave_select', and return format. The output schema likely details the structure, so the description doesn't need to exhaustively list return values, making it well-rounded for this context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately focuses on output semantics, explaining the numbered list structure and tier groupings. This adds value beyond the schema by clarifying what the tool returns, which is helpful given the presence of an output schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and resource 'all available models with selection numbers', specifying it shows models from all tiers with unique numbers. It distinguishes from siblings by mentioning these numbers are used with 'conclave_select' to create custom conclaves, providing specific differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context by explaining that the numbers are used with 'conclave_select' for custom conclave creation, giving a specific when-to-use scenario. However, it doesn't explicitly state when not to use this tool or compare it to alternatives like 'conclave_quick' or 'conclave_full', which could help further differentiate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
conclave_quickA
Query the conclave for quick parallel opinions (Stage 1 only).
Fast and cheap - queries all conclave models in parallel and returns their individual responses. No peer ranking or synthesis. Good for getting diverse perspectives quickly.
If a custom conclave is active (via conclave_select), it will be used instead of the tier-based config.
Args: question: The question to ask the conclave tier: Model tier - "premium" (frontier), "standard" (default), "budget" (cheap/fast) Ignored if custom conclave is active.
Returns: Individual responses from each conclave model
| Name | Required | Description | Default |
|---|---|---|---|
| question | Yes | ||
| tier | No | standard |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it's 'Fast and cheap', queries 'all conclave models in parallel', returns 'individual responses' without synthesis, and mentions the interaction with conclave_select for custom conclaves. It doesn't cover rate limits, authentication needs, or error handling, but provides substantial operational context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly structured and concise: purpose statement first, key characteristics next, behavioral notes, then parameter details in labeled sections. Every sentence earns its place with no redundancy or fluff. The use of sections (Args, Returns) enhances readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 2 parameters with 0% schema coverage and no annotations, the description does an excellent job explaining parameters and behavioral context. The existence of an output schema means it doesn't need to detail return values. It could mention more about error cases or prerequisites, but covers the essential context well for this query tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds meaningful semantics for both parameters: 'question' is described as 'The question to ask the conclave', and 'tier' gets detailed explanation of values ('premium', 'standard', 'budget') with defaults and the override rule when custom conclave is active. This goes well beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verbs ('Query the conclave for quick parallel opinions') and distinguishes it from siblings by specifying 'Stage 1 only', 'Fast and cheap', 'No peer ranking or synthesis', and 'Good for getting diverse perspectives quickly'. It explicitly differentiates from tools like conclave_full or conclave_ranked that likely involve synthesis or ranking.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool ('Good for getting diverse perspectives quickly') and when not to use it ('Stage 1 only', 'No peer ranking or synthesis'). It also mentions the alternative of using a custom conclave via conclave_select, though it could be more explicit about other sibling alternatives like conclave_full or conclave_ranked.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
conclave_rankedA
Query the conclave with peer rankings (Stage 1 + 2).
Medium cost - gets individual opinions, then has each model anonymously evaluate and rank all responses. Returns aggregate "street cred" scores showing which models performed best on this specific question.
If a custom conclave is active (via conclave_select), it will be used instead of the tier-based config.
Args: question: The question to ask the conclave tier: Model tier - "premium" (frontier), "standard" (default), "budget" (cheap/fast) Ignored if custom conclave is active.
Returns: Individual responses plus aggregate rankings
| Name | Required | Description | Default |
|---|---|---|---|
| question | Yes | ||
| tier | No | standard |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses key behavioral traits: the two-stage process (individual opinions then anonymous ranking), cost level ('medium cost'), and the effect of 'conclave_select'. However, it doesn't cover important aspects like rate limits, authentication needs, error handling, or what 'street cred' scores entail, leaving gaps for a tool with no annotation support.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loaded, starting with the core purpose. The sentences are efficient, but the 'Args' and 'Returns' sections could be integrated more seamlessly, and some phrasing ('medium cost') is slightly vague, slightly reducing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given complexity (multi-stage ranking process), no annotations, and an output schema present, the description is mostly complete. It covers the process, parameters, and return overview, but lacks details on output structure or error cases, which the output schema might handle, making it adequate but not fully comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains 'question' as 'the question to ask the conclave' and 'tier' with values and default, adding meaning beyond the bare schema. However, it doesn't detail format constraints for 'question' or fully explain 'tier' implications beyond the list, resulting in partial compensation for the low coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool 'queries the conclave with peer rankings' and specifies it involves 'Stage 1 + 2' processing, which distinguishes it from simple query tools. However, it doesn't explicitly differentiate from siblings like 'conclave_full' or 'conclave_quick' in terms of ranking methodology, leaving some ambiguity about sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for usage: it mentions 'medium cost' and explains when to use it (for getting individual opinions and aggregate rankings). It also notes that a custom conclave via 'conclave_select' overrides the tier parameter, offering some alternative guidance. However, it lacks explicit when-not-to-use scenarios or comparisons to specific siblings like 'conclave_estimate' or 'conclave_quick'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
conclave_resetA
Clear custom conclave selection and return to tier-based config.
After reset, queries will use the tier parameter (premium/standard/budget) instead of the custom model selection.
Returns: Confirmation that custom selection was cleared
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden and does well by disclosing the behavioral outcome ('clear custom selection', 'return to tier-based config', 'queries will use tier parameter') and return value ('Confirmation that custom selection was cleared'), though it lacks details on permissions or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is front-loaded with the core action in the first sentence, followed by outcome and return details in clear, efficient sentences. Every sentence adds value without waste, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no annotations, but has output schema), the description is nearly complete by explaining the reset action, post-reset behavior, and return value. It could slightly improve by mentioning any prerequisites or errors, but covers essentials well.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, so no parameter info is needed. The description appropriately focuses on behavior and output, earning a baseline 4 for not adding unnecessary details beyond the empty schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Clear custom conclave selection') and the resource affected ('tier-based config'), distinguishing it from siblings like conclave_config or conclave_select that likely configure or choose models rather than resetting to defaults.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It implicitly indicates usage context ('return to tier-based config') and the effect ('queries will use the tier parameter'), but does not explicitly state when to use this vs. alternatives like conclave_config or what triggers a reset need, missing explicit exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
conclave_selectA
Create a custom conclave from model numbers.
Select specific models by their numbers (from conclave_models). The first model in the list becomes the chairman.
This custom selection persists until server restart or conclave_reset.
Args: models: Comma-separated model numbers, e.g. "1,5,11,14" First number = chairman, rest = conclave members
Returns: Confirmation of the new conclave configuration
Example: conclave_select(models="31,1,11,21") creates: - Chairman: model #31 (deepseek-r1) - Members: models #1, #11, #21
| Name | Required | Description | Default |
|---|---|---|---|
| models | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does so well. It discloses key behavioral traits: the custom selection persists until server restart or conclave_reset, the first model becomes chairman, and it references sibling tools (conclave_models, conclave_reset) for context. It doesn't mention permissions, rate limits, or error handling, but covers persistence and structure adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with the core purpose, followed by usage details, persistence, args, returns, and an example. Every sentence adds value without redundancy, and the example efficiently illustrates the tool's behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (custom conclave creation with persistence), no annotations, and an output schema present, the description is complete. It covers purpose, usage, parameters, behavioral traits, and includes an example, making it sufficient for an AI agent to understand and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 0%, so the description must compensate, which it does excellently. It explains the 'models' parameter as comma-separated model numbers, specifies the first number is chairman and the rest are members, provides an example format, and clarifies the mapping to specific models (e.g., model #31 = deepseek-r1).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Create a custom conclave from model numbers' with specific actions like selecting models and designating a chairman. It distinguishes from siblings by focusing on custom selection rather than configuration, estimation, or resetting.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: to create a custom conclave from specific model numbers, with the first model as chairman. It mentions persistence until server restart or conclave_reset, but does not explicitly state when to use alternatives like conclave_quick or conclave_ranked.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
8 tool updates
v0.2.0- First observed
conclave_config - First observed
conclave_estimate - First observed
conclave_full - First observed
conclave_models - First observed
conclave_quick - First observed
conclave_ranked - First observed
conclave_reset - First observed
conclave_select
TDQS
Scored across 8 tools
Each tool has a clearly distinct purpose with no ambiguity. For example, conclave_quick, conclave_ranked, and conclave_full represent distinct stages of query processing, while conclave_select and conclave_reset manage custom configurations, and conclave_models and conclave_config provide informational views. The descriptions clearly differentiate their roles.
All tool names follow a consistent snake_case pattern with a 'conclave_' prefix and descriptive suffixes (e.g., conclave_config, conclave_estimate, conclave_full). This uniformity makes the tool set predictable and easy to understand, enhancing usability for agents.
With 8 tools, the server is well-scoped for its purpose of managing and querying a conclave of models. Each tool serves a specific function, such as configuration, estimation, querying at different stages, and model selection, without redundancy or unnecessary complexity.
The tool set provides complete coverage for the conclave domain, including configuration viewing, cost estimation, querying at all stages (quick, ranked, full), model listing, custom selection, and resetting. There are no obvious gaps; agents can perform the full lifecycle from setup to querying and cleanup.
Maintenance
Related MCP Connectors
Multi-LLM council: 25+ frontier models in parallel, consensus scoring, verdict-first code review.
Convene a panel of expert AI personas to debate any decision from every side.
Consensus 'best X for Y' rankings from ChatGPT, Claude, Gemini & Grok. Search, verdicts, history.
A second opinion for AI agents: one prompt across several live Gonka models + roles, one call.
Related MCP Servers
- AlicenseAqualityDmaintenanceEnables AI assistants to create council hearings where multiple LLMs (Claude, GPT, Gemini, Grok) debate topics and synthesize verdicts with trust scores and diverse perspectives.25 npmApache 2.0
- AlicenseAqualityDmaintenanceMulti-AI Consensus Tool: Query multiple AI models in parallel, synthesize responses for better accuracy, and reduce AI bias through ensemble decision-making.131MIT
- AlicenseAqualityAmaintenanceEnables Claude to chat with various AI models and obtain multi-model consensus for complex decisions.3585 npm2MIT
- AlicenseAqualityDmaintenanceEnables querying multiple AI models in parallel (Claude, Gemini, O3) and synthesizing their responses using anonymous analysis to reduce bias, providing a comprehensive answer.145 npm193MIT