mcp-tournament
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| OPENROUTER_API_KEY | Yes | API key for OpenRouter. Required for all roles by default. | |
| TOURNAMENT_RESULTS_DIR | No | Directory for results output. Defaults to './results'. | |
| TOURNAMENT_MODEL_JUDGE_RULES | No | Model override for the rules judge role (e.g., 'openai/gpt-5.4-mini'). | |
| TOURNAMENT_MODEL_PARTICIPANT | No | Model override for the participant agent role (e.g., 'deepseek/deepseek-v3.2'). | |
| TOURNAMENT_MODEL_SYNTHESIZER | No | Model override for the synthesizer role (e.g., 'moonshotai/kimi-k2.5'). |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
| prompts | {
"listChanged": true
} |
| resources | {
"listChanged": true
} |
| completions | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| tournament_optionsA | Free discovery. Lists benches, scenarios, model tiers with prices per million tokens, judge personas, provider setup, defaults, and limits. Fetches the model catalog but makes no paid model calls. |
| tournament_plan_runA | Free preview. Checks your choices against the model catalog, fills in defaults, and shows the plan, rough cost, and any provider setup still needed. Makes no model calls and creates no run folder. Show the preview and get the user's yes before tournament_evaluate. |
| tournament_list_benchesA | Free local read (no API calls, no network). Lists every registered evaluation bench (plugin) and its scenarios, with IDs and descriptions. Call this first to discover valid |
| tournament_leaderboardA | Free local read (no API calls, no network). Reads the best score per model across all saved tournament runs on disk, sorted highest first. Optionally filter to one bench with |
| tournament_get_runA | Free local read (no API calls, no network). Reads the full detail of one saved tournament run by ID: plugin, models, judges, scenarios, per-model per-scenario scores, and any failures. Use tournament_leaderboard or the tournament://runs resource to find run IDs. |
| tournament_quick_testA | The cheap check: one model, one scenario, one judge. Makes real, paid model calls and writes a result to disk. OpenRouter is the default. An anthropic: ref bills your Anthropic API key. Optional judge picks the judge model and persona; turns sets 1-10 turns. No confirm form. Call tournament_options if you need bench or scenario IDs. |
| tournament_evaluateA | Costs money. Makes paid model calls and writes results to disk. Call tournament_plan_run first and get the user's yes in chat. Clients that support forms will also get a confirm form. Runs 1-4 models across a bench's scenarios and saves a leaderboard. Can take several minutes. judgePanel picks judge models and personas per seat and overrides judges and judgeModels. turns sets one turn count for every scenario. participantModel picks the simulated user model. synthesizerModel reconciles judge scores. OpenRouter is the default; anthropic: refs bill the user's Anthropic API key. Use tournament_quick_test for a cheap check. |
| tournament_create_benchA | Free, local, no model calls. Saves a new reusable bench (a named set of scenarios and scoring criteria) to disk. Once saved, it is immediately usable by tournament_evaluate and tournament_quick_test as |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
| setup_tournament | Guides the user through options, missing choices, a free preview, and confirmation before a paid run. |
| compare_models | Runs a fresh tournament comparing the given models on a bench and explains the result. This calls tournament_evaluate, which makes real API calls to every candidate model and every judge, so it costs money and takes a few minutes. Use choose_model_for_task instead if an existing leaderboard would already answer the question. |
| choose_model_for_task | Recommends a model for a plain-language task description, using existing results first. Free unless it decides a fresh run is warranted (it will ask before running one, since that costs money). Use this before compare_models when the user has not already named the specific models to compare. |
| explain_run | Embeds a run's markdown report and asks for a plain-English explanation aimed at a non-technical reader. Free, local, no API calls (the run must already exist). Use this when someone wants an existing run explained rather than a new one created. |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
| benches | Every registered evaluation bench (plugin) and its scenarios, with plain-language descriptions. Free, local, no API calls. Read this first when deciding which bench fits a task, or before calling tournament_evaluate to see valid scenario IDs. |
| leaderboard | Best score ever recorded per model, across every run in the results directory (one row per model, its single highest overallAverage). Free, local, no API calls. Use this for "which model has historically done best" questions; use tournament://runs/{runId} for a single run's detail instead. |
| runs | Every tournament run on disk, newest first, as a lightweight index: run ID, plugin, creation time, candidate model IDs, and the winning model name (null if the run was interrupted or has no leaderboard yet). Free, local, no API calls. Use this to find a runId to pass to tournament://runs/{runId} or tournament://runs/{runId}/report. |
TDQS
Scored across 8 tools
Most tools have clearly distinct roles (quick_test vs evaluate by cost/scale, leaderboard vs get_run by aggregate vs single). However, tournament_list_benches and tournament_options overlap on bench/scenario discovery, and plan_run vs evaluate require reading to distinguish preview from execution.
Names follow a consistent tournament_ prefix with snake_case. Most use verb_noun (list_benches, plan_run, get_run, create_bench), but tournament_options and tournament_leaderboard are noun-only, a minor deviation from the pattern.
Eight tools is well-scoped for a tournament/evaluation server. Each tool earns its place: discovery, planning, cheap vs full execution, results reading, and bench creation are all distinct responsibilities.
The evaluation lifecycle is well covered: discover, plan, execute (cheap and full), read results, and create custom benches, with paid-call safeguards. Minor gaps exist around managing saved artifacts (update/delete benches or runs) and cancelling long-running evaluations.