ModelPick
Provides setup integration for GitHub Copilot CLI, writing MCP configuration, skills, and AGENTS.md setup files.
Provides setup integration for OpenAI Codex CLI, writing AGENTS.md, agent skills, and an MCP add hint.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
Your agent can write code, call tools, and ship a feature. Then it picks one model for everything — a frontier model to classify a support ticket, to generate a barrel prop, to draft a thumbnail.
The result is predictable: 60–95% of the spend goes to work that did not need a frontier model, and the hard nodes still fail because the model was chosen for the wrong modality.
ModelPick is a decision layer, not another gateway. Before each node, it answers three questions in one call:
Which model? Ranked for this exact task, with hard capability filters (context, text-in-image, rigging, clip length, open weights).
What will it really cost? Cost per solved task — list price × expected attempts, not $/M tokens.
How do I connect it? The env var, key URL, docs, and API model id.
npx -y github:Leizhidong-creator/modelpick pick "create a stylized 3D sword for a Unity game"Why this exists
Three things are broken in model selection today:
1. Token prices hide the real bill. A model that is 10× cheaper per token but needs 1.8× the attempts and 2× the output tokens is not 10× cheaper. ModelPick computes the denominator that matters: dollars per usable output.
2. Coding agents are blind to creative nodes. A 3D pipeline is not a text problem. Text-in-image is not image generation. A 30-second film is not a 5-second clip. ModelPick models 20 task nodes across 5 modalities, with capability filters that reject the wrong model before scoring.
3. Every tool is a new integration. ModelPick ships as a zero-dependency CLI, a stdio MCP server, a portable SKILL.md, and a Claude Code plugin. Same decision engine everywhere; no gateway, no lock-in, no key storage.
The wedge: other tools route text prompts. ModelPick routes work — and prices it per solved task across every modality.
How is this different from a gateway or a coding router?
Gateway (OpenRouter, LiteLLM, Portkey) | Coding router (claude-code-router) | ModelPick | |
Optimizes | access, failover, provider margin | model switching inside a coding loop | best price-performance at each node |
Modalities | mostly text; OpenRouter's live catalog is ~4% non-text | text | text, image, video, 3D, audio |
Decision | route by price/availability | rules or a classifier | capability filters + benchmark-weighted utility |
Keys | held by the gateway | local | never stored; you call the provider |
Evidence | live prices | config | dated catalog + public benchmark + raw failures |
Gateways are useful, and ModelPick can recommend one as the setup path. But a gateway cannot tell you that a 3D node should use Tripo instead of Meshy, or that a "headline text" image needs a model with a text-rendering score above 90. That is the decision layer this project owns.
More: research/competitive-landscape.md (15 products, 10 features, 53 sources).
Related MCP server: AI Model Advisor MCP Server
30-second quickstart
# 1. Ask for a recommendation (no key, no signup, no network)
npx -y github:Leizhidong-creator/modelpick pick "generate a product photo with the headline 'Fresh Roast'"
# 2. Plan a multi-step deliverable
npx -y github:Leizhidong-creator/modelpick plan "30 second product launch video"
# 3. Install into your agent (writes MCP config + skill)
npx -y github:Leizhidong-creator/modelpick setup claude-codeOr clone it (recommended while the npm package is pending):
git clone https://github.com/Leizhidong-creator/modelpick
cd modelpick
node bin/modelpick.js pick "refactor this 40k-line repo, cheapest that works"
node bin/modelpick.js setupWorks with the agents you already use
modelpick setup detects your tools and writes the correct files — idempotently, with backups, and without touching sibling config.
Agent | One command | What it writes |
Claude Code |
|
|
OpenAI Codex CLI |
|
|
Cursor |
|
|
DeepSeek Harness |
|
|
GitHub Copilot CLI |
|
|
Any MCP client |
| stdio JSON-RPC; no keys required for recommendations |
Claude Code plugin (one command inside Claude):
/plugin marketplace add Leizhidong-creator/modelpick
/plugin install modelpick@modelpickPlatform-agnostic skill: .agents/skills/modelpick/SKILL.md is read by Codex, Cursor, Copilot CLI, Devin/Windsurf, and DeepSeek Harness. Claude Code gets its own copy.
MCP tools exposed: pick_model, plan_workflow, compare_models, get_setup, list_models, explain_task.
The number that matters: cost per solved task
raw cost per attempt = tokens × price (cache-aware) OR units × unit price
expected attempts = 1 / (1 − 1.15 × (100 − quality) / 100) capped at 4
cost per solved task = raw cost × expected attemptsWorked example — 80K input / 8K output, 60% cache hit, agentic coding:
Model | quality | raw cost | expected attempts | cost per solved task |
Claude Fable 5.2 | 92.3 | $0.59 | 1.10 | $0.64 |
Gemini 4 Carbon | 87.3 | $0.15 | 1.17 | $0.17 |
DeepSeek V4.1 Flash | 70.7 | $0.010 | 1.51 | $0.015 |
Claude Haiku 5.5 | 68.8 | $0.008 | 1.56 | $0.012 |
The cheap model is 53× cheaper and 23.5 quality points worse. The interesting answer is the third row: mid-quality agentic coding at 2.3% of the frontier cost. The right answer depends on the node — which is why the tool shows both sides instead of hiding the trade-off behind a token price.
Quality is a weighted mix of public benchmark signals (BenchLM overall/coding, CursorBench 4.0, capability-specific scores). Expected attempts is a transparent model, shown on every result — not a promise. If a number is estimated, the CLI prints ~.
What it covers
41 models · 18 providers · 20 task nodes · 5 modalities. Current as of 2026-11-19; every price carries a source and read date.
Modality | Nodes | Example models in the catalog |
Text | coding, review, refactor, debug, plan, long-context, research, writing, translation, extraction, classification, data, computer-use | Claude Fable 5.2, GPT-6.2 Astra/Sol, Gemini 4 Carbon/Hydrogen, DeepSeek V4.1 Flash, Claude Haiku 5.5, Kimi K3.1 Pro |
Image | generation, editing, text-in-image | Nano Banana 2.1/Pro, GPT Image 2.6, Seedream 5.5 |
Video | text/image → clip, dialogue, 30s takes | Veo 3.1 / Fast (Vertex), Kling 4.0, Seedance 2.5 |
3D | text/image → game-ready asset, rigging | Tripo H3.1, Hunyuan3D 3.1 Pro, Meshy 7.1, TRELLIS 2 (open) |
Audio | TTS, STT, music | Eleven v4, Voxtral TTS 3, GPT Transcribe, Voxtral Transcribe 3, Lyria 3.5, MiniMax Music 3 |
The catalog is a dated snapshot, not a live price feed. modelpick update refreshes from the repo; modelpick doctor warns when it is stale.
Benchmark, not vibes
Metric | Development set | Held-out set | Target |
Top-1 acceptable | 100% (115 runs) | 100% (35 runs) | ≥75% |
Top-3 acceptable | 100% | 100% | ≥92% |
Intent accuracy | 100% (39 scenarios) | 100% (12 scenarios) | ≥95% |
Hard constraint pass | 100% | 100% | 100% |
Price stability (±20%) | 100% | 100% | ≥80% |
p95 decision latency | 0.5 ms | 0.5 ms | ≤300 ms |
What this is: 39 expert-labeled scenarios expanded into balanced/cost/quality runs, plus a 12-scenario held-out set authored after the benchmark froze. Every run checks task detection, top-1/top-3 acceptability, hard constraints, cost regret, and stability under ±20% price perturbation.
What this is not: a live model-output benchmark. It does not call providers, and it does not prove quality labels are universally correct. Quality scores are derived from public leaderboards; expected attempts is a model. The full methodology, failures, and reproduction commands live in tests/TEST_PLAN.md, tests/results/, and bench/.
Full per-node tables: docs/LEADERBOARD.md.
Value, measured
node bench/value.js simulates 4,000 jobs per task across five task types, using catalog quality as first-pass success and the router's retry curve:
Strategy | First-pass | Cost / solved | Quality parity | Cost vs frontier |
Frontier always | 93.7% | $0.6417 | 100% | 100% |
Value pick | 91.7% | $0.2708 | 97.9% | 58.4% |
Cheapest always | 83.3% | $0.1736 | 88.8% | 34.1% |
The value pick keeps 97.9% of frontier first-pass quality at 58% of the cost. The cheapest model saves another 24 cost points but loses 11 quality points and fails a 90%-parity bar on two of five tasks. This is synthetic; the arithmetic and the definition are in docs/VALUE.md, and the evidence behind every quality score is in docs/QUALITY.md.
The decision engine
request
│
├─ task detection (20 nodes, English + Chinese) ── explicit --task always wins
├─ intent parsing (cheap / fast / best / open-source / budget / exclude)
├─ hard requirements (context, text-in-image, rigging, lip-sync, clip length)
│
├─ capability filter → candidates
├─ benchmark-weighted quality
├─ cache-aware cost × expected attempts
├─ latency + context fit
└─ weighted utility → pick + alternatives + why + trade-offs + setupPresets: balanced (task weights), value (price-performance), quality, cost, speed, privacy. Budgets and quality floors are hard constraints, not suggestions.
The default is a value decision, not a bargain hunt. The value pick is the cheapest model within 6 quality points of the best eligible model — and never worse than the recommendation itself. It is returned as picks.value and value, with the quality gap and the percentage of the best model's cost. The full definition, simulation, and sensitivity analysis are in docs/VALUE.md.
Every answer includes:
recommendation.slug,quality,costPerTask,expectedAttempts,latencySecwhy[]andtradeoffs[]setup.direct(env, key URL, docs, API model) and an OpenRouter fallback for text modelscatalog.version,stale,priceConfidence
Commands
modelpick pick "<task>" Rank models for one node
modelpick plan "<goal>" Multi-step workflow, one model per step
modelpick compare <a> <b> --task Side-by-side cost per solved task
modelpick models Browse the catalog
modelpick tasks List the 20 nodes
modelpick show <model> Endpoint, auth, SDK, curl, and evidence
modelpick add-model <file> Add a private/custom model
modelpick setup [agent] Install MCP + skill
modelpick mcp Run the stdio MCP server
modelpick keys Show required env vars (never values)
modelpick doctor Diagnose install, keys, freshness
modelpick update Refresh the catalog snapshot
modelpick bench Run the decision benchmarkUseful flags: --optimize, --max-cost, --min-quality, --open-weights, --context, --exclude, --task, --json, --explain, --no-color.
# Machine-readable for any agent
modelpick pick "edit this product photo, remove the background" --json
# Quality floor + budget
modelpick pick "implement OAuth" --min-quality 85 --max-cost 0.25
# Open weights only
modelpick pick "refactor this repo" --open-weights
# Only models whose key is already in this shell
modelpick pick "refactor this repo" --available
# Need 400K tokens of context
modelpick pick "read this 400-page document" --context 400000Only have a few API keys?
You do not need every provider. ModelPick separates the best decision from what you can run right now:
modelpick pick "refactor this repo"! key missing: GEMINI_API_KEY, GOOGLE_API_KEY — recommendation is still valid
run now GPT-6.2 Sol · $0.20 · quality 83.9 · key OPENAI_API_KEY--availablerestricts ranking to models whose key exists (or any text model if you haveOPENROUTER_API_KEY).The JSON always includes
availability.bestAvailable,availability.fallbacks, andrecommendation.availability.missing.An
OPENROUTER_API_KEYis a universal fallback for text models;FAL_KEYcovers many media models.If a capability has no accessible model, the tool says so instead of inventing a recommendation.
Add your own model
Bring a private endpoint, a self-hosted vLLM/Ollama model, or a provider the catalog does not cover:
modelpick add-model --example > my-model.json
# edit the slug, provider env var, pricing, and quality scores
modelpick add-model my-model.json
modelpick pick "refactor this repo" # it is now a candidate
modelpick add-model --remove my-modelThe model is written to ~/.modelpick/overrides.json, validated against the full catalog before saving, and never touches the repo. You can also add a whole list with { "models": [...] }.
Connect without guessing the endpoint
pick gives you the env var and key URL. show gives you the full connection card:
modelpick show tripo-h3.1CONNECT
endpoint https://api.tripo3d.ai/v2/openapi
model id h3.1
auth bearer · env TRIPO_API_KEY
platform https://platform.tripo3d.ai
docs https://developers.tripo3d.ai/en/docs
sdk REST / MCP
example
curl -X POST "https://api.tripo3d.ai/v2/openapi/task" \
-H "Authorization: Bearer $TRIPO_API_KEY" \
-H "content-type: application/json" \
-d '{"type":"text_to_model","prompt":"a stylized sword"}'The same object is returned by the MCP get_setup tool. If an endpoint is a provider's common path rather than a freshly verified one, the card says so (typical vs verified).
Built to change
Model prices move weekly. New providers appear monthly. The engine is deliberately data-first:
Layer | File | Change without code |
Models, prices, IDs |
|
|
Task nodes, weights, floors, anchors |
| edit JSON, run |
Retry curve, presets, anchors |
| edit JSON |
Your private models / price overrides |
| add a full model object or patch a slug |
Workflows |
| add steps referencing task ids |
Agent adapters |
| one function per agent |
Custom model example:
{
"models": [
{
"slug": "my-private-model",
"name": "My Private Model",
"provider": { "id": "internal", "name": "Internal", "env": ["INTERNAL_KEY"], "keysUrl": "https://…", "docsUrl": "https://…" },
"modality": "text",
"contextWindow": 200000,
"pricing": { "unit": "mtok", "input": 1, "output": 2, "currency": "USD" },
"quality": { "overall": 88, "coding": 88, "codingAgent": 88, "reasoning": 88, "longContext": 80, "writing": 80, "multimodal": 60, "cheapWork": 70 },
"speed": { "outputTps": 50, "ttftMs": 1000 },
"tags": ["benchmarked"],
"openWeights": false,
"sources": [{ "name": "internal", "url": "https://…", "readAt": "2026-11-19" }]
}
],
"exclude": ["model-you-never-want"],
"policy": { "presets": { "cost": { "quality": 0.35, "cost": 0.65, "speed": 0, "context": 0 } } }
}No fork. No rebuild. modelpick doctor validates the result.
Data, honesty, and limits
Prices: models.dev / OpenRouter for text; BenchLM media pricing, Google Vertex, OpenAI, fal, Tripo, Meshy, ElevenLabs for media. Read date on every source.
Quality: BenchLM BenchAlign and CursorBench 4.0 for text; vendor specs plus public comparisons for media. Media quality is the least independently verified part of the catalog; it is labeled and dated.
Known omissions: Veo is Vertex-only now; Sora 2 is discontinued; GPT-Image-1 and Imagen 4 are retired. These are deliberately not in the catalog.
No telemetry. No key storage. Recommendations work offline.
keysreads presence only.Not a gateway. ModelPick does not proxy requests, mark up tokens, or hold your data. You call the provider directly.
See docs/DATA.md for provenance and research/ for the underlying research.
Development
node scripts/validate.js # catalog + policy + manifests
node --test "tests/**/*.test.js"
node bench/run.js # decision benchmark -> tests/results/
node bench/holdout.js # held-out decision benchmark
node bench/value.js # cost-per-solved-task simulation
node tests/adversarial/run.js # 23 adversarial cases
node scripts/check.js # all of the above
node scripts/make_assets.py # regenerate README/test imagesRequirements: Node ≥ 20.11. Runtime dependencies: zero. Python + Pillow are only needed to regenerate images.
Roadmap
Publish to npm and the MCP registry (
npx modelpick)Live price refresh from provider APIs with signed snapshots
Community-contributed task nodes and quality evidence
Router mode: optionally execute the recommended call through the user's own key
Cost ledger: track actual spend per node across a project
Windows-native MCP shim (no
cmd /c)
If ModelPick saves you money, star the repo — it is the only distribution channel for a tool with no VC and no telemetry.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
AI video, image and audio generation with 140+ AI models from any MCP agent
214 tools for AI agents over one MCP endpoint: image, video, audio, 3D, data, storage, mail, GPUs.
Pay-per-use tool marketplace for AI agents. Search, price-check, and call APIs via MCP.
Free public MCP for AI agents — 193 tools, 44 workflows. No API key.
Related MCP Servers
- AlicenseBqualityFmaintenanceEnables AI assistants to intelligently select and switch between different AI models (OpenAI, Anthropic, etc.) within the same conversation based on task requirements. Provides a unified interface for accessing multiple AI providers through a single MCP tool.1116 npmMIT
- AlicenseAqualityDmaintenanceEnables AI agents to discover, compare, and select the best AI models across multiple providers based on pricing, performance, and capabilities, with real-time cost estimation and benchmarking.91,281 npm1MIT
- AlicenseNot gradedqualityDmaintenanceEnables multi-model leader-worker agent orchestration, workflow execution, and deterministic validation via structured MCP tools.8 npmApache 2.0
- AlicenseBqualityCmaintenanceEnables AI agents to consume 50+ NVIDIA NIM models including LLMs, image generation, and vision through 10 MCP tools with rich metadata and advanced filtering.11226 npm1MIT