LLMBenchmark.ai MCP Server: LLM Benchmarks and API Prices
Server Details
Which LLM should I use for this task? Ask in plain words and get the models that score best on the matching benchmark board, within your price, context window and open-weights limits. Covers 400+ models from OpenAI, Anthropic, Google, DeepSeek, Qwen, Mistral and more, with benchmark scores (coding, agentic, reasoning, math, knowledge, design) and API list prices per million tokens, updated daily. Tools: recommend_model, get_leaderboard, compare_models, get_model, get_prices, list_categories. Free, no API key, data under CC BY 4.0. Docs: https://llmbenchmark.ai/api
- Status
- Healthy
- Uptime
- 100.0% over the last 24 hours
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-11-25
- URL
- Repository
- Take-Group/llmbenchmark-mcp
- GitHub Stars
- 0
- ChatGPT
- –
- Claude
- –
TDQS
Scored across 6 tools
Each tool targets a distinct read operation: detail (get_model), comparison (compare_models), ranking (get_leaderboard), pricing (get_prices), category metadata (list_categories), and task-based recommendation (recommend_model). The only mild overlap is between get_leaderboard and recommend_model, since both return ranked models, but recommend_model's task-keyword mapping and price/context filters make the boundary fairly clear.
All names are snake_case verb_noun patterns (compare_models, get_leaderboard, get_model, get_prices, list_categories, recommend_model). Verbs map predictably to intent: get for single/primary retrieval, list for enumeration, compare/recommend for composite operations.
Six tools is well-scoped for a benchmark and pricing reference server, with each tool earning its place and no filler. The surface is compact enough for an agent to reason about without being thin.
Core workflows are covered: browse categories, rank models, inspect a model, compare two, check prices, and get recommendations. There is no explicit list/search-all-models tool, so enumerating the catalog relies on get_prices or get_leaderboard filters—a minor gap an agent can work around.
Publisher details
- Operator
- LLMBenchmark.ai
- Operator website
- https://llmbenchmark.ai
- Vendor relationship
- Independent
- Documentation
- https://llmbenchmark.ai/api
- Privacy policy
- Not available
- Trust center
- Not available
- Restrictions
- None. Free, no API key, 500 requests per IP per UTC day.
Available Tools
6 toolscompare_modelsAInspect
Two models side by side: score on every category board where both were measured, prices, context. Names are matched loosely: 'claude opus', 'gpt-5' or a slug all work.
Inputs: model_a (required), model_b (required). Returns: object
get_leaderboardAInspect
The top models on one category's board. category is one of overall, coding, agentic, reasoning, math, knowledge, design (see list_categories). limit is 1 to 50. overall is the LLMBenchmark Index: a weighted mean of the Vibe Coding Index, the agentic, reasoning, math and knowledge indexes, the Artificial Analysis Intelligence Index and the LMArena text Elo, with a 90% margin.
Inputs: limit, category. Returns: object
get_modelAInspect
One model: maker, release date, prices, context, open weights, and its score and rank on every category board. Name or slug.
Inputs: model (required). Returns: object
get_pricesAInspect
API prices (USD per million tokens), cheapest output price first: the maker's own price when it serves the model on OpenRouter, else the median of providers (price_basis says which), plus the cheapest provider. Filter by model name and/or maker; limit is 1 to 100.
Inputs: limit, maker, model. Returns: object
list_categoriesCInspect
The categories recommend_model and get_leaderboard know, the benchmark board behind each, and the task keywords that select it.
Inputs: no named parameters. Returns: object
recommend_modelAInspect
Which model should I use for this task? Maps the task text to a category (coding, agentic, reasoning, math, knowledge, web design, or overall when nothing matches) by a fixed keyword map, filters by output price per million tokens, context length and open weights, and ranks by that category's benchmark board. Deterministic, no LLM call. Returns top_k models with scores, rank, prices, context, page URLs and a one-line reason. overall is the LLMBenchmark Index: a weighted mean of the Vibe Coding Index, the agentic, reasoning, math and knowledge indexes, the Artificial Analysis Intelligence Index and the LMArena text Elo, with a 90% margin.
Inputs: task (required), top_k, open_weights_only, min_context_tokens, +1 more. Returns: object
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Related MCP Connectors
Compare LLM API prices, search models and providers, and access reviewed benchmark results.
Current AI model rankings, API prices, benchmark results and history from BenchLM.
Live prices, rankings, licences and privacy of AI models: best model per task, costs, alternatives.
Benchmark your own prompt on 100+ LLMs with real API calls; pick the cheapest model that passes.
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceEnables users to ask which LLM fits a task and get deterministic ranked recommendations, leaderboards, side-by-side comparisons, API prices, and model details from daily benchmark scores and prices, with no API key required.MIT
- AlicenseNot gradedqualityDmaintenanceEnables AI agents to compare LLM benchmarks, get pricing, and receive model recommendations for tasks and budgets.1MIT
- AlicenseAqualityBmaintenanceWhich AI model to use and what it costs: LiteLLM's price catalogue with every price change since 2023 and Arena ratings by task. Tools to find and compare models, estimate a job's cost and time, show price history and pick local or cloud; no API keys needed.11MIT
- AlicenseAqualityCmaintenanceEnables users to select the optimal LLM for their specific task by aggregating benchmark data and user intent, returning a ranked shortlist with plain-English rationale.5MIT
Glama MCP Gateway
Add one secure layer between your agents and this server.