@mhdd_24/ai-benchmark-mcp
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@@mhdd_24/ai-benchmark-mcp List all available AI benchmarks"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
@mhdd_24/ai-benchmark-mcp
MCP server for Run standardized model benchmarks.
Same architecture as @mhdd_24/sublime-mcp.
Full documentation: docs/WIKI.md
How it works (30 seconds)
You (chat) → MCP client → ai-benchmark-mcp → AI Benchmark APIs / CLIs / local toolsRelated MCP server: metrillm-mcp
Prerequisites
Requirement | Notes |
Node.js 18+ | ESM TypeScript MCP server |
Credentials / CLIs | See environment variables below |
Install
Option A — npm (after publish)
npm install -g @mhdd_24/ai-benchmark-mcpOption B — npx
npx @mhdd_24/ai-benchmark-mcpOption C — clone and build
git clone https://github.com/Mhdd-24/AI-Benchmark-MCP.git
cd AI-Benchmark-MCP
npm install
npm run build
node dist/index.jsConfigure Cursor
Edit ~/.cursor/mcp.json:
{
"mcpServers": {
"aibench": {
"command": "npx",
"args": ["-y", "@mhdd_24/ai-benchmark-mcp"],
"env": {
"_": "optional"
}
}
}
}Local development:
{
"command": "node",
"args": ["/absolute/path/to/AI-Benchmark-MCP/dist/index.js"]
}Environment variables
Variable | Description |
— | No required env |
Tools
Tool | Description |
| Show run standardized model benchmarks configuration / health. |
| List run standardized model benchmarks entities. |
| Inspect a run standardized model benchmarks artifact. |
| Compare two run standardized model benchmarks items. |
License
ISC
Available Tools
4 toolsaibench_compareC
Compare two run standardized model benchmarks items.
Inputs: left (required), right (required).
aibench_inspectC
Inspect a run standardized model benchmarks artifact.
Inputs: name (required).
aibench_listC
List run standardized model benchmarks entities.
Inputs: limit, query.
aibench_statusC
Show run standardized model benchmarks configuration / health.
Inputs: no named parameters.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 4 tools
The four tools are mostly distinct: status shows configuration/health, list enumerates entities, inspect examines a single artifact, and compare contrasts two items. There is slight potential confusion between 'inspect' and 'compare' since both involve examining artifacts, but their purposes are clear enough.
All tools share the 'aibench_' prefix and use simple verb-based names (status, list, inspect, compare). This is consistent, though the prefix is a bit verbose and 'aibench_status' is more of a noun than a verb_noun pattern.
Four tools is a reasonable, focused set for a benchmark server. It covers the core operations without being bloated, though it is on the smaller side.
The set covers status, listing, inspection, and comparison, which are the main read-only operations for benchmarks. However, there are no tools to run, create, or delete benchmarks, so the surface is incomplete for a full lifecycle.
Maintenance
Related MCP Connectors
Benchmark-first release surface with a read-only MCP endpoint and operator CLI.
Governed data discovery, exact queries, decisions, simulations, and runtime utilities over MCP.
Render, verify, describe, and safely edit Mermaid diagrams through MCP.
One MCP server for all your MCP servers: code mode, per-tool permissions, approvals and audit.
Related MCP Servers
AlicenseNot gradedqualityDmaintenanceEnables running LLM evaluations, experiments, and custom evaluators through a standardized MCP interface.16Apache 2.0- AlicenseNot gradedqualityDmaintenanceEnables benchmarking of local LLM models (performance and quality) and sharing results to a public leaderboard via MCP tools.16 npm8Apache 2.0
- FlicenseNot gradedqualityBmaintenanceEnables running deterministic multi-model prompt regression and golden dataset benchmark suites, exposing structured results and telemetry through the Model Context Protocol for integration with MCP-compliant clients.8-
- AlicenseNot gradedqualityBmaintenanceEnables users to list and inspect benchmark tasks, grade answers, run benchmark evaluations, and retrieve provenance through its MCP interface.27 npmMIT