Skip to main content
Glama
@mhdd_24/ai-benchmark-mcp

@mhdd_24/ai-benchmark-mcp

by Mhdd-24

@mhdd_24/ai-benchmark-mcp

MCP server for Run standardized model benchmarks.

Same architecture as @mhdd_24/sublime-mcp.

Full documentation: docs/WIKI.md


How it works (30 seconds)

You (chat) → MCP client → ai-benchmark-mcp → AI Benchmark APIs / CLIs / local tools

Related MCP server: metrillm-mcp

Prerequisites

Requirement

Notes

Node.js 18+

ESM TypeScript MCP server

Credentials / CLIs

See environment variables below


Install

Option A — npm (after publish)

npm install -g @mhdd_24/ai-benchmark-mcp

Option B — npx

npx @mhdd_24/ai-benchmark-mcp

Option C — clone and build

git clone https://github.com/Mhdd-24/AI-Benchmark-MCP.git
cd AI-Benchmark-MCP
npm install
npm run build
node dist/index.js

Configure Cursor

Edit ~/.cursor/mcp.json:

{
  "mcpServers": {
    "aibench": {
      "command": "npx",
      "args": ["-y", "@mhdd_24/ai-benchmark-mcp"],
      "env": {
        "_": "optional"
      }
    }
  }
}

Local development:

{
  "command": "node",
  "args": ["/absolute/path/to/AI-Benchmark-MCP/dist/index.js"]
}

Environment variables

Variable

Description

—

No required env


Tools

Tool

Description

aibench_status

Show run standardized model benchmarks configuration / health.

aibench_list

List run standardized model benchmarks entities.

aibench_inspect

Inspect a run standardized model benchmarks artifact.

aibench_compare

Compare two run standardized model benchmarks items.


License

ISC

Available Tools

4 tools
aibench_compareC

Compare two run standardized model benchmarks items.

Inputs: left (required), right (required).

aibench_inspectC

Inspect a run standardized model benchmarks artifact.

Inputs: name (required).

aibench_listC

List run standardized model benchmarks entities.

Inputs: limit, query.

aibench_statusC

Show run standardized model benchmarks configuration / health.

Inputs: no named parameters.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    C2.8/5.0

    Scored across 4 tools

    Disambiguation4/5

    The four tools are mostly distinct: status shows configuration/health, list enumerates entities, inspect examines a single artifact, and compare contrasts two items. There is slight potential confusion between 'inspect' and 'compare' since both involve examining artifacts, but their purposes are clear enough.

    Naming Consistency4/5

    All tools share the 'aibench_' prefix and use simple verb-based names (status, list, inspect, compare). This is consistent, though the prefix is a bit verbose and 'aibench_status' is more of a noun than a verb_noun pattern.

    Tool Count4/5

    Four tools is a reasonable, focused set for a benchmark server. It covers the core operations without being bloated, though it is on the smaller side.

    Completeness3/5

    The set covers status, listing, inspection, and comparison, which are the main read-only operations for benchmarks. However, there are no tools to run, create, or delete benchmarks, so the surface is incomplete for a full lifecycle.

    Maintenance

    ActivityStale
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers