claude-eval-mcp
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
claude-eval-mcp
An official Model Context Protocol (MCP) server engineered to connect Claude (Claude Desktop, Claude Code CLI, and autonomous agents) directly to AI/ML evaluation harnesses, token latency profilers, and prompt security red-teaming tools.
Sequence Architecture
sequenceDiagram
autonumber
actor User as Engineer
participant Claude as Claude Desktop / Claude Code
participant MCP as claude-eval-mcp Server
participant Engine as Local Evaluation Engine
User->>Claude: "Evaluate Qwen-2.5-7B on GSM8K and profile TTFT"
Claude->>MCP: Call tool `benchmark_model(model='qwen-2.5-7b', task='gsm8k')`
MCP->>Engine: Run reasoning suite
Engine-->>MCP: Accuracy 82.4%
MCP-->>Claude: JSON Tool Result
Claude-->>User: Format detailed benchmark summary tableRelated MCP server: Claude Code MCP Bridge
Claude Desktop Configuration
Add to your claude_desktop_config.json:
{
"mcpServers": {
"ai-evaluator": {
"command": "python3",
"args": ["-m", "claude_eval.server"]
}
}
}License
MIT License (c) 2024 Soujanya Gupta.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
One MCP endpoint for Claude, GPT & Gemini: 100+ tools + no-code connectors + agent workers.
Find, vet, and run MCP tools through a secure audited gateway with prompt-injection risk scoring
AI Visibility and Content Intelligence tools for Claude and MCP-compatible agents.
Connect ChatGPT and Claude to your Mac, Windows or Linux with secure MCP remote access.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables Claude Code to benchmark llm-d from plain English, providing 35 tools and 5 workflow prompts for probing clusters, planning, deploying, running benchmarks, and analyzing results via the Model Context Protocol.Apache 2.0
- FlicenseNot gradedqualityDmaintenanceExposes Claude Code's file editing, command execution, and test running capabilities as composable MCP tools for any MCP-compatible host, enabling code operations via a stateless bridge.-
- FlicenseNot gradedqualityDmaintenanceEnables Claude or any MCP-compatible client to execute cybersecurity workflows like incident response, cloud audits, and pentests by calling tools with 323 prompts across 8 categories.1-
- FlicenseNot gradedqualityCmaintenanceEnables Claude AI to access and run over 150 integrated cybersecurity tools through a public ngrok tunnel, supporting agentic security workflows via MCP.-