Skip to main content
Glama

claude-eval-mcp

License: MIT Protocol: MCP Python 3.10+

An official Model Context Protocol (MCP) server engineered to connect Claude (Claude Desktop, Claude Code CLI, and autonomous agents) directly to AI/ML evaluation harnesses, token latency profilers, and prompt security red-teaming tools.


Sequence Architecture

sequenceDiagram
    autonumber
    actor User as Engineer
    participant Claude as Claude Desktop / Claude Code
    participant MCP as claude-eval-mcp Server
    participant Engine as Local Evaluation Engine

    User->>Claude: "Evaluate Qwen-2.5-7B on GSM8K and profile TTFT"
    Claude->>MCP: Call tool `benchmark_model(model='qwen-2.5-7b', task='gsm8k')`
    MCP->>Engine: Run reasoning suite
    Engine-->>MCP: Accuracy 82.4%
    MCP-->>Claude: JSON Tool Result
    Claude-->>User: Format detailed benchmark summary table

Related MCP server: Claude Code MCP Bridge

Claude Desktop Configuration

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "ai-evaluator": {
      "command": "python3",
      "args": ["-m", "claude_eval.server"]
    }
  }
}

License

MIT License (c) 2024 Soujanya Gupta.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      Not graded
      quality
      D
      maintenance
      Enables Claude or any MCP-compatible client to execute cybersecurity workflows like incident response, cloud audits, and pentests by calling tools with 323 prompts across 8 categories.
      1
      -