Patronus MCP Serverofficial
A
licenseNot graded
qualityD
maintenanceEnables running LLM evaluations, experiments, and custom evaluators through a standardized MCP interface.
16
Apache 2.0
NM Hard Tools
Officialby neuralmagic
NM Hard Tools
可部署、无状态的 MCP 服务，用于严格的推理评估和基准测试。本仓库拥有服务实现、容器镜像和 Helm charts。
包含的服务：
一个带性能分析的
lm-evaluation-harness控制器和工作节点；一个用于 vLLM forward-pass 基准测试服务的有界 MCP 适配器；
用于 lm-eval、forward-pass、Nyann 和 AgentX 的部署图表。
开发
uv run --extra test pytest
uv run --extra test ruff format --check src tests
uv run --extra test ruff check src tests仓库治理存储在 gov/ 中，并通过 govctl 进行验证：
cargo install govctl --version 0.19.1 --locked
govctl status
govctl checklm-eval 镜像必须使用显式的 Python 基础镜像构建：
podman build -f Dockerfile.lm-eval-service \
--build-arg BASE_IMAGE=python:3.12-slim -t inference-hard-lm-eval:dev .Related MCP server: ArmBench MCP Server
许可证
根据 Apache License 2.0 许可。参见 LICENSE。
This server cannot be deployed
Maintenance
ActivitySlowing
ResponsivenessNo issues
Related MCP Connectors
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
Pay-per-call AI evaluation MCP server. Score LLM outputs against benchmark rubrics via Workers AI.
Hosted MCP server for LLM cost estimation, model comparison, and budget-aware routing.
Benchmark-first release surface with a read-only MCP endpoint and operator CLI.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables benchmarking and inference of LLMs on Arm64 cloud instances with KleidiAI optimizations, providing an MCP-compatible API for serving results.1MIT
Local AI MCPofficial
AlicenseAqualityCmaintenanceUnified MCP server for managing local model runtimes (Ollama, LM Studio, etc.), enabling provider-agnostic discovery, lifecycle management, hardware-fit checks, and delegated inference.1617 npmCreative Commons Attribution Non Commercial No Derivatives 4.0 International- AlicenseNot gradedqualityBmaintenanceEnables benchmarking of local LLM models (performance and quality) and sharing results to a public leaderboard via MCP tools.4 npm8Apache 2.0