Patronus MCP Serverofficial
A
licenseNot graded
qualityD
maintenanceEnables running LLM evaluations, experiments, and custom evaluators through a standardized MCP interface.
16
Apache 2.0
NM Hard Tools
Officialby neuralmagic
NM Hard Tools
Servicios MCP desplegables y sin estado para una evaluación rigurosa de inferencia y benchmarking. Este repositorio contiene las implementaciones de los servicios, las imágenes de contenedor y los gráficos de Helm.
Servicios incluidos:
un controlador y trabajador de
lm-evaluation-harnessperfilado;un adaptador MCP acotado para el servicio de benchmark de forward-pass de vLLM;
gráficos de despliegue para lm-eval, forward-pass, Nyann y AgentX.
Desarrollo
uv run --extra test pytest
uv run --extra test ruff format --check src tests
uv run --extra test ruff check src testsLa gobernanza del repositorio se almacena en gov/ y se valida con govctl:
cargo install govctl --version 0.19.1 --locked
govctl status
govctl checkLa imagen de lm-eval debe construirse con una imagen base de Python explícita:
podman build -f Dockerfile.lm-eval-service \
--build-arg BASE_IMAGE=python:3.12-slim -t inference-hard-lm-eval:dev .Related MCP server: ArmBench MCP Server
Licencia
Licenciado bajo la Licencia Apache, versión 2.0. Consulte LICENSE.
This server cannot be deployed
Maintenance
ActivitySlowing
ResponsivenessNo issues
Related MCP Connectors
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
Pay-per-call AI evaluation MCP server. Score LLM outputs against benchmark rubrics via Workers AI.
Hosted MCP server for LLM cost estimation, model comparison, and budget-aware routing.
Benchmark-first release surface with a read-only MCP endpoint and operator CLI.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables benchmarking and inference of LLMs on Arm64 cloud instances with KleidiAI optimizations, providing an MCP-compatible API for serving results.1MIT
Local AI MCPofficial
AlicenseAqualityCmaintenanceUnified MCP server for managing local model runtimes (Ollama, LM Studio, etc.), enabling provider-agnostic discovery, lifecycle management, hardware-fit checks, and delegated inference.1617 npmCreative Commons Attribution Non Commercial No Derivatives 4.0 International- AlicenseNot gradedqualityBmaintenanceEnables benchmarking of local LLM models (performance and quality) and sharing results to a public leaderboard via MCP tools.4 npm8Apache 2.0