Patronus MCP Serverofficial
A
licenseNot graded
qualityD
maintenanceEnables running LLM evaluations, experiments, and custom evaluators through a standardized MCP interface.
16
Apache 2.0
NM Hard Tools
Officialby neuralmagic
NM Hard Tools
Bereitstellbare, zustandslose MCP-Dienste für rigorose Inferenzbewertung und Benchmarking. Dieses Repository enthält die Dienstimplementierungen, Container-Images und Helm-Charts.
Enthaltene Dienste:
ein profilierter
lm-evaluation-harness-Controller und -Worker;ein begrenzter MCP-Adapter für den vLLM-Forward-Pass-Benchmark-Dienst;
Bereitstellungs-Charts für lm-eval, forward-pass, Nyann und AgentX.
Entwicklung
uv run --extra test pytest
uv run --extra test ruff format --check src tests
uv run --extra test ruff check src testsDie Repository-Governance ist in gov/ gespeichert und wird mit govctl validiert:
cargo install govctl --version 0.19.1 --locked
govctl status
govctl checkDas lm-eval-Image muss mit einem expliziten Python-Basis-Image erstellt werden:
podman build -f Dockerfile.lm-eval-service \
--build-arg BASE_IMAGE=python:3.12-slim -t inference-hard-lm-eval:dev .Related MCP server: ArmBench MCP Server
Lizenz
Lizenziert unter der Apache License, Version 2.0. Siehe LICENSE.
This server cannot be deployed
Maintenance
ActivitySlowing
ResponsivenessNo issues
Related MCP Connectors
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
Pay-per-call AI evaluation MCP server. Score LLM outputs against benchmark rubrics via Workers AI.
Hosted MCP server for LLM cost estimation, model comparison, and budget-aware routing.
Benchmark-first release surface with a read-only MCP endpoint and operator CLI.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables benchmarking and inference of LLMs on Arm64 cloud instances with KleidiAI optimizations, providing an MCP-compatible API for serving results.1MIT
Local AI MCPofficial
AlicenseAqualityCmaintenanceUnified MCP server for managing local model runtimes (Ollama, LM Studio, etc.), enabling provider-agnostic discovery, lifecycle management, hardware-fit checks, and delegated inference.1617 npmCreative Commons Attribution Non Commercial No Derivatives 4.0 International- AlicenseNot gradedqualityBmaintenanceEnables benchmarking of local LLM models (performance and quality) and sharing results to a public leaderboard via MCP tools.4 npm8Apache 2.0