Herramienta de evaluación comparativa MCP vLLM

Esta es una prueba de concepto sobre cómo utilizar MCP para evaluar de forma interactiva vLLM.

No somos nuevos en benchmarking, lea nuestro blog:

Esta es solo una exploración de posibilidades con MCP.

Uso

Clonar el repositorio
Agreguelo a sus servidores MCP:

{
    "mcpServers": {
        "mcp-vllm": {
            "command": "uv",
            "args": [
                "run",
                "/Path/TO/mcp-vllm-benchmarking-tool/server.py"
            ]
        }
    }
}

Entonces puedes indicar un ejemplo como este:

Do a vllm benchmark for this endpoint: http://10.0.101.39:8888 
benchmark the following model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B 
run the benchmark 3 times with each 32 num prompts, then compare the results, but ignore the first iteration as that is just a warmup.

Hacer:

Debido a algunas salidas aleatorias de vllm, es posible que se indique que encontró un JSON no válido. Aún no lo he investigado.

This server cannot be installed

security - not tested

license - not found

quality - not tested

remote-capable server

The server can be hosted and run remotely because it primarily relies on remote services or has no dependency on the local environment.

Una herramienta interactiva que permite a los usuarios evaluar los puntos finales de vLLM a través de MCP, lo que permite realizar pruebas de rendimiento de modelos LLM con parámetros personalizables.

Related MCP Servers

MCP Toolbox
ai-zerolab
-
security
A
license
-
quality
A comprehensive toolkit that enhances LLM capabilities through the Model Context Protocol, allowing LLMs to interact with external services including command-line operations, file management, Figma integration, and audio processing.
Last updated -
17
Python
Apache 2.0
MCP Prompt Tester
rt96-hub
-
security
A
license
-
quality
An MCP server that allows agents to test and compare LLM prompts across OpenAI and Anthropic models, supporting single tests, side-by-side comparisons, and multi-turn conversations.
Last updated -
Python
MIT License
ChuckNorris MCP Server
pollinations
A
security
F
license
A
quality
An experimental MCP gateway that provides specialized LLM enhancement prompts based on the L1B3RT4S repository, primarily intended to enhance weaker models' capabilities.
Last updated -
1
2,012
7
JavaScript
cutterMCP
ap425q
-
security
A
license
-
quality
An MCP server that enables LLMs to autonomously reverse engineer applications through Cutter, allowing them to decompile binaries, analyze code, and rename methods programmatically.
Last updated -
6
Python
Apache 2.0

View all related MCP servers

Appeared in Searches

A service for checking the weather forecast

MCP vLLM Benchmarking Tool