MCP vLLM Benchmarking Tool
This is proof of concept on how to use MCP to interactively benchmark vLLM.
We are not new to benchmarking, read our blog:
This is just an exploration of possibilities with MCP.
Usage
- Clone the repository
- Add it to your MCP servers:
Then you can prompt for example like this:
Todo:
- Due to some random outputs by vllm it may show that it found some invalid json. I have not really looked into it yet.
This server cannot be installed
remote-capable server
The server can be hosted and run remotely because it primarily relies on remote services or has no dependency on the local environment.
An interactive tool that enables users to benchmark vLLM endpoints through MCP, allowing performance testing of LLM models with customizable parameters.
Related MCP Servers
- AsecurityAlicenseAqualityAn MCP server that provides LLMs access to other LLMsLast updated -442557JavaScriptMIT License
- -securityFlicense-qualityAn MCP server that allows Claude to interact with local LLMs running in LM Studio, providing access to list models, generate text, and use chat completions through local models.Last updated -8Python
- -securityAlicense-qualityAn MCP server that allows agents to test and compare LLM prompts across OpenAI and Anthropic models, supporting single tests, side-by-side comparisons, and multi-turn conversations.Last updated -PythonMIT License
- AsecurityFlicenseAqualityA lightweight MCP server that provides a unified interface to various LLM providers including OpenAI, Anthropic, Google Gemini, Groq, DeepSeek, and Ollama.Last updated -6545Python