LLM MCP Bridge
Enables interaction with Ollama's local LLM server through OpenAI-compatible API, providing tools for model listing, chat, benchmarking, and quality evaluation.
Provides tools for interacting with OpenAI's API, including chat, model management, performance benchmarking, coherence evaluation, and quality reporting across different task types.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@LLM MCP Bridge chat prompt="Explain quantum computing in simple terms" model="gpt-4" temperature=0.8"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
LLM MCP Bridge 🌉
Un servidor MCP (Model Context Protocol) agnóstico para cualquier API compatible con OpenAI. Permite analizar y evaluar la calidad de modelos LLM.
🎯 Proveedores Soportados
Cualquier servidor que implemente la OpenAI API:
Proveedor | URL Base Típica |
LM Studio |
|
Ollama |
|
vLLM |
|
LocalAI |
|
llama.cpp |
|
OpenAI |
|
Azure OpenAI |
|
Together.ai |
|
Groq |
|
Anyscale |
|
Related MCP server: LLM API Benchmark MCP Server
🛠️ Herramientas MCP Disponibles
Herramienta | Descripción |
| Obtiene lista de modelos (JSON) |
| Verifica conexión con el servidor |
| Lista modelos (formato legible) |
| Chat con métricas de rendimiento |
| Benchmark con múltiples prompts |
| Evalúa consistencia del modelo |
| Test en diferentes áreas |
| Compara múltiples modelos |
| Reporte completo de calidad |
Parámetros Configurables en Chat
Todas las herramientas aceptan baseURL y apiKey opcionales para override de conexión.
Parámetro | Descripción | Default |
| Texto a enviar al modelo | requerido |
| ID del modelo | modelo por defecto |
| Máximo de tokens | 512 |
| Temperatura (0-2) | 0.7 |
| Nucleus sampling (0-1) | - |
| Top K sampling | - |
| Penalización repetición | - |
| Penalización presencia | - |
| Penalización frecuencia | - |
| Secuencias de parada | - |
| Prompt de sistema | - |
📋 Requisitos
Node.js >= 18
Un servidor LLM con API compatible con OpenAI
🚀 Instalación
cd llm-mcp-bridge
npm install
npm run build⚙️ Configuración en VS Code
Añade a tu archivo mcp.json de VS Code:
LM Studio (local)
{
"servers": {
"llm-local": {
"type": "stdio",
"command": "node",
"args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
"env": {
"LLM_BASE_URL": "http://localhost:1234/v1"
}
}
}
}Ollama
{
"servers": {
"ollama": {
"type": "stdio",
"command": "node",
"args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
"env": {
"LLM_BASE_URL": "http://localhost:11434/v1"
}
}
}
}OpenAI
{
"servers": {
"openai": {
"type": "stdio",
"command": "node",
"args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
"env": {
"LLM_BASE_URL": "https://api.openai.com/v1",
"LLM_API_KEY": "sk-..."
}
}
}
}Groq
{
"servers": {
"groq": {
"type": "stdio",
"command": "node",
"args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
"env": {
"LLM_BASE_URL": "https://api.groq.com/openai/v1",
"LLM_API_KEY": "gsk_..."
}
}
}
}Múltiples proveedores
{
"servers": {
"llm-lmstudio": {
"type": "stdio",
"command": "node",
"args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
"env": {
"LLM_BASE_URL": "http://localhost:1234/v1"
}
},
"llm-ollama": {
"type": "stdio",
"command": "node",
"args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
"env": {
"LLM_BASE_URL": "http://localhost:11434/v1"
}
},
"llm-openai": {
"type": "stdio",
"command": "node",
"args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
"env": {
"LLM_BASE_URL": "https://api.openai.com/v1",
"LLM_API_KEY": "sk-..."
}
}
}
}🔧 Variables de Entorno
Variable | Descripción | Default |
| URL del servidor LLM |
|
| API Key (requerida para servicios cloud) | - |
📖 Ejemplos de Uso
Verificar conexión
@llm_statusObtener modelos (JSON)
@llm_get_modelsChat con métricas
@llm_chat prompt="Explica qué es machine learning" temperature=0.5 maxTokens=256Chat con otro servidor (override)
@llm_chat prompt="Hola" baseURL="http://localhost:11434/v1"Benchmark
@llm_benchmark prompts=["Hola", "¿Qué hora es?", "Cuenta hasta 10"]Reporte de calidad
@llm_quality_reportComparar modelos
@llm_compare_models prompt="Escribe un haiku sobre la luna"🏗️ Estructura del Proyecto
llm-mcp-bridge/
├── src/
│ ├── index.ts # Servidor MCP principal
│ ├── llm-client.ts # Cliente OpenAI-compatible
│ └── tools.ts # Definiciones de herramientas MCP
├── dist/ # Código compilado
├── package.json
├── tsconfig.json
└── README.md📊 Métricas de Calidad
El servidor analiza:
Latencia: Tiempo total de respuesta (ms)
Tokens/segundo: Velocidad de generación
Coherencia: Consistencia entre múltiples ejecuciones
Capacidades: Rendimiento en diferentes tipos de tareas
Razonamiento
Programación
Creatividad
Conocimiento factual
Seguir instrucciones
🤝 Contribuir
¡Las contribuciones son bienvenidas! Abre un issue o pull request.
📄 Licencia
MIT
Available Tools
8 toolsllm_benchmarkC
Ejecuta un benchmark con múltiples prompts para evaluar rendimiento del modelo
Inputs: prompts (required), baseURL, apiKey, model, +4 more.
llm_chatC
Envía un prompt al modelo y recibe una respuesta con métricas de rendimiento (latencia, tokens/s)
Inputs: prompt (required), baseURL, apiKey, model, +9 more.
llm_compare_modelsC
Compara el rendimiento de múltiples modelos con el mismo prompt
Inputs: prompt (required), baseURL, apiKey, models, +1 more.
llm_evaluate_coherenceC
Evalúa la coherencia del modelo ejecutando el mismo prompt múltiples veces
Inputs: prompt (required), baseURL, apiKey, model, +2 more.
llm_get_modelsC
Obtiene la lista de modelos disponibles en el servidor LLM (compatible con OpenAI API: LM Studio, Ollama, vLLM, OpenAI, etc.)
Inputs: baseURL, apiKey.
llm_quality_reportC
Genera un reporte completo de calidad del modelo incluyendo benchmark, coherencia y capacidades
Inputs: baseURL, apiKey, model.
llm_statusB
Verifica el estado de conexión con el servidor LLM y lista los modelos disponibles
Inputs: baseURL, apiKey.
llm_test_capabilitiesC
Prueba las capacidades del modelo en diferentes áreas: razonamiento, código, creatividad, hechos, instrucciones
Inputs: baseURL, apiKey, model.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 8 tools
Most tools have distinct purposes like benchmarking, chatting, and comparing models, but there is overlap between llm_get_models and llm_status, as both list available models. The descriptions help differentiate them, but an agent might still be confused about which to use for model listing.
All tool names follow a consistent 'llm_' prefix with descriptive suffixes in snake_case, such as llm_benchmark and llm_chat. This predictable pattern makes it easy for agents to understand and navigate the toolset without confusion.
With 8 tools, the count is well-scoped for evaluating and testing LLM models. Each tool serves a specific function in performance analysis, quality assessment, and model management, fitting the server's purpose without being overwhelming or insufficient.
The toolset covers key aspects of LLM evaluation, including benchmarking, chatting, model comparison, coherence testing, and capability assessment. A minor gap is the lack of tools for model configuration or fine-tuning, but core workflows are well-covered for quality and performance analysis.
Maintenance
Related MCP Connectors
Open AI models through one OpenAI-compatible API: chat, live model list, prices and cost estimates.
Open-weight and abliterated LLMs on one API: compare models and rankings, chat, check credits.
OpenAI-compatible BYOK LLM gateway: per-task routing, shadow evaluation, promotion gates.
Benchmark your own prompt on 100+ LLMs with real API calls; pick the cheapest model that passes.
Related MCP Servers
- AlicenseAqualityDmaintenanceEnables AI agents to connect to and chat with multiple LLM models (OpenAI, OpenRouter, custom endpoints) with conversation history management and model switching capabilities.318 npm2MIT
- AlicenseNot gradedqualityNot gradedmaintenanceEnables benchmarking of Large Language Model APIs by measuring performance metrics such as generation throughput, prompt throughput, and Time To First Token (TTFT) with configurable concurrency levels and parameters.1MIT
- AlicenseNot gradedqualityCmaintenanceAggregates dozens of pre-built connectors into one OpenAI-compatible endpoint for AI clients, with a web UI for installing, configuring, monitoring, and managing them.MIT
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to route chat completion requests to task-appropriate models via LiteLLM/OpenRouter, track usage and budgets, and run side-by-side eval comparisons of models based on cost, latency, and response quality.101 npmMIT