Skip to main content
Glama

LLM MCP Bridge 🌉

Un servidor MCP (Model Context Protocol) agnóstico para cualquier API compatible con OpenAI. Permite analizar y evaluar la calidad de modelos LLM.

🎯 Proveedores Soportados

Cualquier servidor que implemente la OpenAI API:

Proveedor

URL Base Típica

LM Studio

http://localhost:1234/v1

Ollama

http://localhost:11434/v1

vLLM

http://localhost:8000/v1

LocalAI

http://localhost:8080/v1

llama.cpp

http://localhost:8080/v1

OpenAI

https://api.openai.com/v1

Azure OpenAI

https://{resource}.openai.azure.com/

Together.ai

https://api.together.xyz/v1

Groq

https://api.groq.com/openai/v1

Anyscale

https://api.endpoints.anyscale.com/v1

Related MCP server: Polybrain MCP Server

🛠️ Herramientas MCP Disponibles

Herramienta

Descripción

llm_get_models

Obtiene lista de modelos (JSON)

llm_status

Verifica conexión con el servidor

llm_list_models

Lista modelos (formato legible)

llm_chat

Chat con métricas de rendimiento

llm_benchmark

Benchmark con múltiples prompts

llm_evaluate_coherence

Evalúa consistencia del modelo

llm_test_capabilities

Test en diferentes áreas

llm_compare_models

Compara múltiples modelos

llm_quality_report

Reporte completo de calidad

Parámetros Configurables en Chat

Todas las herramientas aceptan baseURL y apiKey opcionales para override de conexión.

Parámetro

Descripción

Default

prompt

Texto a enviar al modelo

requerido

model

ID del modelo

modelo por defecto

maxTokens

Máximo de tokens

512

temperature

Temperatura (0-2)

0.7

topP

Nucleus sampling (0-1)

-

topK

Top K sampling

-

repeatPenalty

Penalización repetición

-

presencePenalty

Penalización presencia

-

frequencyPenalty

Penalización frecuencia

-

stop

Secuencias de parada

-

systemPrompt

Prompt de sistema

-

📋 Requisitos

  • Node.js >= 18

  • Un servidor LLM con API compatible con OpenAI

🚀 Instalación

cd llm-mcp-bridge
npm install
npm run build

⚙️ Configuración en VS Code

Añade a tu archivo mcp.json de VS Code:

LM Studio (local)

{
  "servers": {
    "llm-local": {
      "type": "stdio",
      "command": "node",
      "args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
      "env": {
        "LLM_BASE_URL": "http://localhost:1234/v1"
      }
    }
  }
}

Ollama

{
  "servers": {
    "ollama": {
      "type": "stdio",
      "command": "node",
      "args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
      "env": {
        "LLM_BASE_URL": "http://localhost:11434/v1"
      }
    }
  }
}

OpenAI

{
  "servers": {
    "openai": {
      "type": "stdio",
      "command": "node",
      "args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
      "env": {
        "LLM_BASE_URL": "https://api.openai.com/v1",
        "LLM_API_KEY": "sk-..."
      }
    }
  }
}

Groq

{
  "servers": {
    "groq": {
      "type": "stdio",
      "command": "node",
      "args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
      "env": {
        "LLM_BASE_URL": "https://api.groq.com/openai/v1",
        "LLM_API_KEY": "gsk_..."
      }
    }
  }
}

Múltiples proveedores

{
  "servers": {
    "llm-lmstudio": {
      "type": "stdio",
      "command": "node",
      "args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
      "env": {
        "LLM_BASE_URL": "http://localhost:1234/v1"
      }
    },
    "llm-ollama": {
      "type": "stdio",
      "command": "node",
      "args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
      "env": {
        "LLM_BASE_URL": "http://localhost:11434/v1"
      }
    },
    "llm-openai": {
      "type": "stdio",
      "command": "node",
      "args": ["/ruta/a/llm-mcp-bridge/dist/index.js"],
      "env": {
        "LLM_BASE_URL": "https://api.openai.com/v1",
        "LLM_API_KEY": "sk-..."
      }
    }
  }
}

🔧 Variables de Entorno

Variable

Descripción

Default

LLM_BASE_URL

URL del servidor LLM

http://localhost:1234/v1

LLM_API_KEY

API Key (requerida para servicios cloud)

-

📖 Ejemplos de Uso

Verificar conexión

@llm_status

Obtener modelos (JSON)

@llm_get_models

Chat con métricas

@llm_chat prompt="Explica qué es machine learning" temperature=0.5 maxTokens=256

Chat con otro servidor (override)

@llm_chat prompt="Hola" baseURL="http://localhost:11434/v1"

Benchmark

@llm_benchmark prompts=["Hola", "¿Qué hora es?", "Cuenta hasta 10"]

Reporte de calidad

@llm_quality_report

Comparar modelos

@llm_compare_models prompt="Escribe un haiku sobre la luna"

🏗️ Estructura del Proyecto

llm-mcp-bridge/
├── src/
│   ├── index.ts       # Servidor MCP principal
│   ├── llm-client.ts  # Cliente OpenAI-compatible
│   └── tools.ts       # Definiciones de herramientas MCP
├── dist/              # Código compilado
├── package.json
├── tsconfig.json
└── README.md

📊 Métricas de Calidad

El servidor analiza:

  • Latencia: Tiempo total de respuesta (ms)

  • Tokens/segundo: Velocidad de generación

  • Coherencia: Consistencia entre múltiples ejecuciones

  • Capacidades: Rendimiento en diferentes tipos de tareas

    • Razonamiento

    • Programación

    • Creatividad

    • Conocimiento factual

    • Seguir instrucciones

🤝 Contribuir

¡Las contribuciones son bienvenidas! Abre un issue o pull request.

📄 Licencia

MIT

Available Tools

8 tools
llm_benchmarkC

Ejecuta un benchmark con múltiples prompts para evaluar rendimiento del modelo

ParametersJSON Schema
NameRequiredDescriptionDefault
baseURLNoURL del servidor OpenAI-compatible (ej: http://localhost:1234/v1, http://localhost:11434/v1)
apiKeyNoAPI Key (requerida para OpenAI/Azure, opcional para servidores locales)
promptsYesLista de prompts para el benchmark
modelNoID del modelo
maxTokensNoMax tokens por respuesta (default: 256)
temperatureNoTemperatura (default: 0.7)
topPNoTop P para nucleus sampling
runsNoEjecuciones por prompt (default: 1)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It mentions 'ejecuta un benchmark' but doesn't disclose behavioral traits such as whether this is a read-only operation, potential side effects (e.g., resource consumption, rate limits), authentication needs beyond the apiKey parameter, or what the output looks like (e.g., metrics, timing data). This leaves significant gaps for a tool with 8 parameters and no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence in Spanish that directly states the tool's purpose without any fluff. It's appropriately sized and front-loaded, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (8 parameters, no annotations, no output schema, and multiple sibling tools), the description is incomplete. It lacks details on behavioral transparency, usage guidelines, and output expectations, which are crucial for an AI agent to invoke this tool correctly in context. The high parameter count and evaluation nature suggest more guidance is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, meaning all parameters are documented in the schema itself. The description adds no additional meaning beyond what's in the schema (e.g., it doesn't explain how 'prompts' are used in the benchmark or the significance of 'runs'). With high schema coverage, the baseline score of 3 is appropriate as the description doesn't compensate but also doesn't detract.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('ejecuta un benchmark') and the resource ('múltiples prompts para evaluar rendimiento del modelo'), providing a specific purpose. However, it doesn't explicitly differentiate this benchmarking tool from sibling tools like 'llm_compare_models' or 'llm_evaluate_coherence', which might also involve model evaluation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With siblings like 'llm_compare_models' and 'llm_evaluate_coherence' that might overlap in evaluation tasks, there's no indication of context, prerequisites, or exclusions to help the agent choose appropriately.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_chatC

Envía un prompt al modelo y recibe una respuesta con métricas de rendimiento (latencia, tokens/s)

ParametersJSON Schema
NameRequiredDescriptionDefault
baseURLNoURL del servidor OpenAI-compatible (ej: http://localhost:1234/v1, http://localhost:11434/v1)
apiKeyNoAPI Key (requerida para OpenAI/Azure, opcional para servidores locales)
promptYesEl prompt a enviar al modelo
modelNoID del modelo (opcional)
maxTokensNoMáximo de tokens a generar (default: 512)
temperatureNoTemperatura 0-2 (default: 0.7)
topPNoTop P para nucleus sampling (0-1)
topKNoTop K para sampling
repeatPenaltyNoPenalización por repetición
presencePenaltyNoPenalización por presencia (-2 a 2)
frequencyPenaltyNoPenalización por frecuencia (-2 a 2)
stopNoSecuencias de parada
systemPromptNoPrompt de sistema opcional

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions receiving performance metrics (latency, tokens/s), it doesn't describe other critical behaviors: whether this is a read-only or mutating operation, authentication requirements (beyond the apiKey parameter), rate limits, error handling, or what the response structure looks like. For a tool with 13 parameters and no annotations, this is a significant gap in transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that clearly states the core functionality and output. It's appropriately sized and front-loaded with the essential action and result, with zero wasted words or redundant information. Every part of the sentence earns its place by conveying purpose and output metrics.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (13 parameters, no annotations, no output schema, and multiple sibling tools), the description is incomplete. It adequately states the purpose but lacks usage guidelines, behavioral details (like mutation status or error handling), and doesn't address how it differs from siblings. Without an output schema, it should ideally describe the response format beyond just mentioning metrics, but it doesn't. This leaves significant gaps for an AI agent to understand the tool fully.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds no parameter-specific information beyond what's already in the schema. With 100% schema description coverage, all 13 parameters are documented in the input schema (e.g., baseURL, apiKey, prompt, model, maxTokens, etc.). The description doesn't explain how parameters interact or provide additional context about their usage, so it meets the baseline of 3 where the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Envía un prompt al modelo y recibe una respuesta con métricas de rendimiento (latencia, tokens/s)' which translates to 'Sends a prompt to the model and receives a response with performance metrics (latency, tokens/s)'. This specifies the verb (send prompt), resource (model), and output (response with metrics). However, it doesn't explicitly differentiate from sibling tools like llm_benchmark or llm_test_capabilities that might also involve model interaction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With sibling tools like llm_benchmark, llm_compare_models, and llm_test_capabilities, there's no indication of whether this is for general chat, performance testing, or other specific contexts. The agent must infer usage from the name and description alone without explicit direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_compare_modelsC

Compara el rendimiento de múltiples modelos con el mismo prompt

ParametersJSON Schema
NameRequiredDescriptionDefault
baseURLNoURL del servidor OpenAI-compatible (ej: http://localhost:1234/v1, http://localhost:11434/v1)
apiKeyNoAPI Key (requerida para OpenAI/Azure, opcional para servidores locales)
promptYesPrompt para comparar
modelsNoLista de modelos a comparar
maxTokensNoMax tokens (default: 256)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It mentions comparing performance but doesn't disclose key behavioral traits such as what 'rendimiento' (performance) entails (e.g., speed, accuracy, cost), whether it's a read-only operation, if it requires authentication, or any rate limits. This leaves significant gaps for an agent to understand the tool's behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence in Spanish that directly states the tool's purpose without any wasted words. It's front-loaded and appropriately sized for its function, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of comparing multiple LLM models and the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'performance' means, how results are returned, or any prerequisites like authentication needs. For a tool with 5 parameters and no structured output, more context is needed to guide effective use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all 5 parameters. The description adds no additional meaning beyond the schema, such as explaining how 'models' are selected or what 'maxTokens' affects in the comparison. Baseline 3 is appropriate as the schema handles parameter documentation adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Compara el rendimiento de múltiples modelos con el mismo prompt' (Compares the performance of multiple models with the same prompt). It specifies the verb 'compara' (compares) and the resource 'múltiples modelos' (multiple models), but doesn't explicitly differentiate it from sibling tools like llm_benchmark or llm_evaluate_coherence, which might have overlapping functionality.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With siblings like llm_benchmark, llm_evaluate_coherence, and llm_test_capabilities, there's no indication of how this tool differs in context or when it's preferred over others, leaving usage unclear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_evaluate_coherenceC

Evalúa la coherencia del modelo ejecutando el mismo prompt múltiples veces

ParametersJSON Schema
NameRequiredDescriptionDefault
baseURLNoURL del servidor OpenAI-compatible (ej: http://localhost:1234/v1, http://localhost:11434/v1)
apiKeyNoAPI Key (requerida para OpenAI/Azure, opcional para servidores locales)
promptYesPrompt para evaluar
modelNoID del modelo
runsNoNúmero de ejecuciones (default: 3)
temperatureNoTemperatura (default: 0.7)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the core behavior (running the same prompt multiple times to evaluate coherence) but lacks details on what 'coherence' means, how results are measured or returned, whether this is a read-only operation, rate limits, or authentication requirements. The description is too vague for a tool with 6 parameters and no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence in Spanish that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, with every word earning its place. No structural issues or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (6 parameters, no annotations, no output schema), the description is incomplete. It doesn't explain what 'coherence' entails, how results are returned, or the tool's behavioral traits. For an evaluation tool with multiple parameters and no structured output, more context is needed to guide effective use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description doesn't add any meaningful parameter semantics beyond what's in the schema—it doesn't explain how parameters like 'runs' or 'temperature' specifically affect coherence evaluation. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Evalúa la coherencia del modelo ejecutando el mismo prompt múltiples veces' (Evaluates model coherence by running the same prompt multiple times). It specifies the verb (evaluates) and resource (model coherence), but doesn't explicitly differentiate from sibling tools like llm_benchmark or llm_quality_report, which might have overlapping evaluation functions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like llm_benchmark or llm_quality_report, nor does it specify contexts where coherence evaluation is preferred over other types of model testing or comparison. Usage is implied but not articulated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_get_modelsC

Obtiene la lista de modelos disponibles en el servidor LLM (compatible con OpenAI API: LM Studio, Ollama, vLLM, OpenAI, etc.)

ParametersJSON Schema
NameRequiredDescriptionDefault
baseURLNoURL del servidor OpenAI-compatible (ej: http://localhost:1234/v1, http://localhost:11434/v1)
apiKeyNoAPI Key (requerida para OpenAI/Azure, opcional para servidores locales)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool is compatible with various LLM servers but does not describe behavioral traits such as whether it requires authentication (implied by the 'apiKey' parameter but not stated), rate limits, error handling, or what the output looks like (e.g., list format). This is a significant gap for a tool with no annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that states the purpose and adds compatibility context. It is appropriately sized and front-loaded with the core function. However, it could be slightly more structured by separating the compatibility note into a second sentence for clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (a tool to fetch models from various LLM servers), lack of annotations, and no output schema, the description is incomplete. It does not explain what the return value includes (e.g., model names, IDs, capabilities) or handle behavioral aspects like authentication needs. This leaves gaps for the agent to understand the tool fully.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, so the input schema already documents both parameters ('baseURL' and 'apiKey') with descriptions. The description adds no additional meaning beyond what the schema provides—it does not explain parameter usage, dependencies, or examples. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Obtiene la lista de modelos disponibles en el servidor LLM' (Gets the list of available models on the LLM server). It specifies the verb ('obtiene') and resource ('lista de modelos'), but does not explicitly differentiate it from sibling tools like 'llm_status' or 'llm_test_capabilities', which might also involve server interaction. The mention of compatibility with various LLM servers (OpenAI API, LM Studio, etc.) adds useful context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention sibling tools like 'llm_status' (which might check server status) or 'llm_compare_models' (which might compare models), leaving the agent to infer usage based on tool names alone. There is no explicit 'when' or 'when not' context provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_quality_reportC

Genera un reporte completo de calidad del modelo incluyendo benchmark, coherencia y capacidades

ParametersJSON Schema
NameRequiredDescriptionDefault
baseURLNoURL del servidor OpenAI-compatible (ej: http://localhost:1234/v1, http://localhost:11434/v1)
apiKeyNoAPI Key (requerida para OpenAI/Azure, opcional para servidores locales)
modelNoID del modelo a evaluar

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool generates a report but doesn't describe what the report contains, its format, whether it's saved or returned, execution time, rate limits, or authentication needs beyond what's implied by the parameters. For a tool with no annotations and potentially complex behavior, this is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence in Spanish that directly states the tool's purpose and scope. It's appropriately sized and front-loaded with the main action. However, it could be slightly more structured by explicitly mentioning it's a comprehensive report that aggregates multiple evaluation aspects.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of generating a quality report with multiple components (benchmark, coherence, capabilities), no annotations, no output schema, and sibling tools that handle individual aspects, the description is incomplete. It doesn't explain what the output looks like, how it differs from using the specialized tools, or any behavioral details. This leaves significant gaps for an AI agent to understand the tool's full context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters (baseURL, apiKey, model) with descriptions. The description adds no additional meaning about the parameters beyond what the schema provides. According to the rules, with high schema coverage (>80%), the baseline score is 3 even with no param info in the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Genera un reporte completo de calidad del modelo' (Generates a complete quality report for the model). It specifies the verb 'genera' and resource 'reporte de calidad del modelo', and mentions the report includes 'benchmark, coherencia y capacidades' (benchmark, coherence, and capabilities). However, it doesn't explicitly differentiate from sibling tools like llm_benchmark, llm_evaluate_coherence, or llm_test_capabilities, which appear to cover similar aspects individually.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools such as llm_benchmark, llm_evaluate_coherence, or llm_test_capabilities, which seem to handle specific components of the quality report. There's no indication of prerequisites, context, or exclusions for using this comprehensive report tool over the more specialized ones.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_statusB

Verifica el estado de conexión con el servidor LLM y lista los modelos disponibles

ParametersJSON Schema
NameRequiredDescriptionDefault
baseURLNoURL del servidor OpenAI-compatible (ej: http://localhost:1234/v1, http://localhost:11434/v1)
apiKeyNoAPI Key (requerida para OpenAI/Azure, opcional para servidores locales)

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions connection verification and model listing but fails to describe what 'estado de conexión' entails (e.g., response format, error handling), whether it's a read-only operation, or any rate limits. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence in Spanish that front-loads the core purpose without unnecessary details. Every word earns its place, making it appropriately sized and well-structured for quick comprehension.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is minimally adequate. It states what the tool does but lacks details on behavioral traits, usage context, or return values. Without annotations or output schema, it should do more to compensate, but it meets the bare minimum for a simple status-check tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds no additional meaning beyond what the schema provides about baseURL or apiKey. Baseline 3 is appropriate when the schema does the heavy lifting, though the description doesn't compensate with extra context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with specific verbs ('Verifica el estado de conexión' and 'lista los modelos disponibles') and identifies the resource (servidor LLM). It distinguishes from siblings like llm_get_models by combining connection verification with model listing, though it doesn't explicitly contrast with them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like llm_get_models or llm_test_capabilities. It lacks explicit when/when-not instructions or prerequisites, leaving usage context implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_test_capabilitiesC

Prueba las capacidades del modelo en diferentes áreas: razonamiento, código, creatividad, hechos, instrucciones

ParametersJSON Schema
NameRequiredDescriptionDefault
baseURLNoURL del servidor OpenAI-compatible (ej: http://localhost:1234/v1, http://localhost:11434/v1)
apiKeyNoAPI Key (requerida para OpenAI/Azure, opcional para servidores locales)
modelNoID del modelo

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions testing capabilities but doesn't describe what the tool actually does behaviorally—e.g., whether it runs predefined tests, generates reports, requires specific permissions, has side effects, or returns structured results. For a testing tool with zero annotation coverage, this leaves critical behavioral traits unspecified.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded, stating the purpose in a single sentence. It efficiently lists capability areas without unnecessary elaboration. However, it could be slightly more structured by explicitly naming the tool's output or behavioral context, but it avoids waste and is appropriately sized for its purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of testing model capabilities, no annotations, and no output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., test results, scores, reports), how it interacts with the model, or any behavioral details. This leaves significant gaps for an agent to understand the tool's full context and usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters (baseURL, apiKey, model) with descriptions. The description adds no parameter-specific information beyond what's in the schema, such as how these inputs relate to the testing process. With high schema coverage, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Prueba las capacidades del modelo en diferentes áreas' (Test the model's capabilities in different areas). It specifies the verb 'prueba' (test) and the resource 'modelo' (model), with examples of capability areas (reasoning, code, creativity, facts, instructions). However, it doesn't explicitly differentiate from siblings like llm_benchmark or llm_evaluate_coherence, which may have overlapping testing functions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like llm_benchmark or llm_evaluate_coherence, nor does it specify prerequisites, contexts, or exclusions for usage. The agent must infer usage from the purpose alone, which is insufficient for informed selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

B3.3/5.0
Disambiguation3/5

Most tools have distinct purposes like benchmarking, chatting, and comparing models, but there is overlap between llm_get_models and llm_status, as both list available models. The descriptions help differentiate them, but an agent might still be confused about which to use for model listing.

Naming Consistency5/5

All tool names follow a consistent 'llm_' prefix with descriptive suffixes in snake_case, such as llm_benchmark and llm_chat. This predictable pattern makes it easy for agents to understand and navigate the toolset without confusion.

Tool Count5/5

With 8 tools, the count is well-scoped for evaluating and testing LLM models. Each tool serves a specific function in performance analysis, quality assessment, and model management, fitting the server's purpose without being overwhelming or insufficient.

Completeness4/5

The toolset covers key aspects of LLM evaluation, including benchmarking, chatting, model comparison, coherence testing, and capability assessment. A minor gap is the lack of tools for model configuration or fine-tuning, but core workflows are well-covered for quality and performance analysis.

Maintenance

ActivityInactive
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • -
    license
    C
    quality
    Not graded
    maintenance
    Enables interaction with OpenAI-compatible APIs (like Ollama) through MCP tools. Provides access to chat completions, model listings, and embeddings generation from local or remote OpenAI-style endpoints.
    3

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/ramgeart/llm-mcp-bridge'

If you have feedback or need assistance with the MCP directory API, please join our Discord server