LLM MCP Bridge

Instructions

Implementation Reference

• src/tools.ts:430-486 (handler)

  The core handler function that implements the llm_quality_report tool. It performs a benchmark, coherence evaluation, capabilities test on the specified model, compiles metrics, and returns a formatted markdown report with an overall quality score.

  async llm_quality_report(args: z.infer<typeof CapabilitiesSchema>) {
    const client = getClient(args);
    let output = `# 📋 Reporte de Calidad del Modelo\n\n`;
    output += `*Generando reporte completo...*\n\n`;
  
    // 1. Benchmark básico
    const benchmarkPrompts = [
      "Explica qué es la inteligencia artificial en una oración.",
      "¿Cuánto es 25 * 4?",
      "Traduce 'Hello World' al español.",
    ];
  
    const benchmark = await client.runBenchmark(benchmarkPrompts, {
      model: args.model,
      maxTokens: 100,
    });
  
    output += `## 📊 Benchmark de Rendimiento\n\n`;
    output += `- Latencia promedio: **${benchmark.summary.avgLatencyMs.toFixed(0)} ms**\n`;
    output += `- Velocidad: **${benchmark.summary.avgTokensPerSecond.toFixed(2)} tokens/s**\n`;
    output += `- Tokens generados: ${benchmark.summary.totalTokensGenerated}\n\n`;
  
    // 2. Coherencia
    const coherence = await client.evaluateCoherence(
      "¿Cuál es el sentido de la vida?",
      { model: args.model, runs: 3, temperature: 0.7 }
    );
  
    output += `## 🎯 Coherencia\n\n`;
    output += `- Consistencia: **${(coherence.consistency * 100).toFixed(1)}%**\n`;
    output += `- Longitud promedio de respuesta: ${coherence.avgLength.toFixed(0)} chars\n\n`;
  
    // 3. Capacidades
    const capabilities = await client.testCapabilities({ model: args.model });
  
    output += `## 🧠 Capacidades\n\n`;
    output += `| Área | Latencia | Velocidad |\n`;
    output += `|------|----------|----------|\n`;
    
    const areas = ["reasoning", "coding", "creative", "factual", "instruction"] as const;
    for (const area of areas) {
      const r = capabilities[area];
      output += `| ${area} | ${r.latencyMs}ms | ${r.tokensPerSecond.toFixed(1)} tok/s |\n`;
    }
  
    output += `\n## 📈 Puntuación General\n\n`;
    const avgSpeed = benchmark.summary.avgTokensPerSecond;
    const speedScore = Math.min(100, avgSpeed * 2);
    const coherenceScore = coherence.consistency * 100;
    const overallScore = (speedScore + coherenceScore) / 2;
  
    output += `- Velocidad: ${speedScore.toFixed(0)}/100\n`;
    output += `- Coherencia: ${coherenceScore.toFixed(0)}/100\n`;
    output += `- **Puntuación Total: ${overallScore.toFixed(0)}/100**\n`;
  
    return { content: [{ type: "text" as const, text: output }] };
  },

• src/tools.ts:182-194 (schema)

  MCP tool schema definition for 'llm_quality_report', including name, description, and input schema (model ID and optional connection properties). Part of the exported 'tools' array.

    {
      name: "llm_quality_report",
      description: "Genera un reporte completo de calidad del modelo incluyendo benchmark, coherencia y capacidades",
      inputSchema: {
        type: "object" as const,
        properties: {
          ...connectionProperties,
          model: { type: "string", description: "ID del modelo a evaluar" },
        },
        required: [],
      },
    },
  ];

• src/index.ts:42-44 (registration)

  Registers the list tools handler which returns the array of tool definitions including llm_quality_report.

  server.setRequestHandler(ListToolsRequestSchema, async () => {
    return { tools };
  });

• src/index.ts:76-77 (registration)

  Dispatch case in the CallToolRequestHandler that routes execution to the llm_quality_report handler.

  case "llm_quality_report":
    return await toolHandlers.llm_quality_report(args as any);

• src/tools.ts:47-49 (schema)

  Zod schema used for type inference in the llm_quality_report handler arguments, matching the tool's input schema.

  export const CapabilitiesSchema = ConnectionConfigSchema.extend({
    model: z.string().optional().describe("ID del modelo a usar"),
  });