chimeraforge
Chimeraforge
Un planificador de despliegue de LLM local-first y agnóstico al modelo. Convierte "qué modelo, cuantización, GPU y backend -- cuántos, cabrá, cumplirá mi SLO, cuánto costará" en una respuesta rápida, honesta y medida, desde tu shell, tu Python o tu asistente de IA.
uvx chimeraforge plan --model-size 8b --hardware "RTX 4090 24GB"El principio de confianza
Cada número está etiquetado como medido, estimado o desconocido, y la herramienta se niega a falsificar los que no puede respaldar. La VRAM y la caché de KV se calculan a partir de la arquitectura real de un modelo (exacto). El rendimiento es una búsqueda medida cuando está disponible; de lo contrario, una estimación explícita de límite de ancho de banda -- nunca presentada como datos que no son. La calidad por debajo del corpus incluido informa desconocido, no una puntuación inventada. Un plan con 0 resultados nombra la puerta exacta que rechazó a cada candidato en lugar de un genérico "no se encontró nada". Sin telemetría, sin llamadas a casa, funciona aislado.
Dale un modelo -- una clase de tamaño, un repositorio de Hugging Face, una etiqueta de Ollama o anulaciones manuales para un modelo no publicado -- y busca el espacio (modelo x cuantización x backend x número de GPUs x paralelismo de tensor/pipeline) contra VRAM, calidad, latencia, coste, energía y una puerta de seguridad opcional, y luego devuelve la configuración más barata que cumple tu SLO.
12 comandos, una herramienta: plan - suggest - measure - validate - catalog - safety - bench - eval - compare - refit - report - mcp.
El corpus empírico se remonta a los Informes Técnicos TR108-TR137 (~204.000 mediciones reales en GPUs de consumo). Consulta el CHANGELOG para el historial completo de funciones.
Related MCP server: llm-cost
Instalación
Pruébalo sin instalar:
uvx chimeraforge plan --model-size 8b --hardware "RTX 4090 24GB"
pipx run chimeraforge plan --model-size 8b --hardware "RTX 4090 24GB"Instala de verdad:
pip install chimeraforge # planner + model resolution (HF/Ollama) + suggest/measure/safety/bench
pip install chimeraforge[bench] # + GPU environment metadata for benchmarks (pynvml)
pip install chimeraforge[mcp] # + MCP server so Claude/GPT/Cursor can call the planner
pip install chimeraforge[eval] # + quality evaluation (BERTScore, ROUGE-L)
pip install chimeraforge[refit] # + coefficient refitting (numpy, scipy)
pip install chimeraforge[all] # everythingPython 3.10+. La instalación principal cubre el planificador y los comandos orientados a red (httpx es una dependencia principal). plan / suggest / catalog funcionan completamente sin conexión; bench / measure / safety necesitan un backend en ejecución (Ollama, vLLM o TGI). Windows / macOS / Linux.
Inicio rápido
# Plan a registry size class on your GPU
chimeraforge plan --model-size 8b --hardware "RTX 4090 24GB" --request-rate 2.0
# Plan ANY model -- a Hugging Face repo or an Ollama tag
chimeraforge plan --model Qwen/Qwen2.5-7B-Instruct --hardware "RTX 4090 24GB"
chimeraforge plan --model ollama:qwen3:14b --ollama-url http://localhost:11434
# Split a model too big for one GPU across several (tensor parallelism)
chimeraforge plan --model meta-llama/Llama-3.3-70B-Instruct --hardware "H100 80GB" --tp 4
# Shrink the KV-cache, print the cost/latency/quality trade-off menu
chimeraforge plan --model-size 8b --hardware "RTX 4080 12GB" --kv-quant q8 --pareto
# Benchmark a live model and plan on the MEASURED numbers
chimeraforge plan --model qwen3:14b --measure
# Discover + rank what fits your GPU and budget
chimeraforge suggest --source ollama --hardware "RTX 4090 24GB" --budget 500Servidor MCP -- dale a Claude / GPT / Cursor los mismos números
El dimensionado de GPU es exactamente donde fallan los asistentes: precios y especificaciones de hardware con fecha de corte de entrenamiento, además de aritmética de caché de KV/lote propensa a errores hecha de memoria. chimeraforge mcp ejecuta un servidor MCP stdio para que un asistente llame al planificador real contra datos medidos en lugar de adivinar.
pip install "chimeraforge[mcp]"Claude Code:
claude mcp add --transport stdio chimeraforge -- uvx --from "chimeraforge[mcp]" chimeraforge mcpClaude Desktop / Cursor (añade a tu archivo de configuración MCP):
{
"mcpServers": {
"chimeraforge": {
"command": "uvx",
"args": ["--from", "chimeraforge[mcp]", "chimeraforge", "mcp"]
}
}
}El --from "chimeraforge[mcp]" incluye el SDK de MCP; uvx ejecuta el servidor en un entorno autocontenido. Si ya has hecho pip install "chimeraforge[mcp]" en el entorno que tu cliente lanza, puedes usar en su lugar "command": "chimeraforge", "args": ["mcp"].
Expone tres herramientas: chimeraforge_plan (la búsqueda completa de puertas), chimeraforge_resolve_model (fundamenta un id de modelo en sus parámetros/arquitectura reales) y chimeraforge_list_hardware. Cada resultado lleva la misma procedencia medido / estimado / desconocido que el CLI, y las descripciones de las herramientas le dicen al modelo que las prefiera sobre su propio conocimiento. chimeraforge_plan también devuelve un campo launch -- el comando de servicio para la configuración recomendada -- para que el asistente pueda responder "y cómo lo ejecuto" sin inventar banderas.
Comandos
plan -- planificador de capacidad predictivo
chimeraforge plan --model-size 8b --hardware "RTX 4090 24GB" --request-rate 2.0
chimeraforge plan --model Qwen/Qwen2.5-7B-Instruct --hardware "RTX 4090 24GB" # any HF repo
chimeraforge plan --model ollama:qwen3:14b --ollama-url http://localhost:11434 # any Ollama tag
chimeraforge plan --model meta-llama/Llama-3.3-70B-Instruct --hardware "H100 80GB" --tp 4 # multi-GPU
chimeraforge plan --model-size 3b --kv-quant q4 --pareto # smaller KV cache, trade-off menu
chimeraforge plan --model-size 8b --hardware "RTX 4090 24GB" --launch # + the serve command to actually run it
chimeraforge plan --model-size 3b --workload agent --safety-target 0.85 --jsonPlanifica cualquier modelo: clase de tamaño de registro, repositorio HF (
org/name), etiqueta de Ollama o anulaciones manuales (--params-b/--n-layers/...).Busca (modelo x cuantización x backend x N-réplicas x lote/GPU) a través de un pipeline de 5 puertas: VRAM -> calidad -> seguridad (opcional) -> latencia -> presupuesto.
Modela la física real de servicio: batching continuo (vLLM/TGI), división prefill/decode (TTFT + TPOT), concurrencia limitada por caché de KV y cola consciente de varianza (
--workload).Ajusta modelos demasiado grandes para una GPU:
--tensor-parallel/--tp {N|auto}fragmenta pesos + KV entre N GPUs (estilo Megatron, con comunicaciones modeladas);--pipeline-parallel/--pp {N|auto}divide capas entre N etapas en su lugar (más barato en interconexiones lentas, necesita batching para llenar el pipeline). Aún no combinables.Sirve lo que el backend sirve: los cuantos GGUF se ofrecen en Ollama; vLLM/TGI reciben FP16 y FP8 (solo en GPUs con núcleos de tensor FP8 -- Ada/Hopper/Blackwell/CDNA3). El planificador ya no sugiere un checkpoint GGUF en vLLM con un aumento de velocidad de llama.cpp.
Cuantización de caché de KV (
--kv-quant {fp16,q8,q4}) reduce la caché y aumenta la concurrencia máxima -- la mayor ganancia en contexto largo.Realismo de costes (
--duty-cycle,--gpu-price-multiplier): el precio principal de $/1M-tok valora una flota saturada. También pagas por el margen aprovisionado y por cada hora inactiva, así que la cifra efectiva en un 8B a 2 req/s es $2.71/1M a pleno ciclo y $9.04/1M al 30%, frente a $0.92 a capacidad. El precio spot/reservado es tu entrada, no una suposición incluida.Punto de equilibrio entre autoalojamiento y API (
--compare-api): valora tu carga de trabajo contra APIs alojadas y reporta el volumen mensual donde el autoalojamiento empieza a ganar. Los precios son una instantánea con fecha y una URL de fuente por proveedor, marcada como obsoleta pasados 90 días -- nunca presentada como una cotización en vivo -- y una API de frontera se etiqueta como un nivel de calidad diferente en lugar de pasarse como equivalente.Caché de prefijo (
--prefix-cache-hit-rate): el tráfico de chatbot y agente reutiliza un prompt de sistema largo, así que la mayor parte del prefill ya está en caché. Con un prompt de 4k y un 90% de acierto, un 8B pasa de 166ms a 17ms de TTFT. Por defecto es 0 y nunca se infiere, y el KV que un prefijo compartido ahorra no se deduce deliberadamente -- subdimensionar el KV es lo que convierte "cabe" en un OOM.Modelos de razonamiento (
--reasoning-tokens N): los tokens de pensamiento ocultos son decodificados por la GPU y se mantienen en KV aunque el llamador nunca los vea. Contar solo la salida visible subestima la decodificación por la proporción de razonamiento -- 1000 tokens ocultos llevaron un plan de 8B de 363ms a 6128ms p95 en nuestra propia comprobación. Por defecto es 0 y nunca se infiere: la proporción es una propiedad de tu carga de trabajo, no de los pesos.KV consciente de la forma de atención: MLA (DeepSeek-V2/V3) cachea un latente comprimido en lugar de K/V por cabeza -- dimensionarlo como GQA sobreestima la caché de DeepSeek-V3 en 57x -- y los modelos de ventana deslizante dejan de crecer la caché más allá de la ventana. Una ventana cuyo patrón de capas no está declarado no se aplica, porque subdimensionar el KV es lo que convierte "cabe" en un OOM.
Consciente de Mixture-of-Experts: la VRAM se dimensiona sobre los parámetros totales (cada experto permanece residente) mientras que el rendimiento y el TTFT usan los parámetros activos (un token solo lee los expertos a los que se enruta). Tratar un modelo MoE como denso subestima su rendimiento en 3.6x en Mixtral-8x7B y ~18x en DeepSeek-V3. Los recuentos activos se derivan de la geometría real de expertos del modelo y coinciden con las cifras publicadas.
Energía (
--electricity-rate): coste mensual de kWh,$/1M-tok (+energía)y tok/s por vatio, reportados junto al (no incluidos en) la puerta de presupuesto.Exportación de comando de lanzamiento (
--launch): emite el comandovllm serve/ollama run/ TGIdocker runpara la configuración ganadora, con la longitud de contexto del propio plan, el grado de TP/PP, el tamaño de lote y el dtype de KV rellenados -- las banderas que son propensas a error de calcular a mano. No fabrica lo que no puede derivar: un nivel de cuantización GGUF se convierte en una nota para servir el checkpoint equivalente nativo, no una bandera--quantizationinventada.Procedencia por predicción (
medido/estimado/desconocido); explica la puerta vinculante cuando nada cabe.Validado en datos de registro: VRAM R^2=0.968, rendimiento R^2=0.859, RMSE de calidad=0.062, MAPE de latencia=1.05% (supera el M/D/1 analítico en 20.4x, TR133). Sin ML -- tablas de búsqueda empíricas con interpolación de primeros principios (límite de ancho de banda para modelos fuera de registro).
suggest -- descubre y clasifica modelos
chimeraforge suggest --source ollama --hardware "RTX 4090 24GB" --budget 500
chimeraforge suggest --source hf --hf-limit 8 --hardware "RTX 4080 12GB"
chimeraforge suggest --source catalog --hardware "RTX 4080 12GB" # offline, after `catalog --build`Tira candidatos de un Ollama en vivo (/api/tags), el HF Hub (generación de texto superior) y/o el catálogo local; resuelve cada uno a parámetros/arquitectura reales, ejecuta la misma búsqueda de puertas y muestra la mejor configuración por modelo.
measure -- benchmark en vivo, planifica con números reales
chimeraforge measure --model qwen3:14b --ollama-url http://localhost:11434
chimeraforge plan --model qwen3:14b --measure # measure then plan in one stepBenchmarkea el modelo en vivo (rendimiento real N=1, tiempo de servicio, escalado de concurrencia) y lo incorpora a un corpus local. plan / suggest prefieren entonces los números medidos automáticamente (la procedencia cambia a medido).
catalog -- catálogo de modelos local
chimeraforge catalog --build # resolve a curated seed (+ --with-ollama) and cache specs
chimeraforge catalog # list the cached catalogPersiste las especificaciones resueltas para que suggest --source catalog clasifique un conjunto conocido y bueno completamente sin conexión.
safety -- pantalla de rechazo en vivo
chimeraforge safety --model llama3.2-3b --prompts harmful.txt --quant Q4_K_M --safety-target 0.85Donde plan --safety-target decide a partir de datos TR134/TR142 incluidos, safety mide: ejecuta tus prompts de prueba contra un modelo en vivo, clasifica los rechazos (basado en reglas -- la línea base de regex TR134), reporta la tasa de rechazo medida frente a los datos de puerta incluidos (esperado, deriva, nivel de riesgo RTSI) y sale 1 por debajo de --safety-target. Tú proporcionas los prompts (--prompts, uno por línea) -- no se incluye ningún corpus de ataque con el paquete; apúntalo a HarmBench / AdvBench / tu propio conjunto. Necesita un Ollama en ejecución.
bench -- benchmark de inferencia en vivo
chimeraforge bench --model llama3.2-3b --runs 5
chimeraforge bench --model llama3.2-3b --all-quants --context 512,1024,2048,4096 --json
chimeraforge bench --model llama3.2-3b --backend vllm --base-url http://localhost:8000Tres perfiles de carga de trabajo (single / batch / servidor-Poisson); mide rendimiento, TTFT y latencia con p50/p90/p95/p99; advertencias de estabilidad basadas en CV; salida JSON.
eval -- evaluación de calidad
chimeraforge eval --task general_knowledge --json
chimeraforge eval --predictions preds.txt --references refs.txt --model llama3.2-3bMétricas: coincidencia exacta, ROUGE-L (respaldo LCS), BERTScore, coherencia -> compuesta (0.2*EM + 0.3*ROUGE + 0.3*BERT + 0.2*coherencia). Niveles de calidad de TR125; 3 tareas integradas (general_knowledge, summarization, code). Pasa --fp16-baseline para clasificar el nivel de caída.
compare -- comparar ejecuciones de benchmark
chimeraforge compare --baseline run1.json --candidate run2.json,run3.json --jsonEmpareja configuraciones por (modelo, backend, quant, carga de trabajo, context_length); calcula deltas de rendimiento/TTFT/duración con un resumen agregado de mejora/regresión.
refit -- actualizar coeficientes del planificador
chimeraforge refit --bench-dir ./results/ --output fitted_models.json --validateMezcla bayesiana (ponderación de confianza por clave), offsets de hardware, reajuste de ley de potencia y una suite de validación de 10 comprobaciones que bloquea la escritura (--validate).
report -- generar informes
chimeraforge report --results-dir ./results/ --format markdown --output report.mdMarkdown (compatible con GitHub) y HTML autocontenido y seguro contra XSS; análisis estadístico (RMSE, MAE, MAPE, R^2) con tablas de percentiles por configuración.
mcp -- servir el planificador a asistentes de IA
chimeraforge mcpEjecuta el servidor MCP stdio descrito arriba. Requiere pip install "chimeraforge[mcp]".
Qué se modela
Dimensión | Cómo se calcula | Procedencia |
VRAM / KV-cache | Primeros principios a partir de la arquitectura real del modelo; sharding consciente de KV-quant y TP/PP | exacto |
Concurrencia máxima | Secuencias limitadas por KV-cache por GPU | exacto |
Rendimiento (decode) | Búsqueda medida, si no, techo de ancho de banda; curva de continuous-batching; comunicaciones TP / burbuja PP | medido / estimado |
TTFT (prefill) | Limitado por cómputo, GPU FP16 TFLOPS x MFU | estimado |
Calidad | Búsqueda compuesta medida, estimación por prior de familia, o desconocida | medido / estimado / desconocido |
Coste | GPU $/h x tamaño de flota (invariante de $/1M-tok en el número de réplicas) | exacto |
Energía | kWh mensuales impulsados por TDP, $/1M-tok (+energía), tok/s-por-vatio | estimado |
Seguridad | Búsqueda de tasa de rechazo TR134/TR142 (puerta opcional) | medido / desconocido |
Hardware: 22 GPUs -- Ada + Blackwell de consumo (series RTX 30/40/50), centro de datos (A100 40/80GB, H100, H200, B200, L4, T4) y AMD MI300X -- cada una con VRAM, ancho de banda, FP16 TFLOPS, TDP e interconexión (NVLink/Infinity Fabric/PCIe).
Limitaciones conocidas (honestas): La decodificación especulativa aún no está modelada. El caché de prefijos modela el ahorro de prefill pero no el ahorro de KV (deliberadamente conservador). Los tokens de razonamiento están modelados, pero la proporción es tu entrada (--reasoning-tokens), nunca inferida. Para MoE, los parámetros activos-vs-totales sí están modelados, pero el paralelismo de expertos y el desequilibrio de carga de enrutamiento no lo están. La cobertura de cuantización para vLLM/TGI es FP16 + FP8 (AWQ/GPTQ aún no); la calidad FP8 se estima, no se mide. El rendimiento de TP y PP son estimaciones modeladas por comunicaciones, no medidas, y no pueden combinarse en un mismo plan. El encolado es analítico (consciente de la varianza), no un simulador de eventos discretos. El corpus incluido se ajusta principalmente en un solo equipo (RTX 4080 12GB); otras GPUs escalan desde ancho de banda/cómputo hasta que measure en la tuya. El servidor MCP es solo stdio (Claude Code/Desktop, Cursor local) -- aún no hay transporte remoto alojado.
Lo que decidió la investigación
La Fase 2 (TR123-TR133, ~106,000 mediciones) se destiló en un marco de despliegue respaldado por artefactos -- las mismas reglas que aplica el planificador:
Decisión | Recomendación | Evidencia |
Backend de agente único | Ollama Q4_K_M | Mayor rendimiento por dólar; calidad dentro de -4.1pp (TR123-TR125) |
Backend multiagente (N>=4) | vLLM FP16 | Ventaja de 2.25x del continuous batching (TR130-TR132) |
Política de compilación | Solo prefill, Linux, Inductor+Triton | Aceleración del 24-60%; decode falla 100% (TR126) |
Cuantización | Q4_K_M por defecto; Q8_0 crítico para calidad; nunca Q2_K | Punto óptimo universal en 5 modelos (TR125) |
Presupuesto de contexto | Ollama para >4K tokens en 12 GB | Desbordamiento de VRAM = caídas de 25-105x (TR127) |
Planificación de capacidad |
| R^2>=0.859 validado; supera a M/D/1 por 20.4x (TR133) |
Cribado de seguridad |
| Tasa de rechazo + riesgo RTSI por configuración; rechaza celdas que colapsan la seguridad (TR134/TR142) |
Hallazgos principales (datos completos en los TR): Rust supera a Python en agente único (+15.2% de rendimiento, -58% TTFT, -67% de memoria -- TR112); Ollama dual alcanza un paralelismo multiagente casi perfecto (~99%) frente al 82.2% en una sola instancia (TR110/TR113/TR114); el continuous batching de vLLM ofrece una ventaja de 2.25x en N=8, limitado por el ancho de banda de la memoria de la GPU, no por la pila (TR130-TR132).
Investigación completa: docs/archive/technical_reports.md indexa los 32 informes; el archivo completo con metodología y referencias de datos brutos vive en outputs/publish_ready/reports/.
Cómo se generan los números
~204,000 mediciones primarias en 32 informes técnicos (TR108-TR137 + la procedencia de seguridad TR142/TR146), en un RTX 4080 Laptop (12 GB). De-duplicadas: TR137/TR142 son síntesis de datos ya contados.
Rigor: aislamiento de proceso limpio por ejecución (sin sesgo de caché caliente), arranques en frío forzados, 3-5 ejecuciones por configuración para confianza estadística, registro estructurado JSON/CSV con procedencia completa. Cada afirmación se remonta a datos brutos que puedes re-ejecutar.
Contexto del programa: ChimeraForge es el empalme CLI accionable del programa padre Banterhearts (~1,337,000 mediciones primarias + de juez en 54 TRs); la investigación de superficie de ataque de seguridad y pila de servicio vive en repositorios hermanos.
549 pruebas automatizadas (
pytest tests/) cubren los modelos del planificador, búsqueda de puertas, resolvedor, descubrimiento, seguridad, backends de bench y el servidor MCP -- desacopladas de GPU, sin backend en vivo requerido para la suite principal.
Reproduce cualquier número: encuentra la afirmación en un informe bajo outputs/publish_ready/reports/, sigue su referencia a la carpeta de datos, inspecciona el CSV/JSON y re-ejecuta los scripts o notebooks proporcionados. Consulta docs/archive/methodology.md.
Estructura del repositorio
Ruta | Contenido |
| El CLI |
| Benchmarking, monitorización y perfilado de agentes en Python |
| Implementaciones de agente único y multiagente en Rust (Tokio + 4 runtimes alternativos) |
| Archivo canónico de TR (TR108-TR137) + síntesis -- empieza aquí para hallazgos |
| Guías, referencia de API e índice de informes técnicos -- empieza aquí para el cómo |
| Andamiaje de reproducción, líneas base y artefactos brutos de benchmark |
Documentación
docs/README.md-- índice de documentación y navegacióndocs/quick_start.md-- primera ejecución de benchmark (Python + Rust)docs/API.md-- referencia de API de Python para el paquetedocs/archive/technical_reports.md-- índice de los 32 informes técnicosdocs/archive/dual_ollama_setup.md-- requerido para reproducir resultados multiagentedocs/archive/methodology.md/docs/archive/rust_vs_python.md-- metodología y la comparación completa de lenguajes
Contribuciones
Contribuciones bienvenidas -- consulta CONTRIBUTING.md. Buenas áreas: configuraciones de benchmark adicionales, nuevas estrategias de optimización, más modelos/hardware, documentación y herramientas de análisis.
Licencia
MIT -- consulta LICENSE.
Agradecimientos
Realizado como parte del Programa de Investigación de Rendimiento LLM de Banterhearts: la Fase 1 (TR108-TR122) estableció la metodología de medición y la comparación entre lenguajes, la Fase 2 (TR123-TR133) produjo el marco de despliegue y el planificador de capacidad, y la Fase 3 (TR134-TR137) midió el coste de seguridad de la optimización de inferencia -- ahora la puerta de seguridad opcional del planificador.
Repositorio: https://github.com/Sahil170595/Chimeraforge - PyPI: https://pypi.org/project/chimeraforge/ - Estado: Beta, en desarrollo activo
Available Tools
5 toolschimeraforge_compare_apiA
Compare self-hosting against the hosted APIs for a workload: sizes the cheapest feasible GPU fleet, prices the same traffic through each API model, and gives the monthly output-token volume where the two break even. Use for 'is it cheaper to self-host or use the API', 'when does a GPU pay for itself'. API prices come from a dated snapshot -- the result reports its age and flags it when stale; say so rather than quoting an old price as current.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | ||
| hardware | Yes | ||
| duty_cycle | No | ||
| model_size | No | 8b | |
| request_rate | No | ||
| allow_network | No | ||
| prompt_tokens | No | ||
| context_length | No | ||
| latency_slo_ms | No | ||
| quality_target | No | ||
| reasoning_tokens | No | ||
| avg_output_tokens | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations available, the description carries the full behavioral burden and does it well. It discloses that API prices come from a dated snapshot, that the result reports its age and flags staleness, and instructs the agent not to present old prices as current. This is a meaningful, non-obvious behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: the first explains what the tool computes, the second gives real user intents, and the third warns about stale pricing. The outcome is well structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete on purpose, behavior, and the staleness caveat, but it does not cover the 12-parameter input surface or any expected return format beyond the three high-level outputs. For a tool with no output schema and no annotations, that leaves a meaningful completeness gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and there are 12 parameters, so the description needed to compensate. It only implies concepts like workload and traffic without explaining properties such as hardware, model, duty_cycle, latency_slo_ms, quality_target, or avg_output_tokens. An agent would struggle to set these correctly based solely on this description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific action (compare), a specific subject (self-hosting vs. hosted APIs), and the three concrete outputs: cheapest feasible GPU fleet sizing, price comparison for identical traffic, and break-even token volume. This clearly differentiates it from siblings like 'plan' and 'suggest'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit, user-quoted use cases: 'is it cheaper to self-host or use the API' and 'when does a GPU pay for itself'. It does not explicitly name alternatives or say when to use a sibling tool instead, so it stops slightly short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
chimeraforge_list_hardwareA
List known GPUs with VRAM/bandwidth/TDP/interconnect.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It does not disclose any behavioral traits such as output format, size limits, or whether it reflects real-time data. The simple listing implies read-only behavior but this is not explicitly stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, focused sentence that directly states the core action and output characteristics. There is no wasted text, and it is immediately understandable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple, parameterless listing tool with no output schema, the description is adequately complete. It identifies the resource (GPUs) and the key attributes (VRAM/bandwidth/TDP/interconnect). However, it could marginally benefit from a note on data recency or ordering, but this is not critical given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline score is 4. The description correctly implies no input is needed. Since there are no parameters, no additional semantic information is required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'List known GPUs' with specific attributes (VRAM/bandwidth/TDP/interconnect). This is a specific verb+resource and clearly distinguishes from siblings like chimeraforge_plan and chimeraforge_resolve_model.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. It does not mention any context, prerequisites, or exclusions. The description is purely functional with no usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
chimeraforge_planA
Recommend the best (model x quantization x backend x GPU-count) deployment for a workload, or report why nothing fits. Returns candidates with per-number provenance (measured/extrapolated/estimated/unknown). Use for: 'what GPU do I need for ', 'will fit on ', 'how many GPUs for N req/s', 'what will it cost'. Set platform (linux/windows/wsl2/macos) to the deployment OS: engines are offered only where their own docs say they run.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | online | |
| cloud | No | ||
| model | No | ||
| hardware | Yes | ||
| kv_quant | No | fp16 | |
| platform | No | ||
| workload | No | steady | |
| lora_rank | No | ||
| duty_cycle | No | ||
| model_size | No | 3b | |
| grid_region | No | ||
| lora_target | No | qv | |
| tpot_slo_ms | No | ||
| ttft_slo_ms | No | ||
| quality_from | No | ||
| request_rate | No | ||
| think_time_s | No | ||
| allow_network | No | ||
| allow_offload | No | ||
| gpu_overrides | No | ||
| lora_adapters | No | ||
| prompt_tokens | No | ||
| safety_target | No | ||
| session_turns | No | ||
| context_length | No | ||
| latency_slo_ms | No | ||
| quality_target | No | ||
| model_revisions | No | ||
| tensor_parallel | No | ||
| budget_usd_month | No | ||
| reasoning_tokens | No | ||
| avg_output_tokens | No | ||
| pipeline_parallel | No | ||
| use_contributions | No | ||
| host_bandwidth_gbps | No | ||
| gpu_price_multiplier | No | ||
| prefix_cache_hit_rate | No | ||
| max_num_batched_tokens | No | ||
| unified_memory_fraction | No | ||
| carbon_intensity_g_per_kwh | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does disclose two genuine behavioral traits: results come back labeled with per-number provenance (measured/extrapolated/estimated/unknown), which tells the agent how much to trust each figure, and engine availability is gated by the platform's own documentation. It stops short of stating the obvious-but-unconfirmed fact that this is a pure analysis call with no side effects, and says nothing about cost, latency, or what happens when nothing fits beyond a short reason.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences with the core capability front-loaded, followed by return-shape and use-case routing, then a parameter caveat. No filler or repetition. The platform sentence is somewhat tacked on at the end and could be grouped with the framing, but nothing is wasteful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 40-parameter tool with no annotations and no output schema, the description should be doing far more work than it is. It usefully characterizes the return value via provenance labels even without an output schema, but it leaves the semantics of nearly every input undocumented, gives no prerequisite or scoping guidance, and never states the invocation cost of such a large search space.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are 40 parameters at 0% schema description coverage, and the description only gives meaning to one of them (platform). High-impact inputs such as gpu_overrides, allow_offload, use_contributions, quality_target, gpu_price_multiplier, prefix_cache_hit_rate, duty_cycle, safety_target, and the parallel/SLO knobs are left completely opaque, so an agent has little basis for filling them meaningfully rather than relying on defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The first sentence names a specific verb (recommend) and a well-defined resource (model x quantization x backend x GPU-count deployment), and even states the failure mode ('or report why nothing fits') plus the return shape (candidates with provenance). It is very clear what the tool does, but it never names or contrasts with the ambiguous sibling chimeraforge_suggest, leaving the agent to infer the boundary from context alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Use for:' list gives four concrete query shapes ('what GPU do I need for <model>', 'will <model> fit on <gpu>', 'how many GPUs for N req/s', 'what will it cost'), which is strong intent-routing guidance. It also gives an actionable platform rule (set platform to the deployment OS; engines are offered only where their docs say they run). What is missing is any explicit when-not-to-use or named alternative, so the agent must guess whether a 'what GPU do I need' question belongs here or in chimeraforge_suggest.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
chimeraforge_resolve_modelB
Resolve a model id to real params/architecture (grounds hallucinated specs).
| Name | Required | Description | Default |
|---|---|---|---|
| model | Yes | ||
| hf_revision | No | ||
| allow_network | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It hints that results are grounded/real rather than generated, but omits critical behavior: that allow_network defaults to true (this call may hit the network), how hf_revision affects resolution, what happens on failure, and whether results are cached.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; the core action and purpose come first. The parenthetical is slightly jargon-y but earns its place by conveying the anti-hallucination intent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no annotations and no output schema, the description should explain more about the return shape ('params/architecture' is only a gesture) and the network/revision behavior of the tool. Adequate as a minimum but leaves meaningful gaps for a 3-param resolution tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 3 parameters, so the description must compensate. It only loosely implies what 'model' means (a model id) and says nothing about hf_revision or allow_network, leaving two of three parameters undocumented in both schema and description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb+resource: 'Resolve a model id to real params/architecture' clearly states the transformation performed. The parenthetical '(grounds hallucinated specs)' reinforces intent. It is distinguishable from siblings like list_hardware or compare_api, though it never names them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The parenthetical implies the use case (verify model specs rather than trust hallucinated ones), which is a soft usage signal. However, it gives no when-not guidance, no prerequisites, and never points to an alternative among the siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
chimeraforge_suggestA
Rank the models that actually fit and hit the SLO on a given GPU -- the inverse of planning. Use for 'what can I run on a 4090', 'best model for 12GB'. Sources: catalog (offline curated set), ollama (locally installed), hf (top Hub repos).
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| source | No | catalog | |
| hardware | Yes | ||
| hf_limit | No | ||
| ollama_url | No | ||
| request_rate | No | ||
| context_length | No | ||
| latency_slo_ms | No | ||
| quality_target | No | ||
| budget_usd_month | No | ||
| avg_output_tokens | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the behavioral disclosure burden. It communicates core behavior: ranking models against a fit/SLO objective and drawing from three sources: catalog, ollama, and hf. However, it does not disclose external interactions (e.g., reaching out to Ollama or Hugging Face), side effects, failure modes, or whether the operation is read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences carry the core behavior, user-facing examples, and source list without redundancy. The most important action is front-loaded: 'Rank the models...' instead of burying the intent in filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers core purpose and high-level sources, which is enough to make a reasonable first call with the one required parameter. However, with 11 parameters, no output schema, no annotation safety clues, and 0% parameter documentation coverage, the description omits too much detail to fully steer an agent through meaningful variations of source, SLO, latency, and quality trade-offs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, but it only adds meaning to the source concept ('catalog', 'ollama', 'hf') and general context around SLO. The other 10 parameters, including latency_slo_ms, request_rate, quality_target, hf_limit, and budget_usd_month, are left to the agent to infer from their names and defaults, which is risky for a tool with this much configuration surface.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and object: 'Rank the models that actually fit and hit the SLO on a given GPU.' It clearly identifies itself as the inverse of planning, which separates it from its most likely sibling confusion. The example use cases ('what can I run on a 4090', 'best model for 12GB') make the intent immediately understandable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides concrete usage framing with quoted examples and explicitly contrasts it with planning. It would be stronger if it also stated when not to use it or pointed to the sibling tools, but the 'inverse of planning' hint gives the agent enough context to route appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
2 tool updates
v0.65.0- Changed
chimeraforge_plan1 field changed- added
Input schema / properties / model_revisionsAdded value: +{ + "anyOf": [ + { + "additionalProperties": { + "type": "string" + }, + "type": "object" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Model Revisions" +}
- Changed
chimeraforge_resolve_model1 field changed- added
Input schema / properties / hf_revisionAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Hf Revision" +}
1 tool update
v0.50.0- Changed
chimeraforge_plan4 fields changed- added
Input schema / properties / cloudAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Cloud" +} - added
Input schema / properties / session_turnsAdded value: +{ + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Session Turns" +} - added
Input schema / properties / think_time_sAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Think Time S" +} - added
Input schema / properties / use_contributionsAdded value: +{ + "default": false, + "title": "Use Contributions", + "type": "boolean" +}
1 tool update
v0.46.0- Changed
chimeraforge_plan6 fields changed- added
Input schema / properties / carbon_intensity_g_per_kwhAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Carbon Intensity G Per Kwh" +} - added
Input schema / properties / grid_regionAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Grid Region" +} - added
Input schema / properties / latency_slo_ms / anyOfAdded value: +[ + { + "type": "number" + }, + { + "type": "null" + } +] - changed
Input schema / properties / latency_slo_ms / defaultPrevious value: -5000New value: +null - removed
Input schema / properties / latency_slo_ms / typeRemoved value: -"number" - added
Input schema / properties / modeAdded value: +{ + "default": "online", + "title": "Mode", + "type": "string" +}
1 tool update
v0.43.0- Changed
chimeraforge_plan2 fields changed- added
Input schema / properties / platformAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Platform" +} - added
Input schema / properties / unified_memory_fractionAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Unified Memory Fraction" +}
1 tool update
v0.34.0- Changed
chimeraforge_plan3 fields changed- added
Input schema / properties / gpu_overridesAdded value: +{ + "anyOf": [ + { + "additionalProperties": true, + "type": "object" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Gpu Overrides" +} - added
Input schema / properties / max_num_batched_tokensAdded value: +{ + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Max Num Batched Tokens" +} - added
Input schema / properties / quality_fromAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Quality From" +}
3 tool updates
v0.30.0- Added
chimeraforge_compare_api - Changed
chimeraforge_plan10 fields changed- added
Input schema / properties / allow_offloadAdded value: +{ + "default": false, + "title": "Allow Offload", + "type": "boolean" +} - added
Input schema / properties / gpu_price_multiplierAdded value: +{ + "default": 1, + "title": "Gpu Price Multiplier", + "type": "number" +} - added
Input schema / properties / host_bandwidth_gbpsAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Host Bandwidth Gbps" +} - added
Input schema / properties / lora_adaptersAdded value: +{ + "default": 0, + "title": "Lora Adapters", + "type": "integer" +} - added
Input schema / properties / lora_rankAdded value: +{ + "default": 16, + "title": "Lora Rank", + "type": "integer" +} - added
Input schema / properties / lora_targetAdded value: +{ + "default": "qv", + "title": "Lora Target", + "type": "string" +} - added
Input schema / properties / safety_targetAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Safety Target" +} - added
Input schema / properties / tpot_slo_msAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Tpot Slo Ms" +} - added
Input schema / properties / ttft_slo_msAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Ttft Slo Ms" +} - added
Input schema / properties / workloadAdded value: +{ + "default": "steady", + "title": "Workload", + "type": "string" +}
- Added
chimeraforge_suggest
3 tool updates
- First observed
chimeraforge_list_hardware - First observed
chimeraforge_plan - First observed
chimeraforge_resolve_model
TDQS
Scored across 5 tools
Each tool targets a distinct task: model metadata lookup (resolve_model), GPU metadata lookup (list_hardware), forward planning (plan), API-vs-self-host cost analysis (compare_api), and inverse ranking (suggest). The only potential overlap is plan vs suggest, but the description explicitly frames suggest as 'the inverse of planning,' resolving ambiguity.
All names share the consistent chimeraforge_ prefix and mostly use a verb_noun pattern (resolve_model, list_hardware, compare_api). Two entries are bare verbs (plan, suggest), a minor deviation that stays readable and predictable.
Five tools is well-scoped for a GPU/model deployment advisor: model resolution, hardware lookup, forward planning, cost comparison, and inverse suggestion each cover a distinct need. No tool feels redundant or missing at the structural level.
The surface covers the core advisory lifecycle: resolving models, listing hardware, planning deployments, comparing self-host vs API cost, and reverse-fitting models to GPUs. Minor gaps exist (e.g., no general catalog browsing or pricing-refresh beyond the dated snapshot noted in compare_api), but agents can work around them.
Maintenance
Related MCP Connectors
LLM serving capacity planner: VRAM, KV cache, GPU topology, latency, and cost.
Does an LLM fit on your GPU? VRAM, KV cache and GPU count for any Hugging Face model.
Can this LLM run on my GPU? VRAM, speed ceiling and what fits instead, for any model and GPU.
Will this LLM fit on your GPU, multi-GPU rig or Mac? Exact VRAM & KV-cache math. Read-only.
Related MCP Servers
- AlicenseAqualityDmaintenanceEstimates GPU requirements, training/inference costs, and cloud-vs-on-prem TCO for AI workloads using deterministic calculators.121MIT
- AlicenseNot gradedqualityCmaintenanceToken cost math for LLM API calls: current per-million-token rates for 69 models across 17 providers, with local arithmetic for estimates, comparisons and monthly budgets. Rates are verified and date-stamped.44 npm2MIT
- AlicenseAqualityAmaintenanceEvidence-backed private AI deployment intelligence for GPU and LLM inference. Search benchmark evidence, check deployment fit, predict performance, get hardware recommendations, and generate launch configurations.5GPL 3.0
- AlicenseAqualityAmaintenanceReal-time cluster health monitoring, pre-request NLMS latency prediction, and intelligent prompt routing across multi-instance LLM backends (vLLM, Ollama, SGLang, TGI).43Apache 2.0