chimeraforge
Chimeraforge
Ein lokaler, modell-agnostischer LLM-Bereitstellungsplaner. Er verwandelt „welches Modell, welche Quantisierung, welche GPU und welches Backend – wie viele, passt es, erreicht es mein SLO, was kostet es" in eine schnelle, ehrliche, gemessene Antwort – aus deiner Shell, deinem Python oder deinem KI-Assistenten.
uvx chimeraforge plan --model-size 8b --hardware "RTX 4090 24GB"Das Vertrauensprinzip
Jede Zahl ist als measured, estimated oder unknown gekennzeichnet, und das Tool weigert sich, diejenigen zu fälschen, hinter denen es nicht stehen kann. VRAM und KV-Cache werden aus der realen Architektur eines Modells berechnet (exakt). Durchsatz ist ein gemessener Lookup, wenn verfügbar, andernfalls eine explizite Bandbreiten-Roofline-Schätzung – niemals als Daten präsentiert, die es nicht sind. Qualität unterhalb des gebündelten Korpus meldet unknown, nicht eine erfundene Punktzahl. Ein Plan mit 0 Ergebnissen nennt das genaue Gate, das jeden Kandidaten abgelehnt hat, statt eines generischen „nichts gefunden". Keine Telemetrie, kein Phone-Home, funktioniert air-gapped.
Gib ihm ein Modell – eine Größenklasse, ein Hugging-Face-Repository, ein Ollama-Tag oder manuelle Overrides für ein unveröffentlichtes Modell – und er durchsucht den (Modell x Quantisierung x Backend x GPU-Anzahl x Tensor/Pipeline-Parallelismus)-Raum gegen VRAM, Qualität, Latenz, Kosten, Energie und ein optionales Sicherheits-Gate und gibt dann die günstigste Konfiguration zurück, die dein SLO erfüllt.
12 Befehle, ein Tool: plan - suggest - measure - validate - catalog - safety - bench - eval - compare - refit - report - mcp.
Der empirische Korpus geht auf die Technical Reports TR108-TR137 zurück (~204.000 reale Messungen auf Consumer-GPUs). Siehe CHANGELOG für die vollständige Feature-Historie.
Related MCP server: llm-cost
Installation
Teste es ohne Installation:
uvx chimeraforge plan --model-size 8b --hardware "RTX 4090 24GB"
pipx run chimeraforge plan --model-size 8b --hardware "RTX 4090 24GB"Richtig installieren:
pip install chimeraforge # planner + model resolution (HF/Ollama) + suggest/measure/safety/bench
pip install chimeraforge[bench] # + GPU environment metadata for benchmarks (pynvml)
pip install chimeraforge[mcp] # + MCP server so Claude/GPT/Cursor can call the planner
pip install chimeraforge[eval] # + quality evaluation (BERTScore, ROUGE-L)
pip install chimeraforge[refit] # + coefficient refitting (numpy, scipy)
pip install chimeraforge[all] # everythingPython 3.10+. Die Kerninstallation umfasst den Planer und die netzwerkfähigen Befehle (httpx ist eine Kern-Abhängigkeit). plan / suggest / catalog laufen vollständig offline; bench / measure / safety benötigen ein laufendes Backend (Ollama, vLLM oder TGI). Windows / macOS / Linux.
Schnellstart
# Plan a registry size class on your GPU
chimeraforge plan --model-size 8b --hardware "RTX 4090 24GB" --request-rate 2.0
# Plan ANY model -- a Hugging Face repo or an Ollama tag
chimeraforge plan --model Qwen/Qwen2.5-7B-Instruct --hardware "RTX 4090 24GB"
chimeraforge plan --model ollama:qwen3:14b --ollama-url http://localhost:11434
# Split a model too big for one GPU across several (tensor parallelism)
chimeraforge plan --model meta-llama/Llama-3.3-70B-Instruct --hardware "H100 80GB" --tp 4
# Shrink the KV-cache, print the cost/latency/quality trade-off menu
chimeraforge plan --model-size 8b --hardware "RTX 4080 12GB" --kv-quant q8 --pareto
# Benchmark a live model and plan on the MEASURED numbers
chimeraforge plan --model qwen3:14b --measure
# Discover + rank what fits your GPU and budget
chimeraforge suggest --source ollama --hardware "RTX 4090 24GB" --budget 500MCP-Server – gib Claude / GPT / Cursor dieselben Zahlen
GPU-Sizing ist genau der Punkt, an dem Assistenten versagen: Hardware-Preise und -Spezifikationen aus dem Trainings-Cutoff sowie fehleranfällige KV-Cache/Batching-Arithmetik aus dem Gedächtnis. chimeraforge mcp startet einen stdio-MCP-Server, sodass ein Assistent den echten Planer gegen gemessene Daten aufruft, statt zu raten.
pip install "chimeraforge[mcp]"Claude Code:
claude mcp add --transport stdio chimeraforge -- uvx --from "chimeraforge[mcp]" chimeraforge mcpClaude Desktop / Cursor (zur MCP-Konfigurationsdatei hinzufügen):
{
"mcpServers": {
"chimeraforge": {
"command": "uvx",
"args": ["--from", "chimeraforge[mcp]", "chimeraforge", "mcp"]
}
}
}Das --from "chimeraforge[mcp]" zieht das MCP-SDK hinzu; uvx startet den Server in einer eigenständigen Umgebung. Wenn du bereits pip install "chimeraforge[mcp]" in die Umgebung installiert hast, die dein Client startet, kannst du stattdessen "command": "chimeraforge", "args": ["mcp"] verwenden.
Stellt drei Tools bereit: chimeraforge_plan (die vollständige Gate-Suche), chimeraforge_resolve_model (verankert eine Modell-ID in ihren realen Parametern/Architektur) und chimeraforge_list_hardware. Jedes Ergebnis trägt dieselbe measured / estimated / unknown-Herkunft wie die CLI, und die Tool-Beschreibungen sagen dem Modell, diese seinem eigenen Wissen vorzuziehen. chimeraforge_plan gibt außerdem ein launch-Feld zurück – den Serve-Befehl für die empfohlene Konfiguration –, sodass der Assistent „und wie führe ich es aus" beantworten kann, ohne Flags zu erfinden.
Befehle
plan – prädiktiver Kapazitätsplaner
chimeraforge plan --model-size 8b --hardware "RTX 4090 24GB" --request-rate 2.0
chimeraforge plan --model Qwen/Qwen2.5-7B-Instruct --hardware "RTX 4090 24GB" # any HF repo
chimeraforge plan --model ollama:qwen3:14b --ollama-url http://localhost:11434 # any Ollama tag
chimeraforge plan --model meta-llama/Llama-3.3-70B-Instruct --hardware "H100 80GB" --tp 4 # multi-GPU
chimeraforge plan --model-size 3b --kv-quant q4 --pareto # smaller KV cache, trade-off menu
chimeraforge plan --model-size 8b --hardware "RTX 4090 24GB" --launch # + the serve command to actually run it
chimeraforge plan --model-size 3b --workload agent --safety-target 0.85 --jsonPlant jedes Modell: Registry-Größenklasse, HF-Repo (
org/name), Ollama-Tag oder manuelle Overrides (--params-b/--n-layers/...).Durchsucht den (Modell x Quantisierung x Backend x N-Replikate x Batch/GPU)-Raum durch eine 5-Gate-Pipeline: VRAM -> Qualität -> Sicherheit (optional) -> Latenz -> Budget.
Modelliert reale Serving-Physik: Continuous Batching (vLLM/TGI), Prefill/Decode-Aufteilung (TTFT + TPOT), KV-Cache-gebundene Nebenläufigkeit und varianzbewusstes Queueing (
--workload).Passt Modelle, die zu groß für eine GPU sind:
--tensor-parallel/--tp {N|auto}verteilt Gewichte + KV auf N GPUs (Megatron-Stil, mit Kommunikationsmodellierung);--pipeline-parallel/--pp {N|auto}teilt stattdessen Schichten auf N Stufen auf (günstiger bei langsamen Interconnects, benötigt Batching, um die Pipeline zu füllen). Noch nicht kombinierbar.Serviert, was das Backend serviert: GGUF-Quantisierungen werden auf Ollama angeboten; vLLM/TGI erhalten FP16 und FP8 (nur auf GPUs mit FP8-Tensor-Cores – Ada/Hopper/Blackwell/CDNA3). Der Planer schlägt kein GGUF-Checkpoint auf vLLM mehr vor, das mit einem llama.cpp-Speedup bepreist wird.
KV-Cache-Quantisierung (
--kv-quant {fp16,q8,q4}) verkleinert den Cache und erhöht die maximale Nebenläufigkeit – größter Gewinn bei langem Kontext.Kostenrealismus (
--duty-cycle,--gpu-price-multiplier): Die Schlagzeilen-$/1M-tok-Preise setzen eine gesättigte Flotte voraus. Du zahlst auch für bereitgestellte Reserve und für jede Leerlaufstunde, daher liegt der effektive Wert bei einem 8B bei 2 req/s bei $2,71/1M bei voller Auslastung und $9,04/1M bei 30 %, gegenüber $0,92 bei Kapazität. Spot-/Reserved-Preise sind deine Eingabe, keine gebündelte Schätzung.Self-Host vs. API-Break-even (
--compare-api): bepreist deine Workload gegen gehostete APIs und meldet das monatliche Volumen, ab dem Self-Hosting sich lohnt. Preise sind ein datierter Schnappschuss mit einer Quell-URL pro Anbieter, nach 90 Tagen als veraltet markiert – niemals als Live-Angebot präsentiert – und ein Frontier-API wird als andere Qualitätsstufe gekennzeichnet, statt als gleichwertig ausgegeben zu werden.Prefix-Caching (
--prefix-cache-hit-rate): Chatbot- und Agent-Traffic nutzt ein langes System-Prompt wieder, daher ist der meiste Prefill bereits gecacht. Bei einem 4k-Prompt und einer Trefferquote von 90 % sinkt die TTFT eines 8B von 166 ms auf 17 ms. Standardwert 0 und wird nie inferiert, und das KV, das ein gemeinsames Präfix spart, wird bewusst nicht abgezogen – KV zu klein zu dimensionieren ist das, was aus „es passt" einen OOM macht.Reasoning-Modelle (
--reasoning-tokens N): Versteckte Denk-Tokens werden von der GPU dekodiert und im KV gehalten, auch wenn der Aufrufer sie nie sieht. Nur sichtbare Ausgabe zu zählen, unterschätzt den Decode um das Reasoning-Verhältnis – 1000 versteckte Tokens brachten in unserem eigenen Check einen 8B-Plan von 363 ms auf 6128 ms p95. Standardwert 0 und wird nie inferiert: Das Verhältnis ist eine Eigenschaft deiner Workload, nicht der Gewichte.Aufmerksamkeitsform-bewusstes KV: MLA (DeepSeek-V2/V3) cached ein komprimiertes Latent statt K/V pro Kopf – es als GQA zu dimensionieren, überschätzt DeepSeek-V3s Cache um 57x – und Sliding-Window-Modelle stoppen das Cache-Wachstum nach dem Fenster. Ein Fenster, dessen Schichtmuster nicht deklariert ist, wird nicht angewendet, denn KV zu klein zu dimensionieren ist das, was aus „es passt" einen OOM macht.
Mixture-of-Experts-bewusst: VRAM wird auf Gesamt-Parametern dimensioniert (jeder Experte bleibt resident), während Durchsatz und TTFT aktive Parameter verwenden (ein Token liest nur die Experten, zu denen es geroutet wird). Ein MoE-Modell als dicht zu behandeln, unterschätzt seinen Durchsatz um 3,6x bei Mixtral-8x7B und ~18x bei DeepSeek-V3. Aktive Zählungen werden aus der realen Experten-Geometrie des Modells abgeleitet und stimmen mit veröffentlichten Werten überein.
Energie (
--electricity-rate): monatliche kWh-Kosten,$/1M-tok (+energy)und tok/s-pro-Watt, separat (nicht eingerechnet) zum Budget-Gate gemeldet.Launch-Befehl-Export (
--launch): gibt denvllm serve/ollama run/ TGIdocker run-Befehl für die Gewinnerkonfiguration aus, mit der eigenen Kontextlänge des Plans, TP/PP-Grad, Batch-Größe und KV-Datentyp ausgefüllt – die Flags, die fehleranfällig von Hand zu berechnen sind. Es erfindet nichts, was es nicht ableiten kann: Ein GGUF-Quantisierungslevel wird zu einer Notiz, das native äquivalente Checkpoint zu servieren, nicht zu einem erfundenen--quantization-Flag.Pro-Vorhersage-Herkunft (
measured/estimated/unknown); erklärt das bindende Gate, wenn nichts passt.Auf Registry-Daten validiert: VRAM R^2=0,968, Durchsatz R^2=0,859, Qualität RMSE=0,062, Latenz MAPE=1,05 % (schlägt analytisches M/D/1 um 20,4x, TR133). Kein ML – empirische Lookup-Tabellen mit First-Principles-Interpolation (Roofline für Modelle außerhalb der Registry).
suggest – Modelle entdecken & bewerten
chimeraforge suggest --source ollama --hardware "RTX 4090 24GB" --budget 500
chimeraforge suggest --source hf --hf-limit 8 --hardware "RTX 4080 12GB"
chimeraforge suggest --source catalog --hardware "RTX 4080 12GB" # offline, after `catalog --build`Zieht Kandidaten aus einem Live-Ollama (/api/tags), dem HF Hub (Top-Textgenerierung) und/oder dem lokalen Katalog; löst jeden zu realen Parametern/Architektur auf, führt dieselbe Gate-Suche aus und zeigt die beste Konfiguration pro Modell.
measure – live benchmarken, mit echten Zahlen planen
chimeraforge measure --model qwen3:14b --ollama-url http://localhost:11434
chimeraforge plan --model qwen3:14b --measure # measure then plan in one stepBenchmarkt das Live-Modell (echter N=1-Durchsatz, Servicezeit, Nebenläufigkeitsskalierung) und führt es in einen lokalen Korpus ein. plan / suggest bevorzugen dann automatisch die gemessenen Zahlen (Herkunft wechselt zu measured).
catalog – lokaler Modellkatalog
chimeraforge catalog --build # resolve a curated seed (+ --with-ollama) and cache specs
chimeraforge catalog # list the cached catalogPersistiert aufgelöste Spezifikationen, sodass suggest --source catalog einen bekannten, guten Satz vollständig offline bewertet.
safety – Live-Refusal-Screening
chimeraforge safety --model llama3.2-3b --prompts harmful.txt --quant Q4_K_M --safety-target 0.85Wo plan --safety-target aus gebündelten TR134/TR142-Daten entscheidet, misst safety: Es führt deine Probe-Prompts gegen ein Live-Modell aus, klassifiziert Refusals (regelbasiert – die TR134-Regex-Baseline), meldet die gemessene Refusal-Rate gegen die gebündelten Gate-Daten (erwartet, Drift, RTSI-Risikostufe) und beendet mit Exit-Code 1 unter --safety-target. Du stellst die Prompts bereit (--prompts, eine pro Zeile) – kein Angriffskorpus wird mit dem Paket ausgeliefert; richte es auf HarmBench / AdvBench / deinen eigenen Satz. Benötigt ein laufendes Ollama.
bench – Live-Inferenz-Benchmarking
chimeraforge bench --model llama3.2-3b --runs 5
chimeraforge bench --model llama3.2-3b --all-quants --context 512,1024,2048,4096 --json
chimeraforge bench --model llama3.2-3b --backend vllm --base-url http://localhost:8000Drei Workload-Profile (einzeln / Batch / Server-Poisson); misst Durchsatz, TTFT und Latenz mit p50/p90/p95/p99; CV-basierte Stabilitätswarnungen; JSON-Ausgabe.
eval – Qualitätsbewertung
chimeraforge eval --task general_knowledge --json
chimeraforge eval --predictions preds.txt --references refs.txt --model llama3.2-3bMetriken: Exact Match, ROUGE-L (LCS-Fallback), BERTScore, Kohärenz -> Composite (0.2*EM + 0.3*ROUGE + 0.3*BERT + 0.2*Kohärenz). Qualitätsstufen aus TR125; 3 eingebaute Aufgaben (general_knowledge, summarization, code). Übergib --fp16-baseline, um die Drop-Stufe zu klassifizieren.
compare – Benchmark-Läufe vergleichen
chimeraforge compare --baseline run1.json --candidate run2.json,run3.json --jsonMatcht Konfigurationen nach (Modell, Backend, Quantisierung, Workload, Kontextlänge); berechnet Durchsatz-/TTFT-/Dauer-Deltas mit einer aggregierten Verbesserungs-/Regressions-Zusammenfassung.
refit – Planer-Koeffizienten aktualisieren
chimeraforge refit --bench-dir ./results/ --output fitted_models.json --validateBayessches Blending (schlüsselweise Konfidenzgewichtung), Hardware-Offsets, Power-Law-Nachanpassung und eine Validierungssuite mit 10 Checks, die den Write absichert (--validate).
report – Berichte generieren
chimeraforge report --results-dir ./results/ --format markdown --output report.mdMarkdown (GitHub-kompatibel) und eigenständiges, XSS-sicheres HTML; statistische Analyse (RMSE, MAE, MAPE, R^2) mit Per-Konfiguration-Perzentiltabellen.
mcp – den Planer für KI-Assistenten bereitstellen
chimeraforge mcpStartet den oben beschriebenen stdio-MCP-Server. Erfordert pip install "chimeraforge[mcp]".
Was modelliert wird
Dimension | So berechnet | Herkunft |
VRAM / KV-Cache | First-Principles aus realer Architektur; KV-Quant und TP/PP-bewusst | exakt |
Maximale Parallelität | KV-Cache-gebundene Sequenzen pro GPU | exakt |
Durchsatz (Decode) | Gemessener Lookup, sonst Bandbreiten-Roofline; Continuous-Batching-Kurve; TP-Kommunikation / PP-Overhead | gemessen / geschätzt |
TTFT (Prefill) | Compute-gebunden, GPU-FP16-TFLOPS x MFU | geschätzt |
Qualität | Gemessener Composite-Lookup, Familien-Prior-Schätzung oder unbekannt | gemessen / geschätzt / unbekannt |
Kosten | GPU-$/h x Flottengröße ($/1M-Tokens invariant bei Replikatanzahl) | exakt |
Energie | TDP-basiert, monatliche kWh, $/1M-Tokens (+Energie), Tokens/s-pro-Watt | geschätzt |
Sicherheit | TR134/TR142-Refusal-Rate-Lookup (Opt-in-Gate) | gemessen / unbekannt |
Hardware: 22 GPUs – Consumer-Ada + Blackwell (RTX 30/40/50-Serie), Rechenzentrum (A100 40/80GB, H100, H200, B200, L4, T4) und AMD MI300X – jeweils mit VRAM, Bandbreite, FP16-TFLOPS, TDP und Interconnect (NVLink/Infinity Fabric/PCIe).
Bekannte Grenzen (ehrlich): Speculative Decoding ist noch nicht modelliert. Prefix Caching modelliert die Prefill-Ersparnis, aber nicht die KV-Ersparnis (bewusst konservativ). Reasoning-Tokens werden modelliert, aber das Verhältnis ist Ihre Eingabe (--reasoning-tokens), nie abgeleitet. Bei MoE werden aktive-vs-gesamte Parameter modelliert, aber Expertenparallelismus und Lastverteilungs-Routing sind es nicht. Quant-Abdeckung für vLLM/TGI ist FP16 + FP8 (AWQ/GPTQ noch nicht); FP8-Qualität ist geschätzt, nicht gemessen. TP- und PP-Durchsatz sind kommunikationsmodellierte Schätzungen, nicht gemessen, und können nicht in einem Plan kombiniert werden. Queueing ist analytisch (varianzbewusst), kein diskreter Ereignissimulator. Der gebündelte Korpus ist primär auf einem Rig (RTX 4080 12GB) gefittet; andere GPUs skalieren über Bandbreite/Compute, bis Sie auf Ihrer eigenen messen. Der MCP-Server ist nur stdio-fähig (Claude Code/Desktop, lokaler Cursor) – noch kein gehosteter Remote-Transport.
Was die Forschung entschieden hat
Phase 2 (TR123–TR133, ~106.000 Messungen) destilliert in ein artefaktgestütztes Deployment-Framework – dieselben Regeln, die der Planer anwendet:
Entscheidung | Empfehlung | Belege |
Single-Agent-Backend | Ollama Q4_K_M | Höchster Durchsatz pro Dollar; Qualität innerhalb von -4,1pp (TR123–TR125) |
Multi-Agent-Backend (N>=4) | vLLM FP16 | 2,25-facher Vorteil durch Continuous Batching (TR130–TR132) |
Compile-Richtlinie | Nur Prefill, Linux, Inductor+Triton | 24–60 % Beschleunigung; Decode-Abstürze 100 % (TR126) |
Quantisierung | Q4_K_M Standard; Q8_0 qualitätskritisch; nie Q2_K | Universeller Sweet Spot über 5 Modelle (TR125) |
Kontextbudget | Ollama für >4K Tokens bei 12 GB | VRAM-Überlauf = 25–105-fache Einbrüche (TR127) |
Kapazitätsplanung |
| Validierter R²>=0,859; übertrifft M/D/1 um das 20,4-fache (TR133) |
Sicherheits-Screening |
| Refusal-Rate + RTSI-Risiko pro Konfiguration; lehnt sicherheitskritische Zellen ab (TR134/TR142) |
Kernergebnisse (volle Daten in den TRs): Rust schlägt Python im Single-Agent-Betrieb (+15,2 % Durchsatz, -58 % TTFT, -67 % Speicher – TR112); duales Ollama erreicht nahezu perfekte Multi-Agent-Parallelität (~99 %) gegenüber 82,2 % auf einer Instanz (TR110/TR113/TR114); vLLMs Continuous Batching bringt einen 2,25-fachen Vorteil bei N=8, begrenzt durch GPU-Speicherbandbreite, nicht durch den Stack (TR130–TR132).
Vollständige Forschung: docs/archive/technical_reports.md indexiert alle 32 Berichte; das vollständige Archiv mit Methodik und Rohdaten-Referenzen liegt in outputs/publish_ready/reports/.
So entstehen die Zahlen
~204.000 Primärmessungen über 32 technische Berichte (TR108–TR137 plus die TR142/TR146-Sicherheits-Herkunft), auf einem RTX-4080-Laptop (12 GB). Dedupliziert: TR137/TR142 sind Synthesen bereits gezählter Daten.
Strenge Methodik: Frischer Prozess pro Lauf (kein Warm-Cache-Bias), erzwungene Kaltstarts, 3–5 Läufe pro Konfiguration für statistische Konfidenz, strukturiertes JSON/CSV-Logging mit vollständiger Herkunft. Jede Behauptung führt auf Rohdaten zurück, die Sie erneut ausführen können.
Programmkontext: ChimeraForge ist die handlungsorientierte CLI-Variante des übergeordneten Banterhearts-Programms (~1.337.000 Primär- + Judge-Messungen über 54 TRs); die Sicherheits-Angriffsflächen- und Serving-Stack-Forschung lebt in Schwester-Repos.
549 automatisierte Tests (
pytest tests/) decken die Planer-Modelle, Gate-Suche, Resolver, Discovery, Sicherheit, Bench-Backends und den MCP-Server ab – GPU-entkoppelt, kein Live-Backend für die Kern-Suite erforderlich.
Reproduzieren Sie jede Zahl: Finden Sie die Behauptung in einem Bericht unter outputs/publish_ready/reports/, folgen Sie der Referenz zum Datenordner, prüfen Sie das CSV/JSON und führen Sie die bereitgestellten Skripte oder Notebooks erneut aus. Siehe docs/archive/methodology.md.
Repository-Struktur
Pfad | Inhalt |
| Die |
| Python-Agent-Benchmarking, Monitoring, Profiling |
| Rust-Single- und Multi-Agent-Implementierungen (Tokio + 4 alternative Laufzeiten) |
| Kanonisches TR-Archiv (TR108–TR137) + Synthesen – hier zuerst für Ergebnisse |
| Anleitungen, API-Referenz und der technische Berichtsindex – hier zuerst für Anleitungen |
| Reproduktionsgerüst, Baselines und rohe Benchmark-Artefakte |
Dokumentation
docs/README.md– Dokumentationsindex und Navigationdocs/quick_start.md– Erster Benchmark-Lauf (Python + Rust)docs/API.md– Python-API-Referenz für das Paketdocs/archive/technical_reports.md– Index aller 32 technischen Berichtedocs/archive/dual_ollama_setup.md– Voraussetzungen zur Reproduktion der Multi-Agent-Ergebnissedocs/archive/methodology.md/docs/archive/rust_vs_python.md– Methodik und der vollständige Sprachvergleich
Mitwirken
Beiträge willkommen – siehe CONTRIBUTING.md. Gute Bereiche: zusätzliche Benchmark-Konfigurationen, neue Optimierungsstrategien, weitere Modelle/Hardware, Dokumentation und Analysetools.
Lizenz
MIT – siehe LICENSE.
Danksagungen
Durchgeführt im Rahmen des Banterhearts-LLM-Performance-Forschungsprogramms: Phase 1 (TR108–TR122) etablierte die Messmethodik und den sprachübergreifenden Vergleich, Phase 2 (TR123–TR133) erzeugte das Deployment-Framework und den Kapazitätsplaner, und Phase 3 (TR134–TR137) maß die Sicherheitskosten der Inferenzoptimierung – heute das Opt-in-Sicherheitsgate des Planers.
Repository: https://github.com/Sahil170595/Chimeraforge – PyPI: https://pypi.org/project/chimeraforge/ – Status: Beta, aktiv entwickelt
Available Tools
5 toolschimeraforge_compare_apiA
Compare self-hosting against the hosted APIs for a workload: sizes the cheapest feasible GPU fleet, prices the same traffic through each API model, and gives the monthly output-token volume where the two break even. Use for 'is it cheaper to self-host or use the API', 'when does a GPU pay for itself'. API prices come from a dated snapshot -- the result reports its age and flags it when stale; say so rather than quoting an old price as current.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | ||
| hardware | Yes | ||
| duty_cycle | No | ||
| model_size | No | 8b | |
| request_rate | No | ||
| allow_network | No | ||
| prompt_tokens | No | ||
| context_length | No | ||
| latency_slo_ms | No | ||
| quality_target | No | ||
| reasoning_tokens | No | ||
| avg_output_tokens | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations available, the description carries the full behavioral burden and does it well. It discloses that API prices come from a dated snapshot, that the result reports its age and flags staleness, and instructs the agent not to present old prices as current. This is a meaningful, non-obvious behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: the first explains what the tool computes, the second gives real user intents, and the third warns about stale pricing. The outcome is well structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete on purpose, behavior, and the staleness caveat, but it does not cover the 12-parameter input surface or any expected return format beyond the three high-level outputs. For a tool with no output schema and no annotations, that leaves a meaningful completeness gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and there are 12 parameters, so the description needed to compensate. It only implies concepts like workload and traffic without explaining properties such as hardware, model, duty_cycle, latency_slo_ms, quality_target, or avg_output_tokens. An agent would struggle to set these correctly based solely on this description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific action (compare), a specific subject (self-hosting vs. hosted APIs), and the three concrete outputs: cheapest feasible GPU fleet sizing, price comparison for identical traffic, and break-even token volume. This clearly differentiates it from siblings like 'plan' and 'suggest'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit, user-quoted use cases: 'is it cheaper to self-host or use the API' and 'when does a GPU pay for itself'. It does not explicitly name alternatives or say when to use a sibling tool instead, so it stops slightly short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
chimeraforge_list_hardwareA
List known GPUs with VRAM/bandwidth/TDP/interconnect.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It does not disclose any behavioral traits such as output format, size limits, or whether it reflects real-time data. The simple listing implies read-only behavior but this is not explicitly stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, focused sentence that directly states the core action and output characteristics. There is no wasted text, and it is immediately understandable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple, parameterless listing tool with no output schema, the description is adequately complete. It identifies the resource (GPUs) and the key attributes (VRAM/bandwidth/TDP/interconnect). However, it could marginally benefit from a note on data recency or ordering, but this is not critical given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline score is 4. The description correctly implies no input is needed. Since there are no parameters, no additional semantic information is required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'List known GPUs' with specific attributes (VRAM/bandwidth/TDP/interconnect). This is a specific verb+resource and clearly distinguishes from siblings like chimeraforge_plan and chimeraforge_resolve_model.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. It does not mention any context, prerequisites, or exclusions. The description is purely functional with no usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
chimeraforge_planA
Recommend the best (model x quantization x backend x GPU-count) deployment for a workload, or report why nothing fits. Returns candidates with per-number provenance (measured/extrapolated/estimated/unknown). Use for: 'what GPU do I need for ', 'will fit on ', 'how many GPUs for N req/s', 'what will it cost'. Set platform (linux/windows/wsl2/macos) to the deployment OS: engines are offered only where their own docs say they run.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | online | |
| cloud | No | ||
| model | No | ||
| hardware | Yes | ||
| kv_quant | No | fp16 | |
| platform | No | ||
| workload | No | steady | |
| lora_rank | No | ||
| duty_cycle | No | ||
| model_size | No | 3b | |
| grid_region | No | ||
| lora_target | No | qv | |
| tpot_slo_ms | No | ||
| ttft_slo_ms | No | ||
| quality_from | No | ||
| request_rate | No | ||
| think_time_s | No | ||
| allow_network | No | ||
| allow_offload | No | ||
| gpu_overrides | No | ||
| lora_adapters | No | ||
| prompt_tokens | No | ||
| safety_target | No | ||
| session_turns | No | ||
| context_length | No | ||
| latency_slo_ms | No | ||
| quality_target | No | ||
| model_revisions | No | ||
| tensor_parallel | No | ||
| budget_usd_month | No | ||
| reasoning_tokens | No | ||
| avg_output_tokens | No | ||
| pipeline_parallel | No | ||
| use_contributions | No | ||
| host_bandwidth_gbps | No | ||
| gpu_price_multiplier | No | ||
| prefix_cache_hit_rate | No | ||
| max_num_batched_tokens | No | ||
| unified_memory_fraction | No | ||
| carbon_intensity_g_per_kwh | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does disclose two genuine behavioral traits: results come back labeled with per-number provenance (measured/extrapolated/estimated/unknown), which tells the agent how much to trust each figure, and engine availability is gated by the platform's own documentation. It stops short of stating the obvious-but-unconfirmed fact that this is a pure analysis call with no side effects, and says nothing about cost, latency, or what happens when nothing fits beyond a short reason.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences with the core capability front-loaded, followed by return-shape and use-case routing, then a parameter caveat. No filler or repetition. The platform sentence is somewhat tacked on at the end and could be grouped with the framing, but nothing is wasteful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 40-parameter tool with no annotations and no output schema, the description should be doing far more work than it is. It usefully characterizes the return value via provenance labels even without an output schema, but it leaves the semantics of nearly every input undocumented, gives no prerequisite or scoping guidance, and never states the invocation cost of such a large search space.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are 40 parameters at 0% schema description coverage, and the description only gives meaning to one of them (platform). High-impact inputs such as gpu_overrides, allow_offload, use_contributions, quality_target, gpu_price_multiplier, prefix_cache_hit_rate, duty_cycle, safety_target, and the parallel/SLO knobs are left completely opaque, so an agent has little basis for filling them meaningfully rather than relying on defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The first sentence names a specific verb (recommend) and a well-defined resource (model x quantization x backend x GPU-count deployment), and even states the failure mode ('or report why nothing fits') plus the return shape (candidates with provenance). It is very clear what the tool does, but it never names or contrasts with the ambiguous sibling chimeraforge_suggest, leaving the agent to infer the boundary from context alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Use for:' list gives four concrete query shapes ('what GPU do I need for <model>', 'will <model> fit on <gpu>', 'how many GPUs for N req/s', 'what will it cost'), which is strong intent-routing guidance. It also gives an actionable platform rule (set platform to the deployment OS; engines are offered only where their docs say they run). What is missing is any explicit when-not-to-use or named alternative, so the agent must guess whether a 'what GPU do I need' question belongs here or in chimeraforge_suggest.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
chimeraforge_resolve_modelB
Resolve a model id to real params/architecture (grounds hallucinated specs).
| Name | Required | Description | Default |
|---|---|---|---|
| model | Yes | ||
| hf_revision | No | ||
| allow_network | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It hints that results are grounded/real rather than generated, but omits critical behavior: that allow_network defaults to true (this call may hit the network), how hf_revision affects resolution, what happens on failure, and whether results are cached.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; the core action and purpose come first. The parenthetical is slightly jargon-y but earns its place by conveying the anti-hallucination intent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no annotations and no output schema, the description should explain more about the return shape ('params/architecture' is only a gesture) and the network/revision behavior of the tool. Adequate as a minimum but leaves meaningful gaps for a 3-param resolution tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 3 parameters, so the description must compensate. It only loosely implies what 'model' means (a model id) and says nothing about hf_revision or allow_network, leaving two of three parameters undocumented in both schema and description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb+resource: 'Resolve a model id to real params/architecture' clearly states the transformation performed. The parenthetical '(grounds hallucinated specs)' reinforces intent. It is distinguishable from siblings like list_hardware or compare_api, though it never names them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The parenthetical implies the use case (verify model specs rather than trust hallucinated ones), which is a soft usage signal. However, it gives no when-not guidance, no prerequisites, and never points to an alternative among the siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
chimeraforge_suggestA
Rank the models that actually fit and hit the SLO on a given GPU -- the inverse of planning. Use for 'what can I run on a 4090', 'best model for 12GB'. Sources: catalog (offline curated set), ollama (locally installed), hf (top Hub repos).
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| source | No | catalog | |
| hardware | Yes | ||
| hf_limit | No | ||
| ollama_url | No | ||
| request_rate | No | ||
| context_length | No | ||
| latency_slo_ms | No | ||
| quality_target | No | ||
| budget_usd_month | No | ||
| avg_output_tokens | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the behavioral disclosure burden. It communicates core behavior: ranking models against a fit/SLO objective and drawing from three sources: catalog, ollama, and hf. However, it does not disclose external interactions (e.g., reaching out to Ollama or Hugging Face), side effects, failure modes, or whether the operation is read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences carry the core behavior, user-facing examples, and source list without redundancy. The most important action is front-loaded: 'Rank the models...' instead of burying the intent in filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers core purpose and high-level sources, which is enough to make a reasonable first call with the one required parameter. However, with 11 parameters, no output schema, no annotation safety clues, and 0% parameter documentation coverage, the description omits too much detail to fully steer an agent through meaningful variations of source, SLO, latency, and quality trade-offs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, but it only adds meaning to the source concept ('catalog', 'ollama', 'hf') and general context around SLO. The other 10 parameters, including latency_slo_ms, request_rate, quality_target, hf_limit, and budget_usd_month, are left to the agent to infer from their names and defaults, which is risky for a tool with this much configuration surface.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and object: 'Rank the models that actually fit and hit the SLO on a given GPU.' It clearly identifies itself as the inverse of planning, which separates it from its most likely sibling confusion. The example use cases ('what can I run on a 4090', 'best model for 12GB') make the intent immediately understandable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides concrete usage framing with quoted examples and explicitly contrasts it with planning. It would be stronger if it also stated when not to use it or pointed to the sibling tools, but the 'inverse of planning' hint gives the agent enough context to route appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
2 tool updates
v0.65.0- Changed
chimeraforge_plan1 field changed- added
Input schema / properties / model_revisionsAdded value: +{ + "anyOf": [ + { + "additionalProperties": { + "type": "string" + }, + "type": "object" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Model Revisions" +}
- Changed
chimeraforge_resolve_model1 field changed- added
Input schema / properties / hf_revisionAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Hf Revision" +}
1 tool update
v0.50.0- Changed
chimeraforge_plan4 fields changed- added
Input schema / properties / cloudAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Cloud" +} - added
Input schema / properties / session_turnsAdded value: +{ + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Session Turns" +} - added
Input schema / properties / think_time_sAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Think Time S" +} - added
Input schema / properties / use_contributionsAdded value: +{ + "default": false, + "title": "Use Contributions", + "type": "boolean" +}
1 tool update
v0.46.0- Changed
chimeraforge_plan6 fields changed- added
Input schema / properties / carbon_intensity_g_per_kwhAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Carbon Intensity G Per Kwh" +} - added
Input schema / properties / grid_regionAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Grid Region" +} - added
Input schema / properties / latency_slo_ms / anyOfAdded value: +[ + { + "type": "number" + }, + { + "type": "null" + } +] - changed
Input schema / properties / latency_slo_ms / defaultPrevious value: -5000New value: +null - removed
Input schema / properties / latency_slo_ms / typeRemoved value: -"number" - added
Input schema / properties / modeAdded value: +{ + "default": "online", + "title": "Mode", + "type": "string" +}
1 tool update
v0.43.0- Changed
chimeraforge_plan2 fields changed- added
Input schema / properties / platformAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Platform" +} - added
Input schema / properties / unified_memory_fractionAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Unified Memory Fraction" +}
1 tool update
v0.34.0- Changed
chimeraforge_plan3 fields changed- added
Input schema / properties / gpu_overridesAdded value: +{ + "anyOf": [ + { + "additionalProperties": true, + "type": "object" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Gpu Overrides" +} - added
Input schema / properties / max_num_batched_tokensAdded value: +{ + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Max Num Batched Tokens" +} - added
Input schema / properties / quality_fromAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Quality From" +}
3 tool updates
v0.30.0- Added
chimeraforge_compare_api - Changed
chimeraforge_plan10 fields changed- added
Input schema / properties / allow_offloadAdded value: +{ + "default": false, + "title": "Allow Offload", + "type": "boolean" +} - added
Input schema / properties / gpu_price_multiplierAdded value: +{ + "default": 1, + "title": "Gpu Price Multiplier", + "type": "number" +} - added
Input schema / properties / host_bandwidth_gbpsAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Host Bandwidth Gbps" +} - added
Input schema / properties / lora_adaptersAdded value: +{ + "default": 0, + "title": "Lora Adapters", + "type": "integer" +} - added
Input schema / properties / lora_rankAdded value: +{ + "default": 16, + "title": "Lora Rank", + "type": "integer" +} - added
Input schema / properties / lora_targetAdded value: +{ + "default": "qv", + "title": "Lora Target", + "type": "string" +} - added
Input schema / properties / safety_targetAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Safety Target" +} - added
Input schema / properties / tpot_slo_msAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Tpot Slo Ms" +} - added
Input schema / properties / ttft_slo_msAdded value: +{ + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Ttft Slo Ms" +} - added
Input schema / properties / workloadAdded value: +{ + "default": "steady", + "title": "Workload", + "type": "string" +}
- Added
chimeraforge_suggest
3 tool updates
- First observed
chimeraforge_list_hardware - First observed
chimeraforge_plan - First observed
chimeraforge_resolve_model
TDQS
Scored across 5 tools
Each tool targets a distinct task: model metadata lookup (resolve_model), GPU metadata lookup (list_hardware), forward planning (plan), API-vs-self-host cost analysis (compare_api), and inverse ranking (suggest). The only potential overlap is plan vs suggest, but the description explicitly frames suggest as 'the inverse of planning,' resolving ambiguity.
All names share the consistent chimeraforge_ prefix and mostly use a verb_noun pattern (resolve_model, list_hardware, compare_api). Two entries are bare verbs (plan, suggest), a minor deviation that stays readable and predictable.
Five tools is well-scoped for a GPU/model deployment advisor: model resolution, hardware lookup, forward planning, cost comparison, and inverse suggestion each cover a distinct need. No tool feels redundant or missing at the structural level.
The surface covers the core advisory lifecycle: resolving models, listing hardware, planning deployments, comparing self-host vs API cost, and reverse-fitting models to GPUs. Minor gaps exist (e.g., no general catalog browsing or pricing-refresh beyond the dated snapshot noted in compare_api), but agents can work around them.
Maintenance
Related MCP Connectors
LLM serving capacity planner: VRAM, KV cache, GPU topology, latency, and cost.
Does an LLM fit on your GPU? VRAM, KV cache and GPU count for any Hugging Face model.
Can this LLM run on my GPU? VRAM, speed ceiling and what fits instead, for any model and GPU.
Will this LLM fit on your GPU, multi-GPU rig or Mac? Exact VRAM & KV-cache math. Read-only.
Related MCP Servers
- AlicenseAqualityDmaintenanceEstimates GPU requirements, training/inference costs, and cloud-vs-on-prem TCO for AI workloads using deterministic calculators.121MIT
- AlicenseNot gradedqualityCmaintenanceToken cost math for LLM API calls: current per-million-token rates for 69 models across 17 providers, with local arithmetic for estimates, comparisons and monthly budgets. Rates are verified and date-stamped.44 npm2MIT
- AlicenseAqualityAmaintenanceEvidence-backed private AI deployment intelligence for GPU and LLM inference. Search benchmark evidence, check deployment fit, predict performance, get hardware recommendations, and generate launch configurations.5GPL 3.0
- AlicenseAqualityAmaintenanceReal-time cluster health monitoring, pre-request NLMS latency prediction, and intelligent prompt routing across multi-instance LLM backends (vLLM, Ollama, SGLang, TGI).43Apache 2.0