badgr_serve
Deploy a persistent OpenAI-compatible LLM endpoint on GPU compute with required runtime and hourly price caps, returning endpoint URL and deployment IDs until stopped.
Instructions
Provision a persistent OpenAI-compatible endpoint for an LLM model. Returns endpoint_url, deployment_id, receipt_id. The endpoint stays up (billing runs) until badgr_down is called. Hard limits: max_price_per_hour ≤ $5/hr. max_runtime_minutes is REQUIRED — agents must not create open-ended billing. Always confirm with the user before serving.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| gpu | No | GPU type: L40S, A100, H100, RTX_4090 (default: L40S) | L40S |
| name | No | Deployment name (auto-generated if omitted) | |
| model | Yes | HuggingFace model ID, e.g. meta-llama/Llama-3.1-8B-Instruct | |
| region | No | Optional region preference. | |
| gpu_count | No | Number of GPUs (default: 1) | |
| max_price_per_hour | No | Max spend cap in USD/hr (hard limit: $5/hr) | |
| max_runtime_minutes | Yes | Auto-terminate after this many minutes. Required — prevents runaway billing. |