cost_per_token
Calculate $/1M-token serving cost using live vLLM throughput and GPU hourly cost, returning a forecast when throughput data is missing.
Instructions
[READ] Attribute a $/1M-token unit cost from live vLLM throughput.
Multiplies the current generation-throughput gauge by the supplied GPU
hourly cost to derive the cost of serving 1M tokens; degrades to an
insufficient-data forecast when no throughput metric is present.
Args: gpu_hourly_cost: Hourly cost of a single GPU (e.g. cloud on-demand rate). num_gpus: Number of GPUs backing the deployment; defaults to 1. target: Inference target name from config; omit for the default.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| target | No | ||
| num_gpus | No | ||
| gpu_hourly_cost | Yes |