cost_per_token
Derives $/1M-token cost from live vLLM throughput and GPU hourly rate; returns insufficient-data forecast when throughput metrics are unavailable.
Instructions
[READ] Attribute a $/1M-token unit cost from live vLLM throughput.
Multiplies the current generation-throughput gauge by the supplied GPU
hourly cost to derive the cost of serving 1M tokens; degrades to an
insufficient-data forecast when no throughput metric is present.
Args: gpu_hourly_cost: Hourly cost of a single GPU (e.g. cloud on-demand rate). num_gpus: Number of GPUs backing the deployment; defaults to 1. target: Inference target name from config; omit for the default.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"gpu_hourly_cost": "<gpu_hourly_cost>"
},
"name": "cost_per_token"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| target | No | ||
| num_gpus | No | ||
| gpu_hourly_cost | Yes |