kv_cache_stats
Retrieves KV-cache utilization, prefix-cache hit rate, and preemption count to monitor inference cluster health and diagnose performance issues.
Instructions
[READ] KV-cache utilisation, prefix-cache hit rate, and preemption count.
Args: target: Inference target name from config; omit for the default.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| target | No |