kv_cache_stats
Read KV-cache utilization, prefix-cache hit rate, and preemption counts for a vLLM/Ray inference target to diagnose GPU memory pressure and scheduling issues.
Instructions
[READ] KV-cache utilisation, prefix-cache hit rate, and preemption count.
Args: target: Inference target name from config; omit for the default.
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "kv_cache_stats"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| target | No |