diagnose_engine_latency
Diagnose serving engine latency spikes by correlating queue backpressure, KV cache pressure, and cache locality into a ranked root cause with the tuning knob to adjust.
Instructions
[READ][RCA] Rank the probable cause of a latency spike for any serving engine.
Correlates whichever signals the engine exposes (queue backpressure, KV/token-cache pressure, cache locality) into a ranked cause + the knob to turn — the engine-agnostic counterpart to diagnose_latency_spike (vLLM).
Args: target: Inference target name from config; omit for the default.
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "diagnose_engine_latency"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| target | No |