queue_depth
Monitor the number of running versus waiting requests to detect backpressure in GPU inference clusters.
Instructions
[READ] Running vs waiting requests — the leading backpressure signal.
Args: target: Inference target name from config; omit for the default.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| target | No |