model_sleep
Suspend a vLLM inference engine to free GPU memory between request bursts and stop serving until wake. Use level 1 to offload weights to CPU RAM, level 2 to discard them, or dry_run to preview.
Instructions
[WRITE][risk=high] Suspend the engine via Sleep Mode (it stops serving requests).
Frees GPU memory between bursts. level=1 offloads the weights to CPU RAM and wakes fast; level=2 discards them, so waking reloads from disk. The engine serves nothing until model_wake. Pass dry_run=True to preview.
Sleep Mode exists only on servers started with VLLM_SERVER_DEV_MODE=1; on any other server this reports that the route is absent rather than failing vaguely.
Args: level: 1 to offload weights to CPU RAM, 2 to discard them (default 1). dry_run: If True, preview without suspending. target: Inference target name from config; omit for the default.
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "model_sleep"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| level | No | ||
| target | No | ||
| dry_run | No |