set_placement
Assign specific GPUs to AI model roles (chat, embed, rerank, vision) by setting a placement spec that persists to config. Control device indices and tensor splits for manual multi-GPU allocation.
Instructions
Set and apply a manual multi-GPU placement spec (persists to config).
The spec maps a role ("chat"/"embed"/"rerank"/"vision") to a placement, e.g.
{"chat": {"devices": [0, 1], "tensor_split": [1, 1]}}. devices is the
GPU indices (get_gpus lists them); tensor_split is optional per-device
weights (omit for an even split). Omit a role to leave it auto-placed.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| spec | Yes |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||