configure_inference_server
Create or replace a local inference-server declaration without starting it, selecting managed or existing local mode for server configuration.
Instructions
Create or replace a local inference-server declaration without starting it.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mode | Yes | managed requires workload; existing_local must use loopback. | |
| name | Yes | ||
| model | Yes | ||
| base_url | No | Loopback-only OpenAI-compatible server URL. | http://127.0.0.1:8000 |
| provider | No | vllm | |
| workload | No | Declared workload that starts the server; required for managed mode. | |
| operation | Yes | ||
| tokenizer | No | ||
| quantization | No | ||
| model_revision | No | ||
| benchmark_python | No | Absolute Python launcher for sglang.benchmark.serving; it may differ from the server runtime and is required only for sglang. | |
| tokenizer_revision | No | ||
| expected_configuration_id | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||