Deploy a Hugging Face model
deploy_modelServe any model from Hugging Face on a machine of the person's, like a hosted inference API but on their own GPU: vLLM or SGLang for text models, vLLM-Omni or SGLang for image and video models (FLUX, Qwen-Image, Wan), vLLM-Omni for speech and music, llama.cpp for a GGUF file (owner/repo:Q4_K_M). extra_args takes the person's own engine options as they'd paste them. COSTS MONEY like rent_gpu: the machine's hourly price, paid an hour at a time from when it runs until terminate_instance or termination_hours; worst_case_usd is the most it can cost. The model's size is read from Hugging Face and, without node_id, the best-ranked clean machine that holds it is rented on as few GPUs as it allows. A gated or private model needs hf_token (read access is enough; it goes to the machine only and Arrakis keeps none). Returns the instance, the endpoint (an OpenAI-compatible base URL once RUNNING, and its api_key) and ssh_command: the person also gets root SSH. The engine downloads the model first, so it answers a few minutes after RUNNING: get_instance shows endpoint_ready. Same idempotency_key returns the same machine.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | A name you will recognise. | |
| model | Yes | A Hugging Face model id or link, for example Qwen/Qwen3-8B; for llama.cpp, owner/repo:QUANT picks one GGUF file. | |
| engine | No | Default vllm. omni = vLLM-Omni (image, video, speech, audio). | |
| node_id | No | A machine from search_gpus. Default: the one sized for the model. | |
| hf_token | No | A Hugging Face token, for a gated or private model. | |
| gpu_count | No | GPUs of it to rent. Default: the fewest that hold the model. | |
| extra_args | No | The person's own engine options, as they'd paste them: flags, the whole command, or a docker run line (--max-model-len 32768). Read without a shell; the address, key and model stay Arrakis's. | |
| ssh_public_key | No | Your SSH public key. Default: the account's first saved key. | |
| idempotency_key | No | Reuse when retrying. | |
| termination_hours | Yes | Hours until the machine is removed automatically. | |
| max_price_per_gpu_hour | No | The most to pay per GPU-hour (USD). |