draft_job
Drafts Slurm job scripts for MUSICA using tested runbook patterns, returning the script and static checks without submitting anything to the cluster.
Instructions
Draft a Slurm job script for MUSICA from the runbook's tested patterns; nothing is sent to the cluster.
kind: cpu (part of a CPU node: cores, mem_gb), cpu-node (whole CPU nodes), gpu (1-3 GPUs on a shared node:
gpus; shared nodes can take hours to start), gpu-node (whole H100 nodes; with vllm_model: vLLM servers on every
node, vllm_servers_per_node 1/2/4 = TP4/TP2/TP1, and command as their client on the first node).
command: the work (shell lines); with nodes > 1 it runs once on every node ($NODE_RANK, $NNODES, $HEAD).
time: minutes or HH:MM:SS; dev=True uses the dev QoS (at most 10 min, 2 nodes): test there first.
array: e.g. 0-9%2 ($TASK in the script). python_env: a venv path (e.g. $DATA/envs/train); container: a .sif
(prefix commands with run); modules: Lmod modules to load.
runs (gpu-node, 1 node): up to 8 commands in parallel on one whole node, run r on GPU r % 4 ($RUN, own caches,
log logs/--r.log; the job fails if any run failed; command runs first as setup); run_minutes:
one estimate per run (refused if the slowest exceeds time).
array_args: one argument string per array task (sets --array=0-(n-1)%throttle; the task gets $ARGS and
$OUT=results/--a); array_throttle: tasks at once (0 = within the approval limits); not
together with array.
vllm_preset: tested serve settings for some models, 'qwen38-fp8' or 'glm53-fp8' (flags for one TP4 server per
node; model and image default to $DATA/models/ and $DATA/containers/, which you must download
first, or set vllm_model/vllm_image; vllm_args are appended, later flags win); load_cap_s: drop a vLLM server whose weights are not loaded this many seconds
after its node's launch (60-3600).
Drafts check at job start that the env, container and model exist (exit 2 otherwise), set HF_HUB_OFFLINE=1 on
GPUs and OMP_NUM_THREADS=1 for vLLM. Returns the script, the static check (estimate, approvals needed) and notes;
then check_job(script_text=...) for sbatch --test-only and the path checks on the cluster.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| dev | No | ||
| gpus | No | ||
| kind | Yes | ||
| name | Yes | ||
| runs | No | ||
| time | No | 00:10:00 | |
| array | No | ||
| cores | No | ||
| nodes | No | ||
| mem_gb | No | ||
| command | No | ||
| modules | No | ||
| container | No | ||
| vllm_args | No | ||
| array_args | No | ||
| load_cap_s | No | ||
| python_env | No | ||
| vllm_image | No | ||
| vllm_model | No | ||
| description | No | ||
| run_minutes | No | ||
| stage_model | No | ||
| vllm_preset | No | ||
| array_throttle | No | ||
| vllm_servers_per_node | No |