start_streaming
Launch a parallel pipeline that processes each scan independently through cortical tiles, config, and embeddings stages, then combines results after all scans complete.
Instructions
Launch scan-centric streaming pipeline: one worker per scan runs stages 2-4 in parallel.
Each worker owns one ScanId and processes cortical_tiles → config → embeddings sequentially for its scan, using file-presence barriers between stages. Stage 5 (combine) runs once after all workers drain.
Requires embeddings_only mode (training aggregates all subjects and cannot be parallelised per-scan).
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| bids | No | ||
| dataset | Yes | ||
| dry_run | No | ||
| input_dir | Yes | ||
| n_workers | No | ||
| output_dir | Yes | ||
| short_name | No | eval | |
| sk_qc_path | No | ||
| models_path | No | ||
| datasets_root | No | ||
| path_to_graph | Yes | ||
| poll_interval | No | ||
| worker_timeout | No | ||
| embeddings_path | No | champollion_V1 | |
| path_sk_with_hull | Yes | ||
| dataset_localization | No | local |
Output Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||