orieg/yaml-workflow
YAML Workflow
A lightweight workflow engine for CI/CD pipelines, data processing, and DevOps automation. Define reproducible, version-controlled workflows in YAML — run them locally, in CI, or on any machine with Python installed.
Why yaml-workflow?
Most workflow tools require servers, databases, and complex infrastructure. yaml-workflow takes a GitOps approach — workflows are plain YAML files, version-controlled alongside your code:
yaml-workflow | Airflow / Prefect / Dagster | |
Setup |
| Server, database, scheduler, workers |
Configuration | Plain YAML files | Python DAGs + infrastructure config |
Dependencies | 2 (PyYAML, Jinja2) | 50+ packages, Docker, PostgreSQL |
Use case | Local automation, scripts, CI/CD, data pipelines | Enterprise orchestration at scale |
Learning curve | Minutes | Hours to days |
State | File-based, resumable | Database-backed |
Choose yaml-workflow when you need:
Simple task automation without infrastructure overhead
Reproducible pipelines defined in version-controlled YAML
Batch processing with parallel execution
State persistence and workflow resume after failures
A lightweight alternative to shell scripts with better error handling
GitOps-friendly pipelines that live in your repo alongside the code
A single tool that runs the same pipeline locally and in CI
Related MCP server: Atlas
Features
YAML-driven workflow definition with Jinja2 templating
Multiple task types: shell, Python, file, template, HTTP, batch
Workflow composition via
imports— reuse steps across workflowsPlugin system via entry points —
pip install yaml-workflow-mypluginWatch mode —
--watchto re-run on file changesDry-run mode to preview without executing
Workflow visualization (ASCII branching DAG and Mermaid)
Parallel execution with configurable worker pools
State persistence and resume capability
Retry mechanisms with configurable strategies
Namespaced variables (
args,env,steps,batch)Flow control with custom step sequences and conditions
Extensible task system via
@register_taskdecoratorParallel step execution via
depends_on— run independent steps concurrentlySecrets validation — fail fast if required environment variables are missing
Structured output (
--format json) for CI integration and scriptingMCP server — expose workflows as AI agent tools (
pip install yaml-workflow[mcp])Web dashboard — monitor runs and trigger workflows (
pip install yaml-workflow[serve])GitHub Action — run workflows in CI with
uses: orieg/yaml-workflow@v0.9.3
Use Cases
CI/CD pipelines — multi-step build, test, deploy workflows in YAML
Data processing — batch ETL pipelines with retry and resume on failure
DevOps automation — infrastructure tasks with secrets management and notifications
AI/LLM pipelines — orchestrate API calls with auth, retry, and batch processing
Local automation — replace shell scripts with reproducible, parameterized workflows
Quick Start
# Install (isolated CLI — recommended)
pipx install yaml-workflow # Core CLI
pipx install 'yaml-workflow[all]' # + web dashboard + MCP server
# Or with pip
pip install yaml-workflow
# Initialize example workflows
yaml-workflow init
# Run a workflow with parameters
yaml-workflow run workflows/hello_world.yaml name=AliceExample workflow (hello_world.yaml):
name: Hello World
description: A simple greeting workflow
params:
name:
type: string
default: World
steps:
- name: create_greeting
task: template
inputs:
template: "Hello, {{ args.name }}!"
output_file: greeting.txt
- name: show_greeting
task: shell
inputs:
command: cat greeting.txtVisualize workflows
yaml-workflow visualize workflows/data_pipeline.yaml Workflow: Data Pipeline
┌─────────────────┐
│ detect_format │
│ python_code │
└─────────────────┘
│
▼
┌────────────────┐ ┌────────────────┐ ┌────────────────┐ ┌────────────────┐
│ process_json │ │ process_csv │ │ process_xml │ │ handle_unknown │
│ shell │ │ shell │ │ shell │ │ shell │
└────────────────┘ └────────────────┘ └────────────────┘ └────────────────┘
│
▼
┌─────────────────┐
│ generate_report │
│ python_code │
└─────────────────┘Adjacent conditional steps are automatically grouped as branches. Use --format mermaid to export for docs or GitHub rendering.
Dry-run mode
Preview what a workflow would do without executing anything:
yaml-workflow run workflows/hello_world.yaml name=Alice --dry-run[DRY-RUN] Workflow: Hello World
[DRY-RUN] Steps: 2 to execute
[DRY-RUN] Step 'create_greeting' — task: template — WOULD EXECUTE
template: Hello, Alice!
output_file: greeting.txt
[DRY-RUN] Step 'show_greeting' — task: shell — WOULD EXECUTE
command: cat greeting.txt
[DRY-RUN] Complete. 2 step(s) would execute, 0 would be skipped.
[DRY-RUN] No files were written. No tasks were executed.Workflow composition
Reuse steps across workflows with imports:
# main.yaml
imports:
- ./shared/logging_steps.yaml
- ./shared/common_params.yaml
steps:
- name: my_step
task: shell
inputs:
command: echo "runs after imported steps"Imported steps are prepended. Imported params provide defaults that the main workflow can override. Supports transitive imports with circular detection.
Parallel Steps
Run independent steps concurrently with depends_on:
steps:
- name: fetch_api
task: http.request
inputs: {url: "https://api.example.com/data"}
- name: fetch_db
task: python_code
inputs:
code: "result = query_database()"
- name: merge
task: python_code
depends_on: [fetch_api, fetch_db]
inputs:
code: |
api_data = steps["fetch_api"]["result"]
db_data = steps["fetch_db"]["result"]
result = {"merged": True}Watch mode
Automatically re-run on file changes during development:
yaml-workflow run workflows/hello_world.yaml name=Alice --watchMonitors the workflow file and all imported files. Press Ctrl+C to stop.
GitHub Actions
Run workflows in CI with the yaml-workflow action:
- name: Run pipeline
uses: orieg/yaml-workflow@v0.9.3
id: pipeline
with:
workflow: workflows/deploy.yaml
params: |
env=production
version=1.2.0
format: json
- name: Use results
run: echo '${{ steps.pipeline.outputs.result }}'Docker & Kubernetes
Run anywhere without installing Python:
# Run a workflow in Docker
docker run --rm -v $(pwd)/workflows:/app/workflows \
ghcr.io/orieg/yaml-workflow run /app/workflows/pipeline.yaml
# Start the web dashboard
docker run -p 8080:8080 -v $(pwd)/workflows:/app/workflows \
ghcr.io/orieg/yaml-workflowDeploy on Kubernetes with the Helm chart:
helm install my-workflows ./helm/yaml-workflow \
--set-file workflows.files.pipeline\\.yaml=workflows/pipeline.yamlCompatible with ArgoCD (GitOps) and Argo Workflows. See the Kubernetes guide.
More commands
# List available workflows
yaml-workflow list
# Validate a workflow (with JSON output for CI)
yaml-workflow validate workflows/hello_world.yaml --format json
# Resume a failed workflow
yaml-workflow run workflows/hello_world.yaml --resume
# Structured output for scripting
yaml-workflow run workflows/pipeline.yaml --format json --output results.jsonDocumentation
Full documentation is available at orieg.github.io/yaml-workflow.
Getting Started - Installation and first workflow
Task Types - Shell, Python, file, template, and batch tasks
Workflow Structure - YAML configuration reference
Templating - Jinja2 variable substitution
State Management - Persistence and resume
Task Development - Creating custom tasks
API Reference - Full API documentation
Contributing
Contributions are welcome! See the Contributing Guide for development setup and guidelines.
License
This project is licensed under the MIT License - see the LICENSE file for details.
Available Tools
4 toolsdry_run_workflowARead-onlyIdempotent
Preview what a workflow would do without executing any task. Give a workflow (a name from list_workflows or a file path) and optional params; returns {status, outputs, preview} where preview is a human-readable list of the steps that would run with their resolved inputs (the same information as the CLI's --dry-run). Use this to inspect side effects (shell commands, file writes, HTTP calls) before running for real. It does not execute any task — no shell or Python runs and none of the workflow's own side effects occur; only ephemeral logs are written to a temporary workspace.
Inputs: workflow (required), params.
list_workflowsARead-onlyIdempotent
List the workflows available in this server's workflow directory. Returns an object with count and workflows (one entry per workflow, each containing name (its declared name), description, path (the YAML file), and parameters (declared inputs with types and defaults)). Call this first to discover which workflows exist and what inputs each accepts before calling dry_run_workflow or run_workflow. Read-only: it only reads YAML files and never executes anything. Takes no arguments.
Inputs: no named parameters.
run_workflowADestructive
Execute a workflow and return its results. Give a workflow (a name from list_workflows or a file path) and optional params; runs it to completion and returns {status, workflow, outputs} where outputs maps each step name to its result. DESTRUCTIVE: a workflow may run arbitrary shell commands and Python, write files, and make HTTP requests — call dry_run_workflow first if you need to preview side effects, and only run workflows you trust.
Inputs: workflow (required), params.
validate_workflowARead-onlyIdempotent
Validate a single workflow YAML file without running it. Give the file path; returns {valid, error_count, warning_count, issues[]}, where each issue has a level (error/warning/info), message, and optional line, step, and hint. Use this to check a workflow the agent authored or edited before running it, or to explain why a workflow is malformed. Read-only: no tasks run and nothing is written.
Inputs: path (required).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 4 tools
Each tool has a clearly distinct purpose: listing workflows, validating a file, previewing execution without side effects, and actually running a workflow. There is no ambiguity between them.
All tool names are in snake_case and follow a verb_noun pattern (list_workflows, validate_workflow, dry_run_workflow, run_workflow). The use of 'dry_run' as a compound verb is consistent with the pattern.
Four tools is on the lower end but perfectly scoped for this server's purpose—covering discovery, validation, preview, and execution. Each tool earns its place without any redundancy.
The tool surface covers the full lifecycle of using a YAML workflow: list available workflows, validate a file, preview effects, and execute. There are no obvious gaps for the stated domain of running existing workflows.
Maintenance
Related MCP Connectors
Workflow planning, recovery checkpoints, coordination, fixtures, and compatibility tools for agents.
Task manager for AI agents. Orchestrate agents to achieve large goals in unified control plane.
Schedule and manage recurring or one-shot tasks
Build, validate and deploy workflows as Cloudflare Workers in your own account, with approval.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables orchestration of MCP tool calls through declarative YAML-defined directed graphs with data transformation, conditional routing, and observable execution flows.28 npm23MIT
- AlicenseAqualityBmaintenanceA local, auditable multi-model workflow engine that lets you define YAML graphs for orchestrating LLM agents across vendors, with MCP tools for validation, dry-runs, execution, and human approval, all fully observable in a local web interface.83Apache 2.0
- AlicenseNot gradedqualityDmaintenanceEnables deterministic, step-by-step execution of YAML-defined skills and workflows for AI agents, with progressive disclosure of active-step instructions and tools. It manages workflow state, conditional DAG transitions, loops, and persistent sessions via start_workflow, next_step, and end_workflow.MIT

ReadyAgentsofficial
AlicenseAqualityAmaintenanceLocal one-shot YAML/JSON agent workflow CLI with tools, approvals, and resume, plus an optional stdio MCP server. Apache-2.0, BYOK.10Apache 2.0