Skip to main content
Glama

YAML Workflow

PyPI version Python versions CI codecov License: MIT

A lightweight workflow engine for CI/CD pipelines, data processing, and DevOps automation. Define reproducible, version-controlled workflows in YAML — run them locally, in CI, or on any machine with Python installed.

Why yaml-workflow?

Most workflow tools require servers, databases, and complex infrastructure. yaml-workflow takes a GitOps approach — workflows are plain YAML files, version-controlled alongside your code:

yaml-workflow

Airflow / Prefect / Dagster

Setup

pip install yaml-workflow

Server, database, scheduler, workers

Configuration

Plain YAML files

Python DAGs + infrastructure config

Dependencies

2 (PyYAML, Jinja2)

50+ packages, Docker, PostgreSQL

Use case

Local automation, scripts, CI/CD, data pipelines

Enterprise orchestration at scale

Learning curve

Minutes

Hours to days

State

File-based, resumable

Database-backed

Choose yaml-workflow when you need:

  • Simple task automation without infrastructure overhead

  • Reproducible pipelines defined in version-controlled YAML

  • Batch processing with parallel execution

  • State persistence and workflow resume after failures

  • A lightweight alternative to shell scripts with better error handling

  • GitOps-friendly pipelines that live in your repo alongside the code

  • A single tool that runs the same pipeline locally and in CI

Related MCP server: Atlas

Features

  • YAML-driven workflow definition with Jinja2 templating

  • Multiple task types: shell, Python, file, template, HTTP, batch

  • Workflow composition via imports — reuse steps across workflows

  • Plugin system via entry points — pip install yaml-workflow-myplugin

  • Watch mode — --watch to re-run on file changes

  • Dry-run mode to preview without executing

  • Workflow visualization (ASCII branching DAG and Mermaid)

  • Parallel execution with configurable worker pools

  • State persistence and resume capability

  • Retry mechanisms with configurable strategies

  • Namespaced variables (args, env, steps, batch)

  • Flow control with custom step sequences and conditions

  • Extensible task system via @register_task decorator

  • Parallel step execution via depends_on — run independent steps concurrently

  • Secrets validation — fail fast if required environment variables are missing

  • Structured output (--format json) for CI integration and scripting

  • MCP server — expose workflows as AI agent tools (pip install yaml-workflow[mcp])

  • Web dashboard — monitor runs and trigger workflows (pip install yaml-workflow[serve])

  • GitHub Action — run workflows in CI with uses: orieg/yaml-workflow@v0.9.3

Use Cases

  • CI/CD pipelines — multi-step build, test, deploy workflows in YAML

  • Data processing — batch ETL pipelines with retry and resume on failure

  • DevOps automation — infrastructure tasks with secrets management and notifications

  • AI/LLM pipelines — orchestrate API calls with auth, retry, and batch processing

  • Local automation — replace shell scripts with reproducible, parameterized workflows

Quick Start

# Install (isolated CLI — recommended)
pipx install yaml-workflow            # Core CLI
pipx install 'yaml-workflow[all]'     # + web dashboard + MCP server

# Or with pip
pip install yaml-workflow

# Initialize example workflows
yaml-workflow init

# Run a workflow with parameters
yaml-workflow run workflows/hello_world.yaml name=Alice

Example workflow (hello_world.yaml):

name: Hello World
description: A simple greeting workflow

params:
  name:
    type: string
    default: World

steps:
  - name: create_greeting
    task: template
    inputs:
      template: "Hello, {{ args.name }}!"
      output_file: greeting.txt

  - name: show_greeting
    task: shell
    inputs:
      command: cat greeting.txt

Visualize workflows

yaml-workflow visualize workflows/data_pipeline.yaml
  Workflow: Data Pipeline

  ┌─────────────────┐
  │  detect_format  │
  │   python_code   │
  └─────────────────┘
           │
           ▼
  ┌────────────────┐  ┌────────────────┐  ┌────────────────┐  ┌────────────────┐
  │  process_json  │  │  process_csv   │  │  process_xml   │  │ handle_unknown │
  │     shell      │  │     shell      │  │     shell      │  │     shell      │
  └────────────────┘  └────────────────┘  └────────────────┘  └────────────────┘
           │
           ▼
  ┌─────────────────┐
  │ generate_report │
  │   python_code   │
  └─────────────────┘

Adjacent conditional steps are automatically grouped as branches. Use --format mermaid to export for docs or GitHub rendering.

Dry-run mode

Preview what a workflow would do without executing anything:

yaml-workflow run workflows/hello_world.yaml name=Alice --dry-run
[DRY-RUN] Workflow: Hello World
[DRY-RUN] Steps: 2 to execute

  [DRY-RUN] Step 'create_greeting' — task: template — WOULD EXECUTE
    template: Hello, Alice!
    output_file: greeting.txt
  [DRY-RUN] Step 'show_greeting' — task: shell — WOULD EXECUTE
    command: cat greeting.txt

[DRY-RUN] Complete. 2 step(s) would execute, 0 would be skipped.
[DRY-RUN] No files were written. No tasks were executed.

Workflow composition

Reuse steps across workflows with imports:

# main.yaml
imports:
  - ./shared/logging_steps.yaml
  - ./shared/common_params.yaml

steps:
  - name: my_step
    task: shell
    inputs:
      command: echo "runs after imported steps"

Imported steps are prepended. Imported params provide defaults that the main workflow can override. Supports transitive imports with circular detection.

Parallel Steps

Run independent steps concurrently with depends_on:

steps:
  - name: fetch_api
    task: http.request
    inputs: {url: "https://api.example.com/data"}

  - name: fetch_db
    task: python_code
    inputs:
      code: "result = query_database()"

  - name: merge
    task: python_code
    depends_on: [fetch_api, fetch_db]
    inputs:
      code: |
        api_data = steps["fetch_api"]["result"]
        db_data = steps["fetch_db"]["result"]
        result = {"merged": True}

Watch mode

Automatically re-run on file changes during development:

yaml-workflow run workflows/hello_world.yaml name=Alice --watch

Monitors the workflow file and all imported files. Press Ctrl+C to stop.

GitHub Actions

Run workflows in CI with the yaml-workflow action:

- name: Run pipeline
  uses: orieg/yaml-workflow@v0.9.3
  id: pipeline
  with:
    workflow: workflows/deploy.yaml
    params: |
      env=production
      version=1.2.0
    format: json

- name: Use results
  run: echo '${{ steps.pipeline.outputs.result }}'

Docker & Kubernetes

Run anywhere without installing Python:

# Run a workflow in Docker
docker run --rm -v $(pwd)/workflows:/app/workflows \
  ghcr.io/orieg/yaml-workflow run /app/workflows/pipeline.yaml

# Start the web dashboard
docker run -p 8080:8080 -v $(pwd)/workflows:/app/workflows \
  ghcr.io/orieg/yaml-workflow

Deploy on Kubernetes with the Helm chart:

helm install my-workflows ./helm/yaml-workflow \
  --set-file workflows.files.pipeline\\.yaml=workflows/pipeline.yaml

Compatible with ArgoCD (GitOps) and Argo Workflows. See the Kubernetes guide.

More commands

# List available workflows
yaml-workflow list

# Validate a workflow (with JSON output for CI)
yaml-workflow validate workflows/hello_world.yaml --format json

# Resume a failed workflow
yaml-workflow run workflows/hello_world.yaml --resume

# Structured output for scripting
yaml-workflow run workflows/pipeline.yaml --format json --output results.json

Documentation

Full documentation is available at orieg.github.io/yaml-workflow.

Contributing

Contributions are welcome! See the Contributing Guide for development setup and guidelines.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Available Tools

4 tools
dry_run_workflowA
Read-onlyIdempotent

Preview what a workflow would do without executing any task. Give a workflow (a name from list_workflows or a file path) and optional params; returns {status, outputs, preview} where preview is a human-readable list of the steps that would run with their resolved inputs (the same information as the CLI's --dry-run). Use this to inspect side effects (shell commands, file writes, HTTP calls) before running for real. It does not execute any task — no shell or Python runs and none of the workflow's own side effects occur; only ephemeral logs are written to a temporary workspace.

Inputs: workflow (required), params.

list_workflowsA
Read-onlyIdempotent

List the workflows available in this server's workflow directory. Returns an object with count and workflows (one entry per workflow, each containing name (its declared name), description, path (the YAML file), and parameters (declared inputs with types and defaults)). Call this first to discover which workflows exist and what inputs each accepts before calling dry_run_workflow or run_workflow. Read-only: it only reads YAML files and never executes anything. Takes no arguments.

Inputs: no named parameters.

run_workflowA
Destructive

Execute a workflow and return its results. Give a workflow (a name from list_workflows or a file path) and optional params; runs it to completion and returns {status, workflow, outputs} where outputs maps each step name to its result. DESTRUCTIVE: a workflow may run arbitrary shell commands and Python, write files, and make HTTP requests — call dry_run_workflow first if you need to preview side effects, and only run workflows you trust.

Inputs: workflow (required), params.

validate_workflowA
Read-onlyIdempotent

Validate a single workflow YAML file without running it. Give the file path; returns {valid, error_count, warning_count, issues[]}, where each issue has a level (error/warning/info), message, and optional line, step, and hint. Use this to check a workflow the agent authored or edited before running it, or to explain why a workflow is malformed. Read-only: no tasks run and nothing is written.

Inputs: path (required).

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4.4/5.0

    Scored across 4 tools

    Disambiguation5/5

    Each tool has a clearly distinct purpose: listing workflows, validating a file, previewing execution without side effects, and actually running a workflow. There is no ambiguity between them.

    Naming Consistency5/5

    All tool names are in snake_case and follow a verb_noun pattern (list_workflows, validate_workflow, dry_run_workflow, run_workflow). The use of 'dry_run' as a compound verb is consistent with the pattern.

    Tool Count4/5

    Four tools is on the lower end but perfectly scoped for this server's purpose—covering discovery, validation, preview, and execution. Each tool earns its place without any redundancy.

    Completeness5/5

    The tool surface covers the full lifecycle of using a YAML workflow: list available workflows, validate a file, preview effects, and execute. There are no obvious gaps for the stated domain of running existing workflows.

    Maintenance

    ActivityNo data
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      B
      maintenance
      A local, auditable multi-model workflow engine that lets you define YAML graphs for orchestrating LLM agents across vendors, with MCP tools for validation, dry-runs, execution, and human approval, all fully observable in a local web interface.
      8
      3
      Apache 2.0
    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables deterministic, step-by-step execution of YAML-defined skills and workflows for AI agents, with progressive disclosure of active-step instructions and tools. It manages workflow state, conditional DAG transitions, loops, and persistent sessions via start_workflow, next_step, and end_workflow.
      MIT