# PipelineFlow MCP (Lite Edition)

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Protocol: MCP](https://img.shields.io/badge/MCP-2024--11--05-green.svg)](https://modelcontextprotocol.io/)
[![Python: >=3.10](https://img.shields.io/badge/Python->=3.10-yellow.svg)](https://python.org/)

**PipelineFlow MCP** is a lightweight, declarative workflow DAG and data pipeline orchestrator designed for AI agents (Claude Desktop, Cursor, OpenAI Codex, LangGraph).

It enables agents to define multi-step data pipelines, validate dependency acyclicity via topological sorting (Kahn's algorithm), execute transformations predictably, and export visual Mermaid diagrams—without heavy cloud orchestrators like Airflow or Prefect.

---

## 🎯 The Problem

When autonomous AI agents execute multi-step workflows:
1. **Unstructured Command Chains:** Agents run disconnected shell commands or Python scripts with no dependency graph or cycle detection.
2. **Lack of Visual Representation:** Neither humans nor agents can visualize the pipeline topology or inspect intermediate step outputs.
3. **Heavy Orchestrator Bloat:** Enterprise orchestrators (Airflow, Dagster) require databases, workers, and gigabytes of dependencies, making them unsuitable for local agent runtimes.

**PipelineFlow** provides a pure-Python, zero-dependency DAG engine that runs directly inside the MCP protocol.

---

## ✨ Features (Lite Edition)

- 🔄 **Declarative Pipeline DAG:** Add nodes (inputs, filters, transforms, aggregates, outputs) and connect directed edges.
- 🛡️ **Cycle Detection & Topological Sort:** Guarantees deterministic execution order and prevents circular deadlocks using Kahn's algorithm.
- 📊 **Instant Mermaid Diagrams:** Generates clean Mermaid graph diagrams of any pipeline with one tool call.
- ⚡ **Zero-Dependency Engine:** Pure standard library execution with zero cloud setup.

---

## 🚀 Quickstart

### 1. Installation

```bash
git clone https://github.com/johannschiffer/pipelineflow-mcp.git
cd pipelineflow-mcp
pip install -e .
```

### 2. Configure in Claude Desktop or Cursor

Add to your `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "pipelineflow": {
      "command": "pipelineflow",
      "args": ["run"]
    }
  }
}
```

### 3. Run Demo via CLI

```bash
pipelineflow demo
```

---

## 🛠️ MCP Tools Reference

| Tool | Parameters | Description |
|---|---|---|
| `create_pipeline` | `pipeline_id`, `name` | Creates a new pipeline DAG |
| `add_node` | `pipeline_id`, `node_id`, `node_type`, `params` | Adds a node (input, filter, transform, aggregate, custom, output) |
| `connect_nodes` | `pipeline_id`, `from_node`, `to_node` | Connects source node to target node (validates acyclicity) |
| `run_pipeline` | `pipeline_id`, `initial_data` | Executes pipeline in topological dependency order |
| `get_node_output` | `pipeline_id`, `node_id` | Retrieves execution output for any node |
| `export_pipeline_mermaid` | `pipeline_id` | Returns Mermaid diagram string of the pipeline graph |
| `list_pipelines` | *(none)* | Lists all registered pipelines and node counts |

---

## 🚀 Upgrade to PipelineFlow Pro

Need enterprise pipeline caching, parallel branch execution, and interactive HTML dashboards?

The **PipelineFlow Pro Edition** adds:
- ⚡ **Content-Addressable Cache (SQLite WAL):** Caches node execution outputs by SHA-256 hash of inputs, skipping redundant executions.
- 🔀 **Concurrent Branch Runner:** Runs independent DAG branches in parallel using a managed thread pool.
- 🛡️ **Per-Node Retry & Circuit-Breaker Policies:** Configurable exponential backoff and error recovery.
- 🎨 **Interactive HTML Visualizer:** Generates standalone interactive SVG/HTML pipeline dashboards.
- 🐳 **Production Docker & Docker Compose manifests.**
- 🧪 **100% Passing Pytest Suite.**

👉 **[Get PipelineFlow Pro on Gumroad ($14.99 USD)](https://johanld.gumroad.com/l/ykees)**

---

## 📄 License

MIT © 2026 [Johann Schiffer](https://github.com/johannschiffer)