MFlowy Data Modeling Analysis Workflow

MFlowy is an MCP-native modular ML workflow engine: DAG-based layered execution, configuration-driven, supporting traceable and auditable workflows from data processing and model training to visualization analysis.

Core Features

• MCP-native: All capabilities are exposed as MCP tools (pyfunc), and the same set of tools can be invoked through three entry points—MCP server (stdio), JSON runner CLI (cmd), and direct import (host embedding)

• DAG workflow orchestration: Supports complex directed acyclic graph (DAG) topologies, with flexible definition of task dependencies

• Configuration-driven: Define workflows through YAML configuration files without writing code

• Decorator registration: Pure functions + @handler decorator for automatic registration; adding new functionality only requires creating a .py file

• Middleware system: Chain-of-responsibility pattern for handling cross-cutting concerns (data injection, mlflow logging, fail-fast on errors, various logging)

• Data lineage: MLflow tag (mflowy.input_steps) automatically records input dependencies between steps

• Observable execution: Full MLflow tracking (parameters/metrics/models/artifacts) + structured WorkflowResult (per-node run_id/status/output); process logs bound to stderr, color-coded by level in terminal mode

• Extensible execution strategy: JobProvider contract abstracts the execution environment of compute tools, with a built-in local implementation; remote execution is integrated via custom implementations (see docs/REMOTE_MODELING.md)

Related MCP server: Procesio MCP Server

Architecture and Invocation Methods

MFlowy uses MCP as its architectural backbone: src/mflowy/mcp/tools.py defines all compute tools (eda / modeling / explanation / predict / inverse_optimization, etc.), invoked through three entry points:

CLI (cmd) is the command-line channel for the MCP tool layer. It shares the same tool implementations and JobProvider delegation as the MCP server—it is not a separate architecture; the legacy standalone CLI (mflowy run/validate/list-modules, etc.) has been deprecated.

Quick Start

# 构建 wheel（→ dist/mflowy-<version>-py3-none-any.whl）
make build-whl

# MCP server (stdio) — 完全体（数据分析 + 建模）；<whl> 为 wheel 绝对路径（见下方说明）
uvx --index-strategy unsafe-best-match \
    --default-index https://mirrors.aliyun.com/pypi/simple/ \
    --index https://download.pytorch.org/whl/cpu \
    --from "mflowy[modeling] @ file://<whl绝对路径>" \
    mcpSrv

# JSON runner（CLI）— 本地调试
uv run cmd list_modules                                       # 查看支持的步骤及模块列表（base，无数据栈）
uv run cmd list_modules '{"step":"load"}'                     # 查看 load 步骤的模块列表
uv run cmd get_module_info '{"step":"load","module":"csv"}'   # 查看 load 步骤的 csv 模块信息
uv run --extra stats cmd data_profile '{"file_path": "..."}'           # 数据分析工具
uv run --extra modeling cmd modeling '{"modeling_steps_yaml": "...", "name": "...", "desc": "..."}'  # 建模工具

Environment Variables

MCP Client Configuration Example

{
  "mcpServers": {
    "mflowy": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "--index-strategy", "unsafe-best-match",  // torch CPU 索引必需（见下方说明）
        "--default-index", "https://mirrors.aliyun.com/pypi/simple/",  // 可选：包索引镜像（网络可达 PyPI 时省略）
        "--index", "https://download.pytorch.org/whl/cpu",  // [modeling] 需要；仅 [stats] 可省略
        "--from", "mflowy[modeling] @ file:///path/to/mflowy-<version>-py3-none-any.whl",
        "mcpSrv"
      ],
      "env": {
        // "PYTHONPATH": "/path/to/custom_module_parent",
        // "MFLOWY_JOB_PROVIDER": "<custom_module>:<class>",
        // "MLFLOW_TRACKING_URI": "postgresql://user:pwd@host:5432/mlflow"
      }
    }
  }
}

Startup Notes

• Entry point name mcpSrv: Deliberately avoids the identically named mcp CLI bundled with the mcp SDK (mcp.cli:app)—uvx may resolve the mcp command to the SDK side, causing startup failure

• extras inlined in the --from spec: uvx's --extra requires a newer uv version; the inline form has the best compatibility

• <whl> is an absolute path: make build-whl produces dist/mflowy-<version>-py3-none-any.whl; version changes must be kept in sync

• torch CPU index (--index pytorch-cpu + --index-strategy unsafe-best-match) is required for [modeling]: uvx does not read [tool.uv.sources] in pyproject; without it, torch resolves to the full CUDA bundle (2–3GB); unsafe-best-match must be used together with the pytorch index, otherwise the first-index strategy will fail to resolve due to the older requests version on that index

• Development mode can skip the wheel: .mcp.json.example in the repository root uses source-path form and is always up to date

Telemetry

Diagnostic collection for MCP tool invocations, consent-based, defaults to ask; when the endpoint is unreachable it is fully transparent and does not affect tool invocation, covering only the MCP entry point. Privacy contract: see PRIVACY.md; integration and configuration details: see docs/TELEMETRY.md.

Contributing

Contributions of any kind are welcome (features, fixes, documentation, examples). Please read CONTRIBUTING.md (development process and conventions), CODE_OF_CONDUCT.md, PRIVACY.md (telemetry privacy contract), and SECURITY.md (vulnerability disclosure).

License

This project is open-sourced under the MIT License.

Documentation

• Architecture and Contribution Guide

• Architecture Decision Records

• Research Flow Methodology

• Roadmap

• Remote Execution Integration

• Telemetry

• DAG Engine Architecture