mu-pipelines

Open Source Lego like ETL Tool

1stProduct of the week

Visit website

FreeAI Agents16,015 impressions#1 of its week

>log in to comment

Our product is a configuration-driven open source ETL platform where pipelines are defined in simple JSON files—or written in plain English and converted by our AI. It supports ingestion, transformation, data quality, and alerts, with no need for complex architecture or multiple tools. Engineers can mix and match prebuilt "Lego-like" connectors or bring their own code. They choose their runtime (e.g., Python or Spark), infrastructure (on-prem or cloud), sources, destinations, secrets and much more. For migration, users can upload legacy code (SSIS, Informatica, SAS) and our AI agent converts it into clean JSON configurations. Validation blocks help test legacy vs. modern output, once or on a schedule.

AI Agents

Mu-Pipelines is an open-source, configuration-driven ETL platform that builds data pipelines from JSON or plain English.

for: Data engineers and developers who need flexible, code-light ETL/ELT pipelines.
pricing: open source
license: open source

Mosaicsoft-data/mu-pipelines-examples 0 0Pythonupdated 2 years ago

works withAirflowPythonSparkMySQLSalesforceSnowflakeS3

Key features

6 features of mu-pipelines

Modular Architecture — Assemble pipelines from reusable LEGO-like components and built-in connectors.
JSON Configuration — Define entire pipelines in a simple, readable JSON file.
Cloud Agnostic — Deploy pipelines on-prem, in the cloud, or hybrid environments.
Workflow Orchestration — Schedule and chain pipelines using orchestrators like Airflow or ctl-M.
AI-Powered Migration — Convert legacy SSIS, Informatica, or SAS jobs into JSON configs via AI.
Rapid Development Kits — Pre-built solution kits accelerate common use-case implementations.