dataflows 开源项目推荐（26 个）

RaftLib /RaftLib

RaftLib C++ 库，通过类似 C++ iostream 的运算符实现流式/数据流并发

intel /dffml

The easiest way to use Machine Learning. Mix and match underlying ML libraries and data set sources. Generate new datasets or modify existing ones with ease.

Graph-COM /LayerDAG

[ICLR 2025 Spotlight] LayerDAG: A Layerwise Autoregressive Diffusion Model of Directed Acyclic Graphs

ekote /Build-Your-First-End-to-End-Lakehouse-Solution

Big Data CONFERENCE Europe 2025 🇱🇹 AI, Cloud and Data Conference - Build Your First End-to-End Lakehouse Solution (aka.ms/fabconlake)

johnneycao /PowerBI-Open-Data-for-fun

This repository serves as a personal collection of Power BI Dashboard and Report sample files, along with the datasets used in the samples, tutorials, and other related content.

tristan-hyams /RabbitMQ.Dataflows

A set of libraries for rapidly developing RabbitMQ workflows/pipelines.

andreacasalino /DynamicFlow

Templated modern C++ library for implementing DataFlow programming

atlasofdata /atlasofdata.github.io

How flows of data reveal the ecosystems of our planet

ewiger /hql

HQL — Hyper Query Language — is a statically typed language for querying, transforming, connecting and computing over data, knowledge and AI workflows.

rosaia /pypeworks

Easy and lightweight parallelized dataflows in Python

SugamDewan /Fabric-COVID-Reporting-Project

An end-to-end data pipeline in Microsoft Fabric to process and analyze COVID-19 data.

vitorio /tertile-generominos-agentic-ai

Custom agentic AI Generominos cards

dataflows-au /dataflows-sms-claude-plugin

DataFlows SMS plugin for Claude: send, schedule and track business SMS from your DataFlows account

DarrenDavy12 /Flight-Times-ETL-Batch-Processing-Pipeline-with-Azure-Data-Factory

This project demonstrates the development of a robust batch processing pipeline designed to handle large-scale data ingestion and transformation.

Avi-k-dua /covid19-dataengineering-azure

COVID 19 Data Pipeline - Azure Data Engineering: A complete end-to-end ETL pipeline to fetch COVID19 daily and weekly data from API, transform and load to SQL database in Azure using Data Factory, Databricks, Key-Vaults and Data Lake

PrositAS /go-incremental

Incremental computation engine for Go with live what-if worlds: fork the graph into N scenarios that pin some inputs and track the rest live, all updated by one stabilization. Data-shaped topology (Bind), per-key diff collections, O(divergence) scenario cost.

Caio-Analytics /PowerBI-Dataflow-Gen1-Skill

Claude Code skills for reviewing and optimizing Power Query M in legacy Power BI Dataflows Gen1, with Gen1-aware constraints and privacy-conscious code-only workflows.

Puneeth0106 /Microsoft-Fabric-Projects

Three Microsoft Fabric projects collected in one place, showcasing practical end-to-end data engineering workflows. Built primarily with Microsoft Fabric services like dataflowgen2, datafactory, pyspark-notebooks , covering ingestion, transformation, and orchestration patterns in Fabric.

tbal999 /flowtocsv

small ETL library to convert MRASCO & SPAA UK energy industry dataflows into csv

subhamay-bhattacharyya-tf /terraform-google-dataflow

🏗️ Terraform module to deploy and manage Dataflow jobs, templates, staging configuration, worker settings, and service accounts.