data-lakehouse 开源项目推荐（309 个）

pracdata /awesome-open-source-data-engineering

A curated list of open source tools used in analytics platforms and data engineering ecosystem

DataWithBaraa /databricks_bootcamp_2026

A data lakehouse on Databricks with the medallion architecture: PySpark, Spark SQL, Delta Lake and Unity Catalog. The project of the free Databricks bootcamp.

laminlabs /lamindb

生物学开源数据湖库。通过支持生物格式、注册库和本体的血统原生湖屋进行查询、追踪和验证。🍊YC S22

Qbeast-io /qbeast-spark

Qbeast-spark：支持多维索引和高效数据采样的数据源。 大数据，摆脱不必要的干扰！

dominikhei /Local-Data-LakeHouse

Sample Data Lakehouse deployed in Docker containers using Apache Iceberg, Minio, Trino and a Hive Metastore. Can be used for local testing.

taovietducofficial /ITLR-Fullstack-Recommender-RAG

Hybrid recommender system + RAG chatbot for IT courses (FastAPI + Express/TS + PostgreSQL), with an independent Data Lakehouse module (Spark/Kafka/dbt/Trino) and full CI/CD

realdatadriven /etlx

ETL / ELT / Reverse ETL Framework powered by DuckDB, designed to seamlessly integrate and process data from diverse sources. It leverages Markdown as a configuration medium, where YAML blocks define metadata for each data source, and embedded SQL blocks specify the extraction, transformation, and loading logic.

nssalian /floe

Floe: Policy-based table maintenance for Apache Iceberg

ailtonfv /Atlas-Social-de-Hortolandia

Repositório de desenvolvimento (MVP) para estruturação analítica do CadÚnico municipal. Sem dados pessoais — apenas código, dicionários, esquemas e amostras sintéticas. Alinhado à LGPD e boas práticas de governança.

arcesium /swiftlake

SwiftLake: Java SQL engine built on Apache Iceberg and DuckDB for efficient lakehouse reads and writes

PFund-Software-Ltd /pfeed

Data Engine for AI/Algo Trading: Download/Stream -> Clean -> Store. Supports Data Lakehouse Architecture. Clean Once and Forget.

ulbmuenster /dataasee

DatAasee - 面向图书馆的元数据湖

saurabhmahawar /presto_mongodb_workshop

A real-time & batch hybrid lakehouse architecture: Stream live GDELT 2.0 events into MongoDB (Hot Tier) via Kafka, batch-load historical archives into Apache Iceberg (Cold Tier) with Spark, orchestrate with Airflow, and serve zero-duplicate federated SQL analytics using PrestoDB and Apache Superset.

MoTahaAboHeiba /retail-lakehouse-pipeline

End-to-end retail lakehouse on Databricks: dual-path ingestion from PostgreSQL and S3, dbt-modeled galaxy schema with SCD2 history, Airflow orchestration with parallel DAG execution, CI-gated dbt tests, and a dual consumption layer via Power BI and a Genie Agent.

Rudra-G-23 /SQL-Data-Warehouse-Project

This repo provides a step-by-step approach to building a modern data warehouse using PostgreSQL. It covers the ETL (Extract, Transform, Load) process, data modeling, exploratory data analysis (EDA), and advanced data analysis techniques.

chiqors /superset-ducklake

Production-ready Apache Superset with DuckLake integration. Stateless analytics architecture using DuckDB for compute, PostgreSQL for metadata, and S3/GCS/MinIO for data lake storage. Includes Docker Compose, Kubernetes Helm charts, BigQuery Integration, and CI/CD workflows. Supports MotherDuck cloud integration.

mahmoudparsian /data-warehousing

This repository is a place for the Data Warehousing course at the Information Systems & Analytics department, Santa Clara University.

sdebruyn /dbt-fabric

Former fork combining dbt-fabric and dbt-fabricspark

MoTahaAboHeiba /E-Commerce-Lakehouse-Using-Databricks

End-to-end E-Commerce Lakehouse built on Databricks using Medallion Architecture (Bronze → Silver → Gold). Ingests raw CRM and ERP data, applies PySpark transformations, and delivers a star schema via Spark SQL stored as Delta Tables in Unity Catalog.

marcellinus-witarsah /local-data-lakehouse-iceberg

A project of creating a local data lakehouse using open-source tools and using Apache Iceberg as the open table format