apache-spark 开源项目推荐（3,639 个）

mlflow /mlflow

用于代理、LLM和ML模型的开源人工智能工程平台。MLFlow使各种规模的团队能够调试、评估、监控和优化生产质量的人工智能应用程序，同时控制成本并管理对模型和数据的访问。

treeverse /lakeFS

lakeFS - Data version control for your data lake | Git for data

microsoft /SynapseML

简单和分布式机器学习

lakehq /sail

这是一个用Rust语言编写的、用于替代Apache Spark的替代品，能够统一处理批处理任务、流处理任务以及需要大量计算资源的AI相关工作负载。

lw-lin /CoolplaySpark

酷玩 Spark: Spark 源代码解析、Spark 类库等

kubeflow /spark-operator

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

spark-notebook /spark-notebook

Interactive and Reactive Data Science using Scala and Spark.

intel /BigDL

BigDL: Distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray

dotnet /spark

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

big-data-europe /docker-spark

Apache Spark docker image

feathr-ai /feathr

Feathr – 面向企业的可扩展、统一数据与 AI 工程平台

awesome-spark /awesome-spark

A curated list of awesome Apache Spark packages and resources.

ptyadana /SQL-Data-Analysis-and-Visualization-Projects

使用 MySQL、PostgreSQL、SQLite、Tableau、Apache Spark 和 pySpark 的 SQL 数据分析与可视化项目。

OryxProject /oryx

Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

japila-books /apache-spark-internals

Apache Spark的内部结构

san089 /goodreads_etl_pipeline

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

databricks /LearningSparkV2

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

graphframes /graphframes

GraphFrames是Apache Spark的一个包，它提供了基于DataFrame的图形

lensacom /sparkit-learn

PySpark + Scikit-learn = Sparkit-learn

mahmoudparsian /data-algorithms-book

MapReduce, Spark, Java, and Scala for Data Algorithms Book