OBenner/data-engineering-interview-questions：超过2000个数据工程师面试问题

项目介绍

OBenner/data-engineering-interview-questions 是一个托管在 GitHub 上的开源项目，主要使用 Python 编写，目前获得 1.8k 个 Star、565 次 Fork。

仓库全名: OBenner/data-engineering-interview-questions

README 摘要

More than 2000+ questions for preparing a Data Engineer interview. Full list of questions Pick a topic below or use the full list to practice end-to-end. Interview questions for Data Engineer Databases and Data Warehouses GitHub Repo Official page Questions Description Useful links Apache Cassandra Cassandra is a distributed, wide-column store, NoSQL database management system. Awesome Cassandra Greenplum Greenplum…

完整内容见 GitHub 原始 README 或本页。

常见问题

data-engineering-interview-questions 用什么语言开发？

主要使用 Python 编写，目前有 1.8k 个 Star。

如何获取 data-engineering-interview-questions 的源码？

可以直接克隆仓库：git clone https://github.com/OBenner/data-engineering-interview-questions.git，默认分支为 master；安装与运行步骤见仓库 README。

data-engineering-interview-questions 最近还在维护吗？

仓库最近一次更新为 1月14日，当前有 2 个未关闭 Issue；Star 数为 1.8k，可作为社区活跃度的参考。

关于

GitHub 原始描述（英文）

More than 2000+ Data engineer interview questions.

airflow avro aws azure cassandra data-engineering data-structures flink flume hadoop hadoop-hdfs hbase hive impala interview interview-questions kafka nifi spark sql

1.8k Stars
565 Forks
23 Watching

描述取自 GitHub 官方 API，保持仓库作者书写的原文。

简要分析

热度分

2961

价值分

1256

活跃状态

活跃

GitHub

语言占比

Python

贡献者

More than 2000+ questions for preparing a Data Engineer interview.

Full list of questions

Pick a topic below or use the full list to practice end-to-end.

Interview questions for Data Engineer

Databases and Data Warehouses
GitHub Repo	Official page	Questions	Description	Useful links
		Apache Cassandra	Cassandra is a distributed, wide-column store, NoSQL database management system.	Awesome Cassandra
		Greenplum	Greenplum is a big data technology based on MPP architecture and the Postgres open source database technology.	Awesome Greenplum
		MongoDB	MongoDB is a document-oriented database.	Awesome MongoDB
		Apache Hbase	HBase is an open-source non-relational distributed database.	Awesome HBase
		Apache Hive	Apache Hive is a data warehouse software project built on top of Apache Hadoop for providing data query and analysis.	Awesome Hive
		Amazon DynamoDB	Amazon DynamoDB is a fully managed proprietary NoSQL database service.	Awesome DynamoDB Awesome AWS
		Amazon Redshift	Amazon Redshift is a data warehouse product.	Amazon Redshift Utilities Awesome AWS
		BigQuery GCP	BigQuery is a fully-managed, serverless data warehouse.	Awesome BigQuery
		Bigtable GCP	Bigtable is a fully managed wide-column and key-value NoSQL database service.	Awesome Bigtable

Data Formats
		Apache Avro	Avro is a row-oriented remote procedure call and data serialization framework.	Awesome Avro
		Apache Parquet	Apache Parquet is a column-oriented data file format designed for efficient data storage and retrieval.	Parquet format · Docs
		Delta	Delta Lake is a storage framework that enables building a Lakehouse architecture with compute engines	Delta examples
		Apache Iceberg	Apache Iceberg is an open table format for huge analytic datasets.	Iceberg docs
		Apache Hudi	Apache Hudi brings upserts, deletes, and incremental processing to data lakes.	Hudi docs

Big Data Frameworks
		Apache Airflow	Apache Airflow is a workflow management platform for data engineering pipelines.	Awesome Airflow
		Apache Flume	Apache Flume is a distributed, reliable, and available software for efficiently collecting, aggregating, and moving large amounts of log data.	Flume User Guide
		Apache Hadoop	Apache Hadoop is a collection of software utilities that facilitates using a network of many computers to solve problems involving massive amounts of data and computation.	Awesome Hadoop
		Apache Impala	Apache Impala is a parallel processing SQL query engine for data stored in a computer cluster running Apache Hadoop.	Impala docs
		Apache Kafka	Apache Kafka is a distributed event store and stream-processing platform.	Awesome Kafka
		Apache NiFi	Apache NiFi is a software project designed to automate the flow of data between software systems.	Awesome NiFi
		Apache Spark	Apache Spark is unified analytics engine for large-scale data processing.	Awesome Spark
		Apache Flink	Apache Flink is unified stream-processing and batch-processing framework.	Awesome Flink
		Kubernetes	Kubernetes is a system for managing containerized applications across multiple hosts.	Awesome Kubernetes

Cloud providers
		Amazon Web Services	Amazon web service is an online platform that provides scalable and cost-effective cloud computing solutions.	Awesome AWS
		Microsoft Azure	Microsoft Azure is Microsoft's public cloud computing platform.	Awesome Azure
		Google Cloud Platform	Google Cloud Platform is a suite of cloud computing services.	Awesome GCP

Modern Data Stack
		dbt	dbt is a transformation framework for building tested and documented SQL models.	dbt tests

Theory
		DWH Architectures	A data warehouse architecture is a method of defining the overall architecture of data communication processing and presentation that exist for end-clients computing within the enterprise.	Awesome databases
		Change Data Capture (CDC)	CDC captures inserts/updates/deletes from source systems for low-latency ingestion.	Debezium docs
		Data Modeling	Dimensional modeling concepts used to build reliable analytics datasets.	Kimball Group
		Data Quality	Tests, monitoring, and practices to ensure datasets are trusted and correct.	Great Expectations docs
		Data Observability	Monitoring and incident response practices for pipeline and dataset health.	OpenLineage
		Data Governance	Ownership, policies, privacy, and access controls for data platforms.	DataHub
		Cost Optimization	Practical techniques to reduce compute and storage costs while meeting SLAs.	Spark tuning
		Python for Data Engineering	Python fundamentals for reliable, scalable data pipelines and tooling.	PyArrow docs
		Data System Design	System design interview questions for batch/streaming data platforms.	Data mesh overview
		Data Structures	A data structure is a specialized format for organizing, processing, retrieving and storing data.	Awesome Algorithms
		SQL	SQL is a domain-specific language used in programming and designed for managing data held in a relational database management system (RDBMS).	Awesome SQL

Data visualization tools/BI
		Tableau	Tableau is a powerful data visualization tool used in the Business Intelligence.	Tableau Desktop docs
		Looker	Looker is an enterprise platform for BI, data applications, and embedded analytics that helps you explore and share insights in real time.	Looker docs
			Apache Superset	Superset is a modern data exploration and data visualization platform	Superset docs