Data Engineering
Explore recommended reads, summarized by AI
AI and Data Engineering
How artificial intelligence is reshaping data engineering tools and workflows.
Generative AI in Data Pipelines
Semantic Layers and AI Agents
Career and Practice
Guidance on building skills, navigating roles, and shaping data engineering culture.
Career Guidance for Data Engineers
Data Engineering Culture and Industry
Cloud Data Warehouses
Working with managed cloud platforms for storing and querying large datasets.
Apache Doris
BigQuery
Snowflake
Cloud Platform Tools
Data engineering on major cloud providers and their proprietary services.
Databricks and Azure
Microsoft Fabric
Data Quality and Observability
Methods for monitoring, testing, and ensuring the reliability of data systems.
Data Quality and Observability
Infrastructure and Deployment
Containerization, cloud infrastructure, and deployment strategies for data systems.
End-to-End Pipeline Projects
Kubernetes and Infrastructure
Platform Architecture
Approaches to designing and building modern data platforms and infrastructure.
Data Pipeline Architecture
Lakehouse Architecture and Table Formats
Modern Data Platform Design
Processing Frameworks
Engines for batch and distributed data processing at scale.
Apache Spark
DuckDB and Polars
Programming for Data
Python techniques and libraries commonly used in data engineering workflows.
Python for Data Engineering
Stream Processing
Real-time data ingestion and processing with streaming platforms.
Kafka and Flink
Transformation and Modeling
Tools and techniques for transforming raw data into structured, usable formats.
Data Modeling and Warehousing
dbt Development
ETL and ELT Patterns
Workflow Orchestration
Platforms and tools for scheduling and managing data workflows.
