Nicolas Pogeant·3h agodbt Knows the Order, Not the ScheduleOrdering models is dbt’s job. Deciding when anything runs, and what must finish first, belongs to the layer around it
Alexander Daniel Pratama·10h agoOrchestrate your Arcpy in AirflowFor years, all my ETL scripts did its job well. Then one day a colleague asked me to explain how it worked, and I froze because it will…
Michalzarnecki·20h agoFrom Data to Production Decisions, Models and Prototypes: Classification, Streamlit, and Airflow —…So far, we have worked mainly without ready-made answers.
Ahmed Abdulwahid·21h agoApache Airflow for Data Engineers🌪️: From Beginner to Production (Part 2)This is Part 2 of the Apache Airflow Course. Before we dive in, make sure you’ve gone through Part 1, as many of the concepts we’ll discuss…
Ahmed Abdulwahid·2d agoApache Airflow for Data Engineers🌪️: From Beginner to Production (Part 1)Imagine You’re Running a Restaurant 🍳. Every day, you must:
Think Data·2d agoWhen Python Loops Become a Problem in AirflowOne builds tasks when Airflow parses your DAG. The other creates task instances when the DAG runs. Understanding that difference can save…A response icon1A response icon1
Think Data·2d ago15 Airflow Mistakes I Keep Seeing in Production (And How to Fix Them)Your DAG passes code review, the tasks are green, and everything looks fine. Then a backfill runs, two DAGs overlap, or an external service…A response icon1A response icon1
InTouchBistro EngineeringbyBahare Askari·2d agoBuilding idempotent SCD type 2 dimensions with dbt and incremental loadsKeeping full dimension history in dbt: incremental SCD Type 2 loads made idempotent with deterministic surrogate keys.A response icon1A response icon1
Judith Ezenwosu·2d agoConnecting Apache Airflow (Docker/WSL) to a Supabase PostgreSQL DatabaseA complete, beginner-friendly guide to provisioning a Supabase database, troubleshooting network routing restrictions within Windows…
