Causal Machine Learning: Moving Beyond Prediction to Real Decisions
Uber built a whole library for it. Netflix runs it in production. Amazon uses it to diagnose why their servers slow down. Causal machine learning is quietly becoming the most important idea in data science, and most teams are still ignoring it.
Here’s a story that every data scientist recognises. You’ve built a churn model. It’s good, AUC of 0.85, precision looks solid, your stakeholders are happy. The model flags the customers most likely to leave next month, and the retention team sends them all a discount. Churn goes down slightly. Everyone nods approvingly. A few months later, someone in the meeting asks: “Did the discounts actually cause people to stay, or were those customers already going to stay?” The room goes quiet. Nobody knows. The model never told you that.
That gap , between knowing what will happen and knowing why it happens, is exactly what causal machine learning was built to close. And in 2026, it’s not a niche academic curiosity anymore. Fortune Business Insights valued the global Causal AI market at $81.4 billion in 2025, projecting $116 billion for 2026. Uber built an entire open-source library for it. Netflix runs causal inference in their recommendation engine. Amazon’s AWS team uses it to diagnose why latency spikes happen rather than just predict when they will. If those names are doing it in production, it’s worth understanding properly.
Let’s do that. From the ground up, in plain language, with real code at the end.
The Problem With Correlation — A Story You Already Know
There’s a famous, slightly embarrassing fact about data: ice cream sales and drowning rates are positively correlated. As ice cream sales go up, drowning deaths go up. Should we ban ice cream to save lives?
Obviously not. Both happen to spike in summer, when it’s hot. Hot weather is the real driver, a hidden third variable causing both. The correlation is real. The causal relationship is completely wrong.
This is the core problem with traditional machine learning. As DataCamp’s guide puts it: “Traditional ML learns correlations. It finds that customers who receive discount emails tend to buy more, and it uses that pattern to predict future purchases. But correlation doesn’t tell you whether the email caused the purchase. Maybe those customers were already planning to buy.”
That distinction, did the intervention cause the outcome, or did both just happen to occur together?, is the difference between a model that helps you make decisions and a model that helps you observe patterns. Most ML models only do the second thing. Causal ML does both.
“Prediction and causation are two different things. Some would even say that moving from prediction to causation is the only way to make actual data-driven decisions.”
— Dario Radečić, DataCamp, “Causal Machine Learning: From Prediction to Cause and Effect”
What Causal ML Actually Does Differently
The distinction is cleanest in a side-by-side comparison. Traditional ML and causal ML are not competing approaches, they answer genuinely different questions.
The first question in each row helps you predict. The second helps you decide. DataCamp’s framing is exact: “The first helps you predict. The second helps you decide. Both are useful.” But most business decisions are causal by nature, they’re asking “what should we do?” not “what will happen?” and for those questions, a prediction model is the wrong tool.
Judea Pearl’s “ladder of causation” gives you the cleanest mental model for understanding why this matters. The ladder has three rungs. Rung one is association, seeing that two things tend to appear together (“customers who received discounts bought more”). Rung two is intervention, asking what happens if you actively do something (“what happens if we send this customer a discount?”). Rung three is counterfactual, imagining an alternate history that never happened (“would this customer have bought if we hadn’t sent the discount?”). Traditional ML lives almost entirely on rung one. Causal ML operates on all three.
Three Concepts That Underpin Everything
Before the methods make sense, three ideas need to click. They’re not complicated, they just need to be stated precisely.
1. Treatment and Outcome: The Two Sides of Every Causal Question
Every causal question has a treatment, the action or intervention you’re studying, and an outcome, what you’re trying to measure. The drug given to patients is the treatment; recovery time is the outcome. The discount emailed to customers is the treatment; purchase amount is the outcome. The new feature shipped last Tuesday is the treatment; weekly active users is the outcome.
The goal of causal ML is not to measure whether treatment and outcome are correlated, your regular model already does that. The goal is to estimate how much the treatment actually changes the outcome, isolated from everything else that might be moving at the same time.
2. Counterfactuals: The Question You Can Never Directly Answer
Here’s the fundamental problem. You sent the discount to a customer, and they bought. Would they have bought without the discount? You can’t know. You can’t rewind time and run the same person through both versions of reality simultaneously. That unobserved scenario, what would have happened if you’d done something different, is the counterfactual.
Causal ML doesn’t solve this impossibility. What it does is estimate the counterfactual using statistical techniques and assumptions, building a plausible picture of the alternate reality you never got to observe. The quality of your causal estimate depends entirely on how well those assumptions hold up.
3. Confounders: The Hidden Variable That Ruins Everything
A confounder is a variable that affects both your treatment and your outcome, creating a spurious appearance of causation between them. The exercise and heart disease example from DataCamp is perfect: people who exercise more also tend to eat healthier. Diet affects both the likelihood of exercising (treatment) and the risk of heart disease (outcome). Ignore diet, and you’ll wildly overestimate how much exercise alone matters. Diet is the confounder.
⚠️ Confounders are the biggest threat in causal analysis. The scary part: your model won’t tell you when it’s been fooled by one. It’ll happily produce precise-looking estimates from badly confounded data. Identifying and accounting for confounders requires domain knowledge, the one thing no library can provide for you.
Drawing the Map Before You Run the Model: DAGs
Before writing a single line of code, every serious causal analysis starts with a picture. Specifically, a Directed Acyclic Graph (DAG), a diagram that maps the causal relationships between all the variables in your problem.
In the exercise-heart disease example, the DAG has arrows from Diet → Exercise and Diet → Heart Disease, plus an arrow from Exercise → Heart Disease. One glance at that diagram tells you Diet is a confounder, because it has arrows pointing to both the treatment and the outcome.
The DAG also tells you something crucial that most people miss: not every variable should be controlled for. If you control for a mediator, a variable that sits on the causal path between treatment and outcome, you’ll actually block the causal effect you’re trying to measure. If you control for a collider, a variable caused by both treatment and outcome, you’ll introduce bias rather than remove it. Getting this wrong is extremely easy and extremely consequential. The DAG is what keeps you from making those mistakes systematically.
The best part: you don’t need a PhD in statistics to build one. You need domain knowledge. What causes what in your problem? Who talks to whom? Which decisions come before which outcomes? Draw that. Then build your model around it.
The Four Methods And When to Use Each
Propensity Score Methods: Simulating a Randomized Experiment
In an ideal world, you’d run a randomized controlled trial, randomly assign some people to receive the treatment and some not, and just measure the difference. Randomized controlled trials are the gold standard precisely because randomization eliminates confounding by design. But you can’t always randomize. You can’t randomly assign people to smoke cigarettes to study lung cancer. You can’t randomly make some people unemployed to study a job training program.
Propensity score methods address this by estimating the probability that each individual receives the treatment, based on their observed characteristics. That probability is the propensity score. You then either match treated and untreated individuals with similar propensity scores, creating quasi-experimental pairs or reweight the entire sample so the treated and untreated groups look comparable. The goal is to simulate what a randomized trial would have looked like, using the messy observational data you actually have.
Instrumental Variables: When You Can’t Measure the Confounder
Sometimes the confounders you need aren’t in your data at all. You can’t measure motivation. You can’t measure the patient’s private health behaviours. This is where instrumental variables come in.
An instrument is a variable that affects the treatment but has no direct effect on the outcome, it only influences the outcome through the treatment. The classic example: studying the effect of education on earnings. Motivation affects both education and earnings, but you can’t measure it. Distance to the nearest college works as an instrument, it affects whether someone attends, but it doesn’t directly affect their future salary. By exploiting the variation in college attendance driven purely by geographic proximity, you can estimate the causal effect of education while holding unmeasured motivation constant.
Finding a good instrument is genuinely hard. When you have one, it’s powerful. A weak or invalid instrument produces results that are worse than useless.
Causal Forests: When the Effect Varies by Individual
Causal forests extend the random forest algorithm to estimate treatment effects instead of predictions. A standard random forest tells you the predicted outcome. A causal forest tells you how much the treatment changes the outcome, and crucially, it does this separately for different subgroups. A drug might work well for younger patients but poorly for older ones. A discount might move price-sensitive customers but leave loyal ones unmoved. Causal forests detect those differences by splitting the data on characteristics that produce the biggest variation in treatment effects.
The output is a personalised treatment effect estimate for every individual in your dataset. Not one number for everyone, a tailored estimate for each person based on their specific characteristics. This is what makes causal ML actionable at scale.
Double Machine Learning: Removing Bias from Complex Predictions
Double machine learning (DML) is the method that’s probably seen the most adoption in industry over the last two years, because it marries the predictive power of modern ML with the statistical rigour needed for valid causal inference.
The idea: use one ML model to predict the outcome and another to predict the treatment assignment. Then look at the residuals, the parts that neither model could explain. The relationship between those residuals gives you the causal effect. The reason for the two-stage process is that ML models introduce regularisation bias when used for causal estimation directly. DML removes that bias through cross-fitting, training and predicting on different splits of the data to avoid overfitting contaminating your causal estimate.
ATE vs CATE: Two Numbers That Mean Very Different Things
Once you’ve picked a method and run it, you get treatment effect estimates. The two most important ones are easily confused but critically different.
The Average Treatment Effect (ATE) is the average causal effect of the treatment across your entire population. If the ATE of a discount campaign is £5, the discount increased average spending by £5 per customer. It’s a useful overall signal. But it hides a lot.
A £5 average could mean everyone gained exactly £5. It could also mean half your customers gained £15 while the other half saw no change. Or, and this is the uncomfortable one, some customers spent more because of the discount while others actually spent less because the discount undercut their perception of the product’s value. The ATE doesn’t tell you any of that.
Get Gabriel Nguyen’s stories in your inbox
Join Medium for free to get updates from this writer.
The Conditional Average Treatment Effect (CATE) breaks that down. Instead of one number for everyone, CATE gives you treatment effect estimates by subgroup, or even by individual. The discount increased spending by £8 for new customers but only £2 for returning ones. If your budget is limited, that tells you exactly where to point the campaign.
ATE vs CATE — The Practical Difference
ATE answers: “Does the intervention work?” → Good for deciding whether to run a programme at all.
CATE answers: “For whom does the intervention work?” → Essential for deciding who to target, who to exclude, and where to invest limited resources.
The real-world implication: A retention discount with a positive ATE might still be net negative if most of the spending increase comes from customers who were going to stay anyway. CATE lets you target only the customers whose behaviour the discount actually changes, which is the whole point.
The Companies Running This in Production
$116B — Projected global Causal AI market size in 2026 (Fortune Business Insights)
70% — Of AI-driven organisations expected to incorporate causal reasoning by end of 2026
3 — Open-source Python libraries — DoWhy, EconML, CausalML — now making this accessible to any data team
This isn’t theoretical anymore. As Towards Data Science documented in March 2026, the largest tech companies have built causal inference directly into their production systems.
Uber built CausalML, the open-source library now used across the industry, specifically for uplift modelling and treatment effect estimation. Their problem: surge pricing and promotion strategies needed to know not just who was likely to request a ride, but whose behaviour the promotion actually changed. Uber’s engineering team published detailed documentation on how they run causal inference on AI features at scale, including their use of difference-in-differences and synthetic controls for features where A/B testing isn’t practical.
Netflix uses causal bandits for content recommendations, a framework that combines causal inference with reinforcement learning to estimate not just which shows a user might click, but which recommendations causally increase satisfaction and retention. Their “Decision Making at Netflix” engineering blog post is one of the most honest accounts of how a major tech company thinks about causal reasoning at scale.
Amazon’s AWS team uses DoWhy for root cause analysis in microservice architectures, not to predict when latency spikes will happen, but to diagnose which upstream service change caused a downstream performance degradation. In a system with hundreds of interdependent services, knowing the cause is the only thing that lets you fix it efficiently.
Airbnb and Lyft have similarly published engineering blog posts describing how they measure the causal impact of AI features, including cases where standard A/B testing is compromised by network effects (when showing a feature to one user affects the experience of another user it’s connected to).
The Python Toolkit: Three Libraries, One Workflow
The practical barrier used to be expertise, you needed deep familiarity with structural causal models and the backdoor criterion before you could do any of this. That barrier has dropped dramatically. Three open-source Python libraries now make causal ML accessible to any team already comfortable with scikit-learn.
Microsoft / PyWhy — DoWhy
The structured, assumption-driven workflow. Defines the causal graph, identifies the estimand, estimates the effect, then validates it with refutation tests. Use when you want built-in robustness checking, the refutation tests actively try to break your result.
Microsoft / ALICE — EconML
Heterogeneous treatment effects with a scikit-learn-style API. Double ML, causal forests, instrumental variable estimators, all accessible via familiar .fit() and .effect() methods. Use when you need CATE estimates that vary by individual or subgroup.
Uber — CausalML
Uplift modelling and targeting decisions. Identifies which individuals will respond to a treatment, not just who is likely to convert regardless. Use when the business question is “who should we target?” not just “does the intervention work?”
Pattern — All Three Together
The common production pattern: define your causal graph in DoWhy, estimate heterogeneous effects with EconML, interpret and build targeting tiers with CausalML. Each solves a different part of the problem and they complement each other cleanly.
Here’s the core workflow from DataCamp’s practical guide, using DoWhy to estimate the causal effect of a discount on purchase amounts — and then validate that the estimate isn’t driven by a spurious correlation:
from dowhy import CausalModel# Step 1: Define the causal graph
model = CausalModel(
data=df,
treatment="discount",
outcome="purchase_amount",
common_causes=["customer_age", "prior_purchases"]
)# Step 2: Identify the causal estimand
identified_estimand = model.identify_effect()# Step 3: Estimate the causal effect
estimate = model.estimate_effect(
identified_estimand,
method_name="backdoor.linear_regression"
)# Step 4: Refute the estimate — try to break it
refutation = model.refute_estimate(
identified_estimand,
estimate,
method_name="random_common_cause"
)print(f"Estimated ATE: ${estimate.value:.2f}")
print(f"After adding a random confounder: ${refutation.new_effect:.2f}")
The refutation step is the part that most teams skip, and it’s the most important. DoWhy adds a randomly generated variable to your model and checks whether the estimate changes significantly. If it does, your result was fragile. If it doesn’t, you have much more confidence that the causal assumption is sound. No other common ML workflow has a built-in “try to break your own result” step. This one does.
The Mistakes That Will Absolutely Get You
Causal ML is easy to misuse. The tools are accessible now, which means it’s easier than ever to produce precise-looking numbers from badly specified problems. DataCamp identifies four recurring pitfalls that catch even experienced practitioners.
Treating correlation as causation. The whole reason you’re using causal ML is to avoid this. But it’s easy to backslide. Customers who use Feature X more also spend more, so you conclude Feature X drives spending, without checking whether high-spending customers are simply more engaged across the board. Every causal analysis starts with a question: is this a pattern, or is this a cause? If you skip that question, no amount of modelling helps.
Ignoring confounders. This is the most dangerous mistake because the model won’t warn you. A confounder that isn’t in your DAG doesn’t exist to the model. It’ll merrily estimate a causal effect that’s actually just a confounded correlation, and give you confidence intervals around it. The fix is structural, not statistical: draw the DAG first, identify every variable that could plausibly influence both treatment and outcome, and decide how to handle each one before you fit anything.
Misinterpreting treatment effects. An ATE of £5 doesn’t mean every customer gained £5. It’s an average. Some might have gained £20; some might have seen no effect; some might have spent less. And a CATE estimate for a subgroup doesn’t guarantee every individual in that group will respond the same way. Treatment effects are estimates with uncertainty, always check confidence intervals, and never build targeting decisions on point estimates alone.
Letting the model replace domain knowledge. A causal forest will happily produce treatment effect estimates from a badly specified model — it just won’t tell you they’re wrong. The questions that no library can answer, which variables are confounders, what the causal graph looks like, whether the treatment assignment mechanism is what you think it is, require someone who understands the problem. The best causal analyses combine ML’s ability to handle complex data with a human who knows which variables matter and why.
The rule that saves you
Before you run a single model, write down: what is the treatment, what is the outcome, what are the confounders, and what does the causal graph look like? If you can’t answer all four, you’re not ready to estimate a causal effect. The model doesn’t know what it doesn’t know. You have to tell it.
Why This Is the Skill That Matters Most Right Now
There’s a broader shift happening in how organisations think about data. Current correlation-based ML models struggle with explainability, robustness, and adaptation to shifting environments. When the world changes and it does, constantly, models trained on historical correlations degrade. A causal model doesn’t just learn that two things moved together historically; it learns the mechanism. Mechanisms are more stable than correlations. A model built on causal understanding generalises better to new environments.
There’s also the accountability dimension. Increasingly, regulators want to understand why a model made a decision, not just that it was accurate. A model that predicts loan defaults can’t tell you whether the factor it relied on caused higher default rates or merely correlated with them. A causal model can. In healthcare, in finance, in criminal justice, anywhere decisions affect people’s lives, the difference between correlation and causation is not academic. It’s the difference between a model that identifies a real driver of outcomes and one that’s found a proxy that happens to work in the training data.
The churn model that started this article wasn’t wrong. It was just answering the wrong question. The retention team needed to know not who would churn, but whose churning a discount would actually prevent. Those are different populations. Targeting them differently produces different outcomes and different economics. Causal ML is the framework for asking the right question and getting a reliable answer.
The tools are here. The companies doing this at scale are documented. The Python libraries are mature and production-tested. The question is just whether your team is ready to make the shift from “what will happen?” to “what should we do?” and willing to do the domain-knowledge work that makes the answer trustworthy.
Sources and further reading
- DataCamp / Dario Radečić — “Causal Machine Learning: From Prediction to Cause and Effect” (September 2026)
- Towards Data Science — “Causal Inference Is Eating Machine Learning” (March 2026) — market size, Uber/Netflix/Amazon case studies
- freeCodeCamp — “Product Experimentation at Scale: How Airbnb, Netflix, Lyft, and Uber Run Causal Inference” (August 2026)
- Netflix Tech Blog — “Decision Making at Netflix” (2021)
- Medium / Karan Bhutani — “The Causal Revolution in Machine Learning” (March 2025)
- Medium / Alex G. Lee — “Causal AI: Current State-of-the-Art & Future Directions” (March 2025)
- Judea Pearl & Dana Mackenzie — “The Book of Why: The New Science of Cause and Effect” (Basic Books, 2018)
- PyWhy / DoWhy — End-to-End Library for Causal Inference (Microsoft Research)
- Microsoft Research — EconML: ML-Based Heterogeneous Treatment Effects
- Uber / GitHub — CausalML: Uplift Modelling and Causal Inference Library
- DataCamp / Andrea Valenzuela — “What Is Causal AI? Understanding Causes and Effects”
