Causality in Modern Machine Learning

A recurring pattern in applied machine learning: a model performs well in validation and degrades badly in deployment, in ways that seem unrelated to overfitting.

A frequent explanation is that the model learned an association that was not causal, and the association did not survive the change of setting.

The standard illustration is a classifier distinguishing cows from camels which, having only ever seen cows on grass and camels on sand, has largely learned to classify background. It is not broken as a predictor. Within the training distribution the background is genuinely informative, and the model correctly exploited it.

The problem is that the relationship between background and animal is not a stable mechanism. It is an artefact of how the photographs were collected, and it changes when the collection process changes.

Causal relationships behave differently, and this is the property that makes them worth pursuing. The mechanism relating a shape to its label is invariant across settings; the correlation between background and label is not.

So a system relying on causal structure should degrade more gracefully under distribution shift than one exploiting whatever correlations happened to be strongest.

That observation converts into an algorithm, and it is the most practically influential idea connecting causality to machine learning.

The reasoning runs in two steps. Causal mechanisms are stable across environments. Spurious correlations vary across environments. So if you have data from several environments, a feature whose predictive relationship holds in all of them is a candidate causal feature, and one whose relationship shifts is a candidate spurious one.

This reframes the training objective. Ordinary empirical risk minimisation asks for the predictor minimising average error over pooled data, which happily exploits whatever correlations dominate the pool. Invariance-based approaches instead search for a representation whose relationship to the outcome is the same in every environment.

The practical requirement is that you need multiple environments with genuine variation, and enough of it to distinguish stable relationships from unstable ones. Data from different hospitals, countries, time periods, or measurement devices all qualify.

Kiyavash's work uses this structure directly: causal effects can be identifiable under a latent confounder when data is available from multiple environments, provided the target effect remains invariant across them.

The idea is powerful and not a solved problem. How much environment variation is needed, and how reliably these methods work, remains actively researched.

There is a sharper point about where causality is mandatory rather than merely helpful.

A system that only reports predictions can be evaluated on predictive accuracy. A system whose outputs drive actions cannot, because it is being used to answer an interventional question whatever it was trained on.

A model predicting which customers will churn is used to decide whom to contact. A model predicting readmission risk is used to decide whom to treat. A model scoring loan applications determines who receives a loan.

In each case the deployed question is not "what will happen?" but "what will happen if I act?", which is one rung up the ladder from what the training objective addressed.

The failure this produces is specific and worth naming. Suppose customers who receive a retention offer churn more in your historical data, because offers were sent to customers already showing signs of leaving. A predictive model learns that association correctly. Acting on it inverts the policy.

The practical rule: if a model's output changes what happens, it is being used causally, and it should be validated by intervention, meaning an experiment, rather than by held-out predictive accuracy alone.

The question is not how sophisticated the model is but what its output is used for. Only the left branch is answerable by predictive validation alone.

Sequential decision-making is where the connection is tightest, because these settings are explicitly interventional by construction.

A multi-armed bandit chooses an action and observes a reward. That is intervention: the algorithm sets the action rather than observing one that was chosen for other reasons. Exploration exists precisely to generate interventional data where observational data would be confounded.

So bandits and reinforcement learning already do causal inference. The interesting questions arise where the clean assumption breaks.

Unobserved confounders in the environment. If something influences both the reward and the state you observe, standard methods can converge to a wrong policy.

Off-policy evaluation. Estimating how a new policy would perform using data logged under a different one is exactly a causal question about an intervention never taken, and it uses the same re-weighting machinery as observational causal inference.

Transfer. Moving a policy to a new environment is asking whether the learned relationships were mechanisms or artefacts.

This is where Kiyavash's recent direction sits. The European Research Council awarded her an ERC Advanced Grant in 2025 for the project CIEL, Causal Inference for Exploration and Learning, aimed at bridging causal inference and modern learning systems, with a focus on sequential decision-making including bandits, reinforcement learning, and large language models.

Large language models raise the question in an unusually slippery form, and precision matters here because claims in both directions are overstated.

What is established is that these models reproduce causal claims present in their training data fluently. Ask which of two variables causes the other in a well-documented domain and the answer is usually right, because it was written down many times.

That is retrieval of recorded causal knowledge, which is genuinely useful. It is not the same as inferring causal structure from data, and the distinction is easy to lose because the outputs look alike.

Two failure modes follow from the mechanism. The model has no way to distinguish a widely repeated causal claim from a well-established one, so a popular misconception is reproduced with the same confidence as a settled finding. And given an actual dataset, nothing in next-token prediction performs the identification analysis the second lesson described.

The more interesting research direction runs the other way: applying causal reasoning to language models. What in a prompt caused a particular output? Does a fine-tuning intervention change behaviour for the reason assumed? These connect naturally to interpretability, where intervention is already the standard for establishing a mechanism.

An honest accounting, since this area attracts both overclaiming and dismissal.

Two points deserve emphasis.

Causality is not a robustness cure. It helps against shifts that leave causal mechanisms intact while changing spurious correlations. A shift that changes the mechanism itself is not addressed, and neither is a shift into a genuinely novel regime.

The costs are real. Causal methods need assumptions you must defend, multiple environments or experiments you must obtain, and they typically give worse in-distribution accuracy than a model free to exploit every available correlation. You are trading average-case performance for stability, and that trade is only worth making when the deployment distribution actually differs from the training one.

Four claims, and the question to carry away.

Prediction and intervention are different problems. The ladder of causation separates them formally, and a higher rung cannot in general be answered from lower-rung data without an assumption about the data-generating process. Structural causal models are how those assumptions get stated precisely enough to check.

Identification comes before estimation. Given a graph, the backdoor and front-door criteria and instrumental variables say exactly when a causal effect is computable from observational data, and by what formula. When nothing identifies it, more data and more model capacity do not help, and bounds with sensitivity analysis are the honest response.

Discovery has a hard limit. Conditional independencies determine a graph only up to a Markov equivalence class, because chains and forks are observationally identical and only colliders break the symmetry. Allowing latent confounders makes the output less determinate and more honest.

Machine learning needs this wherever outputs drive actions. Invariance across environments turns causal stability into a training signal, sequential decision-making is interventional by construction, and any deployed model that changes what happens is answering a causal question whatever it was trained on.

The durable habit: before trusting a number, ask what assumption makes this causal, and would I defend it?