In machine learning, we are typically concerned with prediction: given a set of features $X$, what is the most likely outcome $Y$? However, in fields like medicine, public policy, and economics, the question is often more complex: "What would have happened to this specific patient if they had received Treatment B instead of Treatment A?"
This is the problem of Individual Treatment Effect (ITE) estimation. It is notoriously difficult because we can never observe both outcomes for the same individual—a dilemma known as the Fundamental Problem of Causal Inference. Recent research has reframed this challenge as a domain adaptation problem, allowing us to use the machinery of deep representation learning to "bridge the gap" between what we saw and what might have been.
The Problem of Selection Bias
In a perfect world, we would run a Randomized Controlled Trial (RCT) where treatments are assigned by a coin flip. In this scenario, the treated and control groups are statistically identical. However, most real-world data is observational. Doctors give treatments to patients they believe will benefit most, and users click on recommendations they were already predisposed to like.
This creates selection bias. The distribution of people who received the treatment (the factual domain) is fundamentally different from the distribution of people who did not (the counterfactual domain). As Fredrik Johansson, Uri Shalit, and David Sontag argue in Learning Representations for Counterfactual Inference, a model trained on biased observational data will fail to generalize to counterfactual scenarios because it hasn't learned to distinguish between the effect of the treatment and the characteristics of the people who usually receive it.
Bridging Domains via Balanced Representations
If we view causal inference as domain adaptation, the "source" is the group that received a specific treatment, and the "target" is the group that did not. To make accurate predictions across these domains, we need to find a representation of the data where the treated and control groups look as similar as possible.
The core mechanism, popularized by Shalit et al., involves training a neural network to learn a latent mapping $\Phi(x)$. This mapping is optimized for two competing goals:
- Predictive Accuracy: The representation must retain enough information to predict the outcome $Y$.
- Balance (Distribution Alignment): The distribution of $\Phi(x)$ for the treated group must be indistinguishable from the distribution of $\Phi(x)$ for the control group.
By minimizing a distance metric—such as the Integral Probability Metric (IPM) or Maximum Mean Discrepancy (MMD)—between these distributions, the model is forced to ignore features that lead to selection bias (the "why" of the treatment) and focus on features that determine the outcome (the "how" of the effect).
Generalization Bounds for "What-Ifs"
Why does this balancing act work? In Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects, Fredrik Johansson and colleagues provide a theoretical justification. They show that the error in estimating a counterfactual outcome is bounded by the sum of the training error on the factual data and the distance between the treated and control distributions in the representation space.
Essentially, if the two groups are too different, the model has to "extrapolate" too far into unknown territory, and the error bound explodes. By forcing the distributions together, we reduce this divergence, making the extrapolation more reliable. This same logic is applied by Amit Sharma and Rathin Desai to quantify error in the presence of hidden confounders, treating the randomized experiment as the gold-standard target domain.
Beyond Medicine: Recommendations and Classification
The "Causal Inference as Domain Adaptation" framework has moved beyond clinical settings into broader AI challenges:
- Recommender Systems: Stephen Bonner introduces CausE in Causal Embeddings for Recommendation, which uses this framing to predict the "lift" of a recommendation. It treats the discrepancy between an existing recommendation policy and a randomized policy as a domain shift.
- Long-tailed Classification: In Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification, Hanwang Zhang and colleagues model imbalanced training data and balanced test sets as distinct domains. They treat the selection variable as a confounder, using causal tools to ensure the model doesn't simply swap one class bias for another.
- Transportability: Elias Bareinboim and Judea Pearl formalize this as "transportability" in Transportability of Causal and Statistical Relations, providing the algorithmic rules for when causal knowledge from one environment can be safely applied to another.
The Path Forward: Disentanglement and Self-Training
The field is now moving toward more sophisticated ways of handling these shifts. Rather than just forcing distributions to overlap, researchers like Ignavier Ng are using structural equation models to identify exactly which causal components are shifting between domains. Others, such as Ruijiang Gao, are using self-training to iteratively "fill in" missing counterfactuals, effectively hallucinating a randomized trial from biased data.
By treating the unobserved counterfactual as a target domain, these researchers have turned a missing data problem into a generalization problem, allowing causal inference to benefit from the rapid progress of deep representation learning.