Abstract
Given a dataset of individuals each described by a covariate vector, a treatment, and an observed outcome on the treatment, the goal of the individual treatment effect (ITE) estimation task is to predict outcome changes resulting from a change in treatment. A fundamental challenge is that in the observational data, a covariate's outcome is observed only under one treatment, whereas we need to infer the difference in outcomes under two different treatments. Several existing approaches address this issue through training with inferred pseudo-outcomes, but their success relies on the quality of these pseudo-outcomes. We propose PairNet, a novel ITE estimation training strategy that minimizes losses over pairs of examples based on their factual observed outcomes. Theoretical analysis for binary treatments reveals that PairNet is a consistent estimator of ITE risk, and achieves smaller generalization error than baseline models. Empirical comparison with thirteen existing methods across eight benchmarks, covering both discrete and continuous treatments, shows that PairNet achieves significantly lower ITE error compared to the baselines. Also, it is model-agnostic and easy to implement.
The Challenge of Counterfactuals
In fields like medicine or retail, estimating the Individual Treatment Effect (ITE)—the change in outcome if a treatment is altered for a specific individual—is crucial. However, the "Fundamental Challenge of Causal Inference" dictates that we only observe the outcome for the treatment actually received (the factual). We never observe the alternative (the counterfactual).
Traditional machine learning approaches often rely on "pseudo-labels" to impute these missing outcomes. If these initial guesses are inaccurate, the model learns from noise, leading to suboptimal predictions.
The PairNet Approach
PairNet addresses this by moving away from noisy imputed labels. Instead, it introduces a Pair Loss function that leverages observed factual outcomes from similar individuals.
- Pairing Strategy: For each individual, the algorithm identifies "neighbors"—other individuals with similar covariates who received a different treatment.
- Pair Loss Function: Rather than supervising on individual outcomes, PairNet minimizes the squared difference between the observed outcome gap of a pair and the model's predicted outcome gap.
- Residual Alignment: Mathematically, Pair Loss decomposes into standard factual losses plus a "Residual Alignment" term. This term encourages the model's errors to be positively correlated for similar individuals, a vital inductive bias for accurately estimating treatment differences.
- Model-Agnostic Design: PairNet is a training strategy, not a specific architecture. It can be integrated into existing models like TARNet, DragonNet, or FlexTENet.
Key Insights: A Better Proxy for Risk
A critical finding is that standard "factual" loss (FLoss) is a poor proxy for the true ITE risk. In experiments, FLoss showed only a 0.32 correlation with true risk. In contrast, PairNet’s loss achieved a significantly stronger 0.82 correlation, providing a much more reliable signal for optimization.
Theoretical analysis further supports this, establishing PairNet as a consistent estimator. The error bounds for PairNet depend on the distance between an individual and their neighbors, which is typically much smaller than the distance between entire treatment and control groups used by standard models.
Empirical Results
PairNet was evaluated against 13 state-of-the-art methods across eight benchmarks, including IHDP, ACIC, and Twins for binary treatments, and TCGA and News for continuous treatments.
As shown in the table above, PairNet consistently achieves the lowest PEHE (Precision in Estimating Heterogeneous Effects) scores across major benchmarks. Notably, it outperforms meta-learners, representation learners, and matching methods. Furthermore, PairNet remains robust even when perfect "twins" are unavailable, as it continues to enforce correct alignment of error residuals.
Conclusion
PairNet offers a robust, model-agnostic framework for ITE estimation. By focusing on observed pairs rather than noisy pseudo-outcomes, it provides tighter theoretical bounds and superior empirical performance. While it requires an overlap in treatment assignments and adds a minor pre-processing step for neighbor search, its ability to improve existing architectures makes it a powerful tool for causal inference.
Summary updated