Abstract
Estimating heterogeneous treatment effect is an important task in causal inference with wide application fields. It has also attracted increasing attention from the machine learning community in recent years. In this work, we reinterpret the heterogeneous treatment effect estimation and propose ways to borrow strength from neural networks. We analyze the strengths and drawbacks of integrating neural networks into heterogeneous treatment effect estimation and clarify the aspects that need to be taken into consideration when designing a specific network. We proposed a specific network under our guidelines. In simulations, we show that our network performs better when the structure of data is complex, and reach a draw under the cases where other methods could be proved to be optimal.
Introduction
In fields such as personalized medicine and targeted marketing, understanding the average treatment effect is often insufficient. Researchers and practitioners increasingly require estimates of the Conditional Average Treatment Effect (CATE)—the expected response of a specific individual based on their unique covariates ($X_i$). However, the "fundamental problem of causal inference" persists: we cannot observe both treated and control outcomes for the same individual simultaneously.
While traditional statistical tools like Lasso or Causal Forests perform well on structured, tabular data, they struggle with "rich" data sources like medical imaging or social media content. These high-dimensional, non-linear inputs require the representational power of deep learning. This paper introduces CausalNet, a framework designed to bridge the gap between deep learning's feature extraction capabilities and the rigorous requirements of causal inference.
Methodology: The CausalNet Architecture
The core innovation of CausalNet is its Diverter Mechanism, which balances information sharing with group-specific learning.
Information Sharing and the Diverter
The initial layers of CausalNet are shared across all observations, allowing the model to learn a joint representation of the input data (e.g., identifying fundamental shapes or textures in an image). Following these shared layers, the network employs a "Diverter" to split the flow into "Control" and "Treatment" paths. Unlike a hard split, the diverter utilizes sigmoid-gating functions to modulate information flow based on the treatment indicator ($T_i$).
The "Soft" Split Advantage
This architecture represents a middle ground between an S-learner (a single model for all data) and a T-learner (separate models for treated and control groups). By allowing the paths to share "strength" in early layers while maintaining distinct identities for the final causal estimate, CausalNet optimizes the use of available data. The network is trained end-to-end on the observed outcome ($Y^{obs}$) using an L2 loss, implicitly learning to estimate the difference between potential outcomes.
Experimental Setup
To validate the model, the authors conducted simulations where ground-truth treatment effects could be precisely controlled. The primary focus was on image-based covariates, where traditional methods typically fail.
Datasets
The study utilized 32x32 images containing geometric shapes (circles), where the treatment effect was directly tied to specific image features like the circle's radius.
- Noiseless Image Data: Clean images used to establish a baseline for feature extraction.
- Noisy Image Data: Images polluted with Gaussian noise to simulate the "messy" nature of real-world data.
- Synthetic Generators: Four mathematical schemes (Linear, Polynomial, Tree-based, and Neural Network-based) were used to generate outcomes, testing the model's robustness across different functional forms.
Results and Analysis
CausalNet demonstrated significant advantages in handling complex data structures:
- Superiority on Complex Data: In image-based simulations, CausalNet was the only method capable of detecting the treatment effect. Traditional methods, including Adjusted Regression and standard S/T-learners, failed to identify the signal, predicting only the average effect.
- Order of Magnitude Improvement: On noisy image data, CausalNet achieved a Mean Squared Error (MSE) an order of magnitude lower than competing methods.
- Robustness: When tested on simpler data favoring linear models or trees, CausalNet remained competitive, reaching a "draw" with specialized methods. This suggests that CausalNet is a safe, high-performance choice even when the underlying data structure is unknown.
Conclusion and Future Work
CausalNet proves that deep learning can effectively uncover causal patterns in high-dimensional data without manual feature engineering. However, challenges remain. Neural networks are more sensitive to hyperparameter tuning and lack the inherent interpretability of causal trees. Future research will explore extending the diverter framework to continuous treatments (e.g., varying drug dosages) and improving the stability of individual treatment effect targets in extremely noisy environments.
Summary updated