Abstract
Treatment effect estimation from observational data is a central problem in causal inference. Methods based on potential outcomes framework solve this problem by exploiting inductive biases and heuristics from causal inference. Each of these methods addresses a specific aspect of treatment effect estimation, such as controlling propensity score, enforcing randomization, etc., by designing neural network architectures and regularizers. In this paper, we propose an adaptive method called Neurosymbolic Treatment Effect Estimator (NESTER), a generalized method for treatment effect estimation. NESTER brings together the ideas used in existing methods based on multi-head neural networks for treatment effect estimation into one framework. To perform program synthesis, we design a Domain Specific Language (DSL) for treatment effect estimation based on inductive biases used in literature. We also theoretically study NESTER's capability for treatment effect estimation. Our comprehensive empirical results show that NESTER performs better than state-of-the-art methods on benchmark datasets without compromising run time requirements.
Introduction
Causal effect estimation—answering "what if" questions like "Would this patient have recovered faster with Drug A instead of Drug B?"—is a cornerstone of observational data analysis. However, accurate estimation is notoriously difficult due to selection bias: individuals receiving a treatment often differ fundamentally from those in the control group.
While deep learning has introduced powerful tools like TARNet and Dragonnet to address these challenges, these models typically rely on fixed, "hand-crafted" architectures. This paper introduces NESTER (NEuroSymbolic causal effecT EstimatoR), a neurosymbolic approach that allows for the automatic synthesis of optimal model architectures tailored to specific datasets. By searching through a domain-specific language of causal building blocks, NESTER discovers architectures that outperform existing state-of-the-art baselines.
The NESTER Framework
NESTER treats the design of a causal estimator as a program synthesis problem. The core of the framework is a Domain-Specific Language (DSL) containing primitives that represent common inductive biases in causal inference. These primitives include standard neural layers (mlp), branching structures for different treatment groups (if-then-else), propensity score modules for de-biasing, and distribution alignment modules (align) to minimize Maximum Mean Discrepancy between groups.
As illustrated in Figure 1, NESTER is a generalization of existing models. It can "rediscover" established architectures like TARNet or CFR if they prove to be the optimal solution for a given dataset. This symbolic representation also enhances interpretability, as researchers can inspect the synthesized "program" to understand the model's underlying logic.
Program Synthesis via A* Search
To navigate the expansive search space of possible architectures, NESTER utilizes an $A^*$ search algorithm combined with Neural Admissible Relaxation (NEAR). This strategy allows the system to estimate the potential performance of partial architectures before committing to full training, making the search process computationally feasible. The resulting synthesized programs are fully differentiable, allowing for end-to-end training via standard gradient descent.
Empirical Evaluation
We evaluated NESTER against 16 baselines on three standard benchmarks: the Infant Health and Development Program (IHDP), the Twins dataset, and the Jobs (LaLonde) dataset.
As shown in Figure 3, NESTER consistently outperformed or matched all baselines. On the IHDP dataset, NESTER achieved an Average Causal Effect (ACE) error of 0.05, a significant improvement over TARNet (0.26) and Dragonnet (0.16). On the Twins dataset, NESTER reduced the ACE error to 0.0028, surpassing the best neural baseline, SNet (0.0040). Notably, the "dPads" variant of NESTER maintained high performance while remaining faster to run than complex fixed-architecture models like SNet on larger datasets.
Conclusion
NESTER represents a significant shift from manual architecture design to automated synthesis in causal inference. By leveraging a DSL of causal primitives and efficient program synthesis, it discovers tailored models that adapt to the unique biases of a given dataset. Future work will focus on expanding the DSL and relaxing the assumption of no latent confounding to handle even more complex real-world scenarios.
Summary updated