NESTER: An Adaptive Neurosymbolic Method for Causal Effect Estimation

Abbavaram Gowtham Reddy, Vineeth N. Balasubramanian

Introduction

Estimating causal effects—such as determining the impact of a medical treatment or a policy change—is a fundamental challenge in data science. While Randomized Controlled Trials (RCTs) are the gold standard, they are often impractical or unethical. Researchers must instead rely on observational data, which presents two major hurdles: the "fundamental problem of causal inference" (we never observe the counterfactual) and selection bias (treatment groups often differ systematically from control groups).

Existing deep learning approaches attempt to mitigate these issues using fixed architectures like TARNet or CFR. However, no single architecture is optimal for every dataset. NESTER (NEuroSymbolic causal effecT EstimatoR) addresses this by moving away from manual design toward automated model synthesis. It adaptively "programs" its own neural network architecture to suit the specific characteristics of a given dataset.

Methodology

NESTER employs Neurosymbolic Program Synthesis (NPS) to navigate a search space defined by a Domain Specific Language (DSL). This DSL consists of primitives tailored for causal inference, including standard MLP heads, shared representation layers, propensity score modules, and alignment modules that use Maximum Mean Discrepancy (MMD) to balance treatment groups.

NESTER synthesizes programs (right) that can replicate or improve upon established architectures like TARNet (left).

To find the optimal architecture, NESTER treats model design as a search problem. It uses the $A^*$ algorithm to explore a "program tree" of potential architectures. To make this search efficient, it utilizes Neural Admissible Relaxations (NEAR), which use neural networks to estimate the potential performance of incomplete programs, allowing the algorithm to prune suboptimal paths early.

Experimental Results

The authors evaluated NESTER against 16 baseline methods across three major benchmarks: IHDP (infant health), Twins (US twin births), and Jobs (employment training).

NESTER consistently outperformed state-of-the-art models like Dragonnet, CFR, and GANITE. For instance, on the IHDP dataset, NESTER achieved an error rate ($\epsilon_{ACE}$) of 0.05, a significant improvement over the next best models like SNet (0.09). It also achieved the best out-of-sample performance on the Twins dataset with an error of 0.0028.

Comparative performance of NESTER against baselines, showing superior results across IHDP, Twins, and Jobs datasets.

Discussion and Conclusion

NESTER represents a shift toward self-designing causal models. By defining the "physics" of causal inference through a DSL, the framework can discover novel combinations—such as using propensity scores to branch neural heads—that human engineers might overlook. This neurosymbolic synergy provides both the interpretability of symbolic programs and the high-capacity learning of deep neural networks.

While highly effective, NESTER's performance is currently bounded by its DSL and the assumption of "no latent confounding." Future research aims to expand the library of causal primitives and address unobserved confounders to further enhance the framework's robustness.

Summary updated

Abstract

Estimating treatment effects from observational data is a central problem in causal inference. Methods to solve this problem exploit inductive biases and heuristics from causal inference to design multi-head neural network architectures and regularizers. In this work, we propose to use neurosymbolic program synthesis, a data-efficient, and interpretable technique, to solve the treatment effect estimation problem. We theoretically show that neurosymbolic programming can solve the treatment effect estimation problem. By designing a Domain Specific Language (DSL) for treatment effect estimation based on the inductive biases used in literature, we argue that neurosymbolic programming is a better alternative to treatment effect estimation than traditional models. Our empirical study reveals that our model, which implicitly encodes inductive biases in a DSL, achieves better performance on benchmark datasets than the state-of-the-art models.
BibTeX
@inproceedings{reddy2024nester,
  author = {Abbavaram Gowtham Reddy and Vineeth N. Balasubramanian},
  title = {{NESTER: An Adaptive Neurosymbolic Method for Causal Effect Estimation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i13.29398},
}
Figure