Abstract: Understanding treatment effects on health-related outcomes using real-world data requires defining a causal parameter and imposing relevant identification assumptions to translate it into a statistical estimand. Semiparametric methods, like the targeted maximum likelihood estimator (TMLE), have been developed to construct asymptotically linear estimators of these parameters. To further establish the asymptotic efficiency of these estimators, two conditions must be met: 1) the relevant components of the data likelihood must fall within a Donsker class, and 2) the estimates of nuisance parameters must converge to their true values at a rate faster than $n^{-1/4}$. The Highly Adaptive LASSO (HAL) satisfies these criteria by acting as an empirical risk minimizer within a class of càdlàg functions with a bounded sectional variation norm, which is known to be Donsker. HAL achieves the desired rate of convergence, thereby guaranteeing the estimators' asymptotic efficiency. The function class over which HAL minimizes its risk is flexible enough to capture realistic functions while maintaining the conditions for establishing efficiency. Additionally, HAL enables robust inference for non-pathwise differentiable parameters, such as the conditional average treatment effect (CATE) and causal dose-response curve, which are important in precision health. While these parameters are often considered in machine learning literature, these applications typically lack proper statistical inference. HAL addresses this gap by providing reliable statistical uncertainty quantification that is essential for informed decision-making in health research.
Introduction
In modern health research, estimating the Average Treatment Effect (ATE) or Conditional Average Treatment Effects (CATE) from observational data is a primary goal. Researchers often turn to semiparametric methods like Targeted Maximum Likelihood Estimation (TMLE) to achieve this. However, TMLE imposes strict "speed limits" on the machine learning models (nuisance parameters) it utilizes. Specifically, these models must satisfy the Donsker condition—meaning they cannot be too volatile—and they must converge to the true value at a rate faster than $n^{-1/4}$.
Most popular machine learning algorithms, such as Random Forests or Neural Networks, do not consistently meet these theoretical requirements. This creates a significant gap between predictive power and the ability to draw valid causal conclusions with reliable statistical significance. The Highly Adaptive LASSO (HAL) was developed to bridge this gap, providing a framework that maintains the flexibility of "black-box" models while adhering to the rigorous mathematical constraints required for valid p-values and confidence intervals.
The Highly Adaptive LASSO Framework
HAL operates as an empirical risk minimizer within the space of càdlàg functions (right-continuous with left limits) characterized by a bounded sectional variation norm. The algorithm transforms data into a massive set of indicator basis functions (0th-order splines) and all their possible interactions. For a dataset with $d$ variables, this can result in up to $n \times 2^{d-1}$ features.
Following feature generation, HAL employs LASSO (L1 regularization) to select the most critical features. A key innovation in HAL is the use of "undersmoothing"—selecting a model slightly more complex than what standard cross-validation would suggest for pure prediction. This ensures the model satisfies the specific score equations necessary for valid statistical inference. As shown in Figure 1, HAL can fit complex, non-linear curves effectively by adjusting the variation norm constraint ($M$).
Theoretical Advantages and Convergence
HAL is mathematically optimized for causal inference. It is flexible enough to capture the jumps and non-linearities common in health data but remains constrained enough by its variation norm to satisfy the Donsker condition. Remarkably, HAL achieves the required $n^{-1/4}$ convergence rate regardless of the data's dimensionality, a rare property in nonparametric statistics.
Furthermore, an undersmoothed HAL can serve as a direct plug-in estimator for treatment effects. This allows researchers to obtain efficient estimates without the need to derive complex canonical gradients for every new research problem. HAL also demonstrates robustness to positivity violations through "Adaptive-TMLE," which stabilizes estimates when certain subgroups in a study have a very low probability of receiving a specific treatment.
Key Results and Inference
Simulations conducted by researchers at UC Berkeley demonstrate that HAL is competitive in predictive accuracy with established methods like Gradient Boosted Machines (GBMs). More importantly, it excels in providing reliable uncertainty quantification.
One of the most significant practical findings is the effectiveness of the "plateau selector" for the penalty term ($\lambda$). As illustrated in Figure 2, by analyzing the width and derivatives of the estimator across different values of $\lambda$, researchers can identify a stable region for inference. Using a nonparametric bootstrap in conjunction with this plateau selector provides significantly better confidence interval coverage in small samples compared to standard Wald-type methods.
Limitations and Future Directions
The primary challenge facing HAL is its computational intensity. Because the number of basis functions grows exponentially with the number of variables, the algorithm can be memory-intensive. Future research is focused on optimizing these computations, perhaps through $k$-means clustering to reduce knot points or screening basis functions more efficiently. Additionally, moving toward higher-order splines may allow for even faster convergence in cases where the underlying data is known to be smooth. Despite these hurdles, HAL represents a major step forward in integrating machine learning with rigorous statistical inference for health research.
Summary updated