Abstract
Causal inference explores the causation between actions and the consequent rewards on a covariate set. Recently deep learning has achieved a remarkable performance in causal inference, but existing statistical theories cannot well explain such an empirical success, especially when the covariates are high-dimensional. Most theoretical results in causal inference are asymptotic, suffer from the curse of dimensionality, and only work for the finite-action scenario. To bridge such a gap between theory and practice, this paper studies doubly robust off-policy learning by deep neural networks. When the covariates lie on a low-dimensional manifold, we prove nonasymptotic regret bounds, which converge at a fast rate depending on the intrinsic dimension of the manifold. Our results cover both the finite- and continuous-action scenarios. Our theory shows that deep neural networks are adaptive to the low-dimensional geometric structures of the covariates, and partially explains the success of deep learning for causal inference.
Introduction
In the realm of causal inference, off-policy learning is a critical task: it involves using historical, observational data to derive an optimal decision-making policy. This is particularly relevant in fields like medicine or automated systems, where we might analyze past records to determine the best treatment strategy or control logic. However, this task is notoriously difficult because we only observe outcomes for actions that were actually taken—never the "counterfactual" outcomes of alternative choices.
While Deep Neural Networks (DNNs) have shown empirical brilliance in handling high-dimensional covariates (such as medical imaging or complex sensor data), theoretical justification for this success has lagged. Traditional statistical models often succumb to the "curse of dimensionality," where the data requirements grow exponentially with the number of input features. This paper addresses this gap by proving that DNNs can bypass this curse by exploiting the intrinsic geometric structure of the data.
The Manifold Hypothesis and Deep Learning
The core premise of this research is the Manifold Assumption. It posits that while data may be presented in a high-dimensional space (dimension $D$), it actually concentrates on a much simpler, low-dimensional manifold (dimension $d$, where $d \ll D$). A common analogy is a crumpled piece of paper: while it exists in three-dimensional space, its intrinsic properties are defined by its two-dimensional surface.
The authors demonstrate that deep ReLU (Rectified Linear Unit) networks are naturally "adaptive" to this structure. Instead of the learning rate being hindered by the ambient dimension $D$, the error rates—specifically the "regret bounds"—depend on the intrinsic dimension $d$. This provides a mathematical foundation for why deep learning remains effective even when the input vector is extremely large.
Methodology: Doubly Robust Learning
To achieve these results, the paper employs a Doubly Robust (DR) estimator. This framework integrates two distinct models:
- The Reward Model: Predicts the expected outcome for a given action and set of covariates.
- The Propensity Model: Estimates the probability that a specific action was taken in the original observational data.
The "double robustness" stems from the fact that the final policy remains accurate if either the reward model or the propensity model is correctly specified. By parameterizing both models with deep ReLU networks, the authors leverage the expressive power of DNNs to capture complex relationships within the manifold.
Key Theoretical Contributions
Non-Asymptotic Regret Bounds
Unlike previous studies that focus on "asymptotic" behavior (what happens as the sample size $n$ goes to infinity), this paper provides non-asymptotic bounds. These are more practical for real-world engineering, as they describe performance with a finite number of samples. For finite action spaces, the regret—the difference between the learned policy's performance and the theoretical optimum—vanishes at a rate of approximately $n^{-\frac{\alpha}{2\alpha + d}}$. Here, $\alpha$ represents the smoothness of the functions (the Hölder index).
Extension to Continuous Action Spaces
A significant hurdle in causal inference theory is moving beyond binary or finite actions (e.g., "Treatment A" vs. "Treatment B") to continuous actions (e.g., determining the exact optimal dosage of a drug). This paper identifies the first non-asymptotic regret bounds for continuous-action scenarios by utilizing a discretization technique. They prove that if the reward function is "unimodal" (possessing a single clear peak), the regret converges at a fast, predictable rate.
Breaking the Smoothness Barrier
Historically, researchers believed that high-dimensional learning required "super-smooth" functions to be tractable. This research shows that as long as the manifold structure exists, fast convergence is possible even with standard levels of smoothness (arbitrary Hölder index $\alpha \ge 1$), making the theory applicable to a much wider range of real-world functions.
Discussion and Limitations
While this work provides a vital "missing link" between theory and practice, several challenges remain:
- Optimization: The proofs assume the ability to find the global optimum of the neural network's loss function. In practice, training is non-convex and complex, though "overparameterization" in modern networks often helps practitioners reach near-optimal solutions.
- Manifold Identification: The theory assumes the data lies on a smooth Riemannian manifold. Real-world data may have "corners," self-intersections, or noise that pushes points slightly off the manifold.
- Computational Complexity: While the sample complexity (amount of data) is optimized, the computational resources required to train two-stage deep models can be significant.
Conclusion
By proving that deep neural networks are adaptive to the low-dimensional geometric structures of covariates, this paper offers a compelling explanation for the success of deep learning in causal inference. It moves the field beyond the curse of dimensionality, providing a rigorous framework for off-policy learning in both finite and continuous action spaces.
Summary updated