Abstract
This technical report serves as a comprehensive expansion of the seminal work by Robins et al. (2008), providing the rigorous mathematical foundations for Higher Order Influence Functions (HOIFs). While the 2008 publication introduced the theory of HOIFs for inference in semi- and non-parametric models, this manuscript includes extensive results and proofs omitted from the original paper due to space constraints. Key additions include the derivation of HOIFs for products of functionals and their application to estimating the mean of a response under monotone missingness and time-varying confounding. The report establishes a framework for minimax-optimal estimation in high-dimensional settings where standard first-order semiparametric methods fail.
Introduction
In modern statistics and causal inference, researchers often seek to estimate a specific functional—such as an average treatment effect—while adjusting for high-dimensional covariates. Standard semiparametric theory relies on first-order influence functions to construct estimators that converge at the parametric $\sqrt{n}$ rate. However, these methods are contingent upon the underlying nuisance functions (e.g., propensity scores or outcome regressions) being sufficiently smooth relative to the dimension of the covariates.
When the dimension of the data is high, the required smoothness often becomes physically implausible, leading to significant bias and the collapse of standard inference. This paper addresses this "curse of dimensionality" by developing a theory of Higher Order Influence Functions, enabling robust inference even when nuisance parameters are complex or "rough."
The Curse of Dimensionality and the 1/4 Threshold
A central contribution of this work is the identification of a critical boundary in estimation theory. The feasibility of $\sqrt{n}$ convergence is determined by the ratio of the smoothness of the nuisance functions ($\beta$) to the dimension of the covariates ($d$).
- The Smooth Regime ($\beta/d > 1/4$): In this regime, the combined smoothness is high enough that first-order semiparametric theory holds. HOIF estimators in this range achieve the parametric rate of $n^{-1/2}$ and are semiparametrically efficient.
- The Rough Regime ($\beta/d \leq 1/4$): Here, the curse of dimensionality prevents $\sqrt{n}$ rates. The authors prove that in this difficult regime, their HOIF-based estimators achieve the minimax rate of $n^{-\frac{4\beta/d}{4\beta/d+1}}$, which is the fastest rate mathematically possible.
Methodology: Higher Order Influence Functions
The authors extend first-order theory by constructing estimators based on higher-order U-statistics. This involves several key technical innovations:
The Truncated Dirac Approach
Mathematically, the existence of higher-order influence functions is often hindered by the requirement of a Dirac delta function, which is not square-integrable. The authors circumvent this by approximating the Dirac delta through a projection operator onto a finite-dimensional subspace of dimension $k$. This "truncated Dirac" approach allows for the construction of valid influence functions for a "truncated parameter" $\tilde{\psi}_k$, which remains close to the true parameter of interest.
Multi-Robustness
The paper generalizes the well-known concept of "double robustness" to multi-robustness. In a standard doubly robust setup, an estimator is consistent if either the treatment model or the outcome model is correctly specified. With an $m^{th}$-order HOIF estimator, the bias is reduced if any one of several different nuisance models is correctly specified, providing multiple layers of protection against model misspecification.
Bias-Variance Tradeoff
The accuracy of the resulting estimators is governed by two primary parameters: the dimension of the projection space ($k$) and the order of the influence function ($m$). Increasing $m$ reduces estimation bias but increases the complexity and variance of the U-statistic. The report provides recursive formulas to derive $m^{th}$-order influence functions from $(m-1)^{th}$-order functions, facilitating practical implementation.
Applications in Causal Inference and Missing Data
While primarily theoretical, the report demonstrates the utility of HOIFs through structured examples in biostatistics:
- Monotone Missing Data: The authors show how to derive HOIFs for the mean of a response $Y$ when data is missing at random over time, a common challenge in longitudinal studies.
- Time-Varying Confounding: The results are applied to estimating the causal effect of time-dependent treatments where confounding variables also change over time, a scenario where standard methods frequently struggle with dimensionality.
- Random Design Advantage: The paper notes that random covariate designs (as opposed to fixed designs) can actually improve convergence rates, as the estimators can exploit the natural clustering of data points.
Limitations and Future Directions
Despite the theoretical breakthrough, the authors identify several hurdles for practical application:
- Unknown Constants: While the rates of convergence are established, the exact constants (the magnitude of the error) remain difficult to determine, complicating small-sample performance assessment.
- Honest Confidence Intervals: Constructing confidence intervals that maintain coverage regardless of the true function's roughness remains an open problem, as it requires knowledge of the model's bias constant.
- Computational Cost: As the order $m$ increases, the computational burden of calculating higher-order U-statistics grows significantly, potentially limiting the use of very high-order estimators in practice.
Conclusion
This technical report provides the most detailed account to date of the theory of Higher Order Influence Functions. By bridging the gap between parametric and non-parametric estimation, it offers a rigorous path forward for inference in high-dimensional settings, providing the mathematical tools necessary to break the curse of dimensionality.
Summary updated