Technical Report: Higher Order Influence Functions and Minimax Estimation of Nonlinear Functionals

James M. Robins, Lingling Li, Eric Tchetgen Tchetgen, Aad van der VaartarXiv 2016 · DOI · arXiv

Abstract

This technical report serves as a comprehensive expansion of the seminal work by Robins et al. (2008), providing the rigorous mathematical foundations for Higher Order Influence Functions (HOIFs). While the 2008 publication introduced the theory of HOIFs for inference in semi- and non-parametric models, this manuscript includes extensive results and proofs omitted from the original paper due to space constraints. Key additions include the derivation of HOIFs for products of functionals and their application to estimating the mean of a response under monotone missingness and time-varying confounding. The report establishes a framework for minimax-optimal estimation in high-dimensional settings where standard first-order semiparametric methods fail.

Introduction

In modern statistics and causal inference, researchers often seek to estimate a specific functional—such as an average treatment effect—while adjusting for high-dimensional covariates. Standard semiparametric theory relies on first-order influence functions to construct estimators that converge at the parametric $\sqrt{n}$ rate. However, these methods are contingent upon the underlying nuisance functions (e.g., propensity scores or outcome regressions) being sufficiently smooth relative to the dimension of the covariates.

When the dimension of the data is high, the required smoothness often becomes physically implausible, leading to significant bias and the collapse of standard inference. This paper addresses this "curse of dimensionality" by developing a theory of Higher Order Influence Functions, enabling robust inference even when nuisance parameters are complex or "rough."

The Curse of Dimensionality and the 1/4 Threshold

A central contribution of this work is the identification of a critical boundary in estimation theory. The feasibility of $\sqrt{n}$ convergence is determined by the ratio of the smoothness of the nuisance functions ($\beta$) to the dimension of the covariates ($d$).

  • The Smooth Regime ($\beta/d > 1/4$): In this regime, the combined smoothness is high enough that first-order semiparametric theory holds. HOIF estimators in this range achieve the parametric rate of $n^{-1/2}$ and are semiparametrically efficient.
  • The Rough Regime ($\beta/d \leq 1/4$): Here, the curse of dimensionality prevents $\sqrt{n}$ rates. The authors prove that in this difficult regime, their HOIF-based estimators achieve the minimax rate of $n^{-\frac{4\beta/d}{4\beta/d+1}}$, which is the fastest rate mathematically possible.

Methodology: Higher Order Influence Functions

The authors extend first-order theory by constructing estimators based on higher-order U-statistics. This involves several key technical innovations:

The Truncated Dirac Approach

Mathematically, the existence of higher-order influence functions is often hindered by the requirement of a Dirac delta function, which is not square-integrable. The authors circumvent this by approximating the Dirac delta through a projection operator onto a finite-dimensional subspace of dimension $k$. This "truncated Dirac" approach allows for the construction of valid influence functions for a "truncated parameter" $\tilde{\psi}_k$, which remains close to the true parameter of interest.

Multi-Robustness

The paper generalizes the well-known concept of "double robustness" to multi-robustness. In a standard doubly robust setup, an estimator is consistent if either the treatment model or the outcome model is correctly specified. With an $m^{th}$-order HOIF estimator, the bias is reduced if any one of several different nuisance models is correctly specified, providing multiple layers of protection against model misspecification.

Bias-Variance Tradeoff

The accuracy of the resulting estimators is governed by two primary parameters: the dimension of the projection space ($k$) and the order of the influence function ($m$). Increasing $m$ reduces estimation bias but increases the complexity and variance of the U-statistic. The report provides recursive formulas to derive $m^{th}$-order influence functions from $(m-1)^{th}$-order functions, facilitating practical implementation.

Applications in Causal Inference and Missing Data

While primarily theoretical, the report demonstrates the utility of HOIFs through structured examples in biostatistics:

  1. Monotone Missing Data: The authors show how to derive HOIFs for the mean of a response $Y$ when data is missing at random over time, a common challenge in longitudinal studies.
  2. Time-Varying Confounding: The results are applied to estimating the causal effect of time-dependent treatments where confounding variables also change over time, a scenario where standard methods frequently struggle with dimensionality.
  3. Random Design Advantage: The paper notes that random covariate designs (as opposed to fixed designs) can actually improve convergence rates, as the estimators can exploit the natural clustering of data points.

Limitations and Future Directions

Despite the theoretical breakthrough, the authors identify several hurdles for practical application:

  • Unknown Constants: While the rates of convergence are established, the exact constants (the magnitude of the error) remain difficult to determine, complicating small-sample performance assessment.
  • Honest Confidence Intervals: Constructing confidence intervals that maintain coverage regardless of the true function's roughness remains an open problem, as it requires knowledge of the model's bias constant.
  • Computational Cost: As the order $m$ increases, the computational burden of calculating higher-order U-statistics grows significantly, potentially limiting the use of very high-order estimators in practice.

Conclusion

This technical report provides the most detailed account to date of the theory of Higher Order Influence Functions. By bridging the gap between parametric and non-parametric estimation, it offers a rigorous path forward for inference in high-dimensional settings, providing the mathematical tools necessary to break the curse of dimensionality.

Summary updated

Abstract

Robins et al, 2008, published a theory of higher order influence functions for inference in semi- and non-parametric models. This paper is a comprehensive manuscript from which Robins et al, was drawn. The current paper includes many results and proofs that were not included in Robins et al due to space limitation. Particular results contained in the present paper that were not reported in Robins et al include the following. Given a set of functionals and their corresponding higher order influence functions, we show how to derive the higher order influence function of their product. We apply this result to obtain higher order influence functions and associated estimators for the mean of a response Y subject to monotone missingness under missing at random. These results also apply to estimating the causal effect of a time dependent treatment on an outcome Y in the presence of time-varying confounding. Finally, we include an appendix that contains proofs for all theorems that were stated without proof in Robins et al, 2008. The initial part of the paper is closely related to Robins et al, the latter parts differ.
BibTeX
@misc{robins2016technical,
  author = {James M. Robins and Lingling Li and Eric Tchetgen Tchetgen and Aad van der Vaart},
  title = {{Technical Report: Higher Order Influence Functions and Minimax Estimation of Nonlinear Functionals}},
  year = {2016},
  eprint = {1601.05820},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1601.05820},
}
Figure