Some new asymptotic theory for least squares series: Pointwise and uniform results

Alexandre Belloni, Victor Chernozhukov, Denis Chetverikov, Kengo Kato

TL;DR

This paper develops new theoretical tools to analyze series estimators for conditional expectations, a flexible approach in econometrics. The authors significantly relax existing conditions on the number of basis functions used, enabling more efficient estimation and providing sharper results for both pointwise and uniform convergence, including novel uniform confidence bands for linear functionals.

What Problem Does This Paper Address?

In econometrics, it's common to estimate conditional expectations (like the expected value of a variable given others) using flexible functional forms when the true relationship is unknown. Series estimators offer this flexibility by approximating the unknown function using a sum of basis functions, where the number of terms ($k$) can grow with the sample size ($n$). However, existing theory often imposes strict conditions on how fast $k$ can grow relative to $n$ (e.g., $k^2/n \to 0$), limiting their practical application. This paper aims to relax these conditions, providing a more general and powerful theoretical framework for series estimation, particularly for complex economic models.

Methods

The authors introduce four key "ingredients" to advance the theory:

  1. Sharp LLNs for matrices: Leveraging non-commutative Khinchin inequalities to derive stronger laws of large numbers for matrix-valued random variables.
  2. Bounds on the Lebesgue factor: Providing tighter bounds on the ratio of $L_\infty$ to $L_2$ norms of approximation errors, which is crucial for uniform convergence.
  3. Maximal inequalities: Developing sharp inequalities for processes whose entropy integrals diverge at a certain rate.
  4. Strong approximations: Establishing strong approximations for series-type empirical processes, allowing for a more precise understanding of their behavior.

These tools enable the authors to:

  • Weaken conditions on $k$: Relaxing the growth rate of $k$ from $k^2/n \to 0$ to $k/n \to 0$ (up to log factors), a significant improvement previously only available for spline series.
  • Derive improved rates: Obtaining $L_2$ rates and pointwise central limit theorems under these weaker conditions, both for correctly and misspecified models.
  • Establish uniform results: Providing uniform rates and functional central limit theorems, including strong approximations for the entire nonparametric function estimate.
  • Develop inference tools: Deriving uniform rates, Gaussian approximations, and crucially, uniform confidence bands for linear functionals of the conditional expectation (e.g., derivatives, average derivatives).

Datasets

Not discussed in the paper. The paper focuses on theoretical development and uses simulated or hypothetical data for examples.

Key Insights and Ideas

  • Relaxing $k/n$ constraints is a game-changer: The ability to use more basis functions ($k$) relative to the sample size ($n$) without sacrificing theoretical guarantees opens up possibilities for more flexible and accurate modeling in complex economic scenarios.
  • Bridging the gap between theory and practice: The new theoretical tools allow for more realistic assumptions about the data and model structure, making the results more applicable to real-world econometric problems.
  • Uniform confidence bands are a major practical contribution: Providing confidence bands that hold uniformly across a range of inputs is essential for robust inference and decision-making in econometrics, especially when dealing with functions rather than single points.
  • Misspecification is handled: The theory explicitly addresses scenarios where the chosen series basis might not perfectly span the true function, a common issue in applied work.

Key Results

  • Weakened condition on $k$: The paper relaxes the condition on the number of approximating functions from $k^2/n \to 0$ to $k/n \to 0$ (up to log factors) for various series bases.
  • Optimal $L_2$ rates: The authors show that series estimators can achieve optimal $L_2$ rates of convergence for Hölder classes of functions under their relaxed conditions, which is a significant improvement over prior work.
  • Uniform convergence: They establish uniform rates of convergence for the series estimator, demonstrating that it can achieve optimal rates for certain function classes.
  • Uniform confidence bands: A key contribution is the derivation of uniform confidence bands for linear functionals of the conditional expectation function, providing a powerful new tool for inference.

Limitations and Future Work

  • Computational complexity: While the theory is advanced, the practical implementation of some of these methods might still be computationally intensive, especially with a large number of basis functions ($k$).
  • "Tailored" function classes: The paper notes that some series methods (like Legendre polynomials) might have poor Lebesgue factors in general but perform well on "tailored" function classes. Identifying and utilizing these classes in practice remains an area for further exploration.
  • Bootstrap methods: The paper mentions that weighted bootstrap methods could be used for inference, analogous to their Gaussian process approximations, but does not fully develop this aspect for brevity. This suggests a potential avenue for future research.
  • Further relaxation of conditions: While conditions are significantly relaxed, there might be scope for further improvements, particularly regarding the growth rates of $k$ and the behavior of the error terms.

Summary updated

Abstract

In applications it is common that the exact form of a conditional expectation is unknown and having flexible functional forms can lead to improvements. Series method offers that by approximating the unknown function based on $k$ basis functions, where $k$ is allowed to grow with the sample size $n$. We consider series estimators for the conditional mean in light of: (i) sharp LLNs for matrices derived from the noncommutative Khinchin inequalities, (ii) bounds on the Lebesgue factor that controls the ratio between the $L^\infty$ and $L_2$-norms of approximation errors, (iii) maximal inequalities for processes whose entropy integrals diverge, and (iv) strong approximations to series-type processes. These technical tools allow us to contribute to the series literature, specifically the seminal work of Newey (1997), as follows. First, we weaken the condition on the number $k$ of approximating functions used in series estimation from the typical $k^2/n \to 0$ to $k/n \to 0$, up to log factors, which was available only for spline series before. Second, we derive $L_2$ rates and pointwise central limit theorems results when the approximation error vanishes. Under an incorrectly specified model, i.e. when the approximation error does not vanish, analogous results are also shown. Third, under stronger conditions we derive uniform rates and functional central limit theorems that hold if the approximation error vanishes or not. That is, we derive the strong approximation for the entire estimate of the nonparametric function. We derive uniform rates, Gaussian approximations, and uniform confidence bands for a wide collection of linear functionals of the conditional expectation function.
BibTeX
@article{belloni2015some,
  author = {Alexandre Belloni and Victor Chernozhukov and Denis Chetverikov and Kengo Kato},
  title = {{Some new asymptotic theory for least squares series: Pointwise and uniform results}},
  journal = {Journal of Econometrics},
  year = {2015},
  doi = {10.1016/j.jeconom.2015.02.014},
}
Figure