Abstract
In many empirical applications, the exact functional form of a conditional expectation is unknown. Series methods offer a flexible solution by approximating these unknown functions using $k$ basis functions, where $k$ grows with the sample size $n$. This paper advances the asymptotic theory for least squares series estimators by introducing four technical innovations: sharp Laws of Large Numbers (LLNs) for matrices derived from noncommutative Khinchin inequalities, bounds on the Lebesgue factor, new maximal inequalities for processes with diverging entropy integrals, and strong approximations to series-type processes. These tools allow for a significant weakening of the requirements for the number of basis functions, moving from the traditional $k^2/n \to 0$ to a more flexible $k/n \to 0$ (up to log factors). The paper establishes $L_2$ rates, pointwise central limit theorems (CLTs), and uniform confidence bands for the entire nonparametric function and its linear functionals, even under model misspecification.
Introduction
In econometrics and statistics, estimating the relationship between variables—specifically the conditional expectation function—often requires moving beyond rigid parametric models. Series estimation is a foundational nonparametric technique that approximates these relationships using a linear combination of $k$ basis functions, such as polynomials, splines, or wavelets. As the sample size $n$ increases, researchers can increase $k$ to reduce approximation bias, though this typically increases the variance of the estimator.
For decades, the theoretical benchmark for these estimators was established by Newey (1997), which required that the number of basis functions grow slowly relative to the sample size ($k^2/n \to 0$) to ensure predictable asymptotic behavior. This paper bridges the gap between that restrictive theoretical bound and the practical need for more complex models, providing a major update to the asymptotic theory of least squares series estimators.
Technical Innovations
The authors introduce four "new ingredients" to the series estimation literature that facilitate more robust inference:
- Non-commutative Khinchin Inequalities: These are utilized to derive sharp LLNs for matrices. This mathematical framework is the primary driver behind weakening the growth condition of $k$ relative to $n$.
- Lebesgue Factor Bounds: These bounds control the ratio between the supremum norm ($L^\infty$) and the average error ($L^2$), ensuring that the approximation remains stable across the entire domain.
- Maximal Inequalities: The paper develops new bounds for empirical processes where traditional entropy integrals might otherwise diverge, allowing for a broader range of basis functions.
- Strong Approximations: By showing that the distribution of the estimated function is "close" to a Gaussian process, the authors enable the construction of valid uniform confidence bands.
These tools are applied to a wide variety of bases, including Legendre polynomials, Fourier series, B-splines, wavelets, and local polynomial partitions.
Key Insights and Results
The $k/n$ Breakthrough
The most significant contribution is the realization that the stability of the Gram matrix (the matrix representing interactions between basis functions) can be guaranteed with far fewer observations than previously thought. By reducing the requirement from $k^2/n \to 0$ to $k/n \to 0$ (modulo log factors) for bounded or local bases, the authors allow researchers to utilize much more complex models on smaller datasets without sacrificing theoretical validity.
Uniformity and Linear Functionals
While previous literature often focused on pointwise convergence, this paper proves that series estimators can achieve the optimal uniform rate of convergence under general conditions. This is critical for constructing uniform confidence bands, which allow researchers to perform inference on the entire shape of a function or its derivatives (e.g., marginal effects) rather than just at specific points. This is particularly relevant for economists interested in "average partial derivatives" or other linear functionals of the conditional expectation function.
Robustness to Misspecification
The paper demonstrates that pointwise CLTs remain valid even if the model is "incorrectly specified"—meaning the approximation error does not vanish as $n$ grows. This provides a theoretical safety net for empirical researchers who may not know the true underlying complexity of the data-generating process.
Empirical Illustration: 1990 US Census Data
To demonstrate the practical utility of these theoretical results, the authors analyze 1990 US Census data, focusing on the conditional expectation of log wages given years of education for prime-age men. This application serves as a benchmark, showing that different series bases (such as splines versus polynomials) yield consistent, high-quality approximations when the $k/n$ conditions are met. The results illustrate how the new theory supports the use of flexible functional forms in standard labor economics problems.
Limitations and Future Directions
While the paper significantly advances series estimation theory, certain areas remain open for further exploration:
- I.I.D. Data: The core results are built on the assumption of independent and identically distributed data. While the authors discuss extensions to dependent data (specifically $\beta$-mixing processes), the primary framework is i.i.d.-centric.
- Bounded Regressors: The proofs rely on the covariate space having a bounded diameter. While this can be relaxed, doing so introduces substantial technical complexity.
- Practical Tuning: A persistent challenge in nonparametric statistics is the selection of $k$. While the paper provides the theoretical limits for $k$, it does not prescribe a universal, data-driven procedure for choosing the exact number of terms to achieve optimal "undersmoothing" for inference in practice.
Conclusion
This research provides a modern foundation for series estimation, offering researchers greater flexibility in model specification and more powerful tools for uniform inference. By weakening the growth conditions for basis functions and establishing strong Gaussian approximations, the paper ensures that least squares series methods remain a robust and theoretically sound choice for nonparametric analysis in the era of large-scale data.
Summary updated