Functional Linear Regression: Dependence and Error Contamination

Cheng Chen, Shaojun Guo, Xinghao QiaoJournal of Business and Economic Statistics 2020 · DOI

The London School of Economics and Political Science

Abstract

Functional linear regression (FLR) is a fundamental topic in functional data analysis. It is commonly assumed that samples of the functional predictor are independent realizations of an underlying stochastic process, observed over a grid contaminated by i.i.d. measurement errors. However, in practice, dynamical dependence across curves is common, and parametric assumptions on error covariance can be unrealistic. This article considers functional linear regression with serially dependent observations of the functional predictor, where the contamination is genuinely functional with a fully nonparametric covariance structure. We propose a novel autocovariance-based generalized method-of-moments (AGMM) estimate of the slope function. We also develop a nonparametric smoothing approach to handle partially observed functional predictors. The asymptotic properties of the resulting estimators are established, and we demonstrate that our proposed method significantly outperforms competing methods through simulations and an analysis of a public financial dataset.

1. Introduction

In Functional Data Analysis (FDA), researchers treat data as continuous curves—such as daily temperature profiles or intraday stock price movements. A primary challenge in this field is Functional Linear Regression (FLR). Traditional FLR frameworks often rely on two simplifying assumptions: the independence of curve samples and the presence of simple "white noise" measurement errors.

A recent study by Cheng Chen, Shaojun Guo, and Xinghao Qiao challenges these assumptions. They introduce a robust framework specifically designed for "curve time series" where observations are serially dependent and noise follows a complex, unknown covariance structure.

2. The Challenge: Serial Dependence and Complex Noise

Standard FLR models typically assume the functional predictor $X(t)$ is observed with i.i.d. measurement errors. In reality, two phenomena often complicate this:

  • Serial Dependence: Curves in a sequence are often related (e.g., today’s market volatility is inherently linked to yesterday’s).
  • Genuinely Functional Noise: Measurement errors may not be simple random spikes but can have a complex, nonparametric covariance structure where errors at different points on the curve are correlated.

Existing Covariance-based Least Squares (CLS) methods often fail in these settings because they cannot distinguish the signal's covariance from the noise's covariance, leading to significant estimation bias.

3. The AGMM Approach: Filtering Through Autocovariance

The core innovation of the proposed method is the use of autocovariance as a natural filter. For an observed noisy predictor $W_t = X_t + e_t$, where $e_t$ is white noise, the autocovariance at any lag $k \neq 0$ satisfies: $$Cov(W_t, W_{t+k}) = Cov(X_t, X_{t+k})$$

Since the noise $e_t$ is uncorrelated over time, the autocovariance of the observed data is identical to that of the underlying clean signal $X_t$. This allows the model to "see through" complex noise without needing to model its specific structure.

3.1 Methodological Components

  • Moment Conditions: The authors develop a Generalized Method-of-Moments (GMM) framework based on these noise-free autocovariances.
  • Regularization: To ensure stable and smooth estimates, the method employs basis expansion with orthonormal functions.
  • Handling Sparsity: For scenarios where curves are only partially observed at random time points, a smoothing version of the estimator is developed using local linear surface smoothing.
  • Phase Transitions: The authors established theoretical convergence rates that reveal "phase transitions," showing exactly how the density of observations per curve affects the accuracy of the final regression model.

4. Empirical Evidence

The AGMM method was validated through extensive simulations and a real-world financial application:

  • Simulations: Across five different scenarios involving varying dimensionality and noise complexity, AGMM significantly outperformed traditional CLS and standard GMM, particularly when noise was high or complex.
  • Financial Market Analysis: The method was applied to 2017 intraday price data for the S&P 500 index and individual stocks (Adobe, Johnson & Johnson, and PepsiCo). Using the market index curve to predict individual stock returns, the AGMM method achieved a lower Mean Squared Prediction Error (MSPE) than competing methods across multiple time horizons.

5. Conclusion

The AGMM approach provides a robust solution for functional linear regression in the presence of serial dependence and complex noise. By utilizing the time-series nature of the data as a feature rather than a limitation, the method effectively mitigates measurement errors that would otherwise bias traditional models. While the method requires the presence of serial dependence, it offers a powerful tool for analyzing high-frequency financial data and other curve time series.

Summary based on research by Cheng Chen, Shaojun Guo, and Xinghao Qiao.

Summary updated

Abstract

Functional linear regression is an important topic in functional data analysis. It is commonly assumed that samples of the functional predictor are independent realizations of an underlying stochastic process, and are observed over a grid of points contaminated by iid measurement errors. In practice, however, the dynamical dependence across different curves may exist and the parametric assumption on the error covariance structure could be unrealistic. In this article, we consider functional linear regression with serially dependent observations of the functional predictor, when the contamination of the predictor by the white noise is genuinely functional with fully nonparametric covariance structure. Inspired by the fact that the autocovariance function of observed functional predictors automatically filters out the impact from the unobservable noise term, we propose a novel autocovariance-based generalized method-of-moments estimate of the slope function. We also develop a nonparametric smoothing approach to handle the scenario of partially observed functional predictors. The asymptotic properties of the resulting estimators under different scenarios are established. Finally, we demonstrate that our proposed method significantly outperforms possible competing methods through an extensive set of simulations and an analysis of a public financial dataset.
BibTeX
@article{chen2020functional,
  author = {Cheng Chen and Shaojun Guo and Xinghao Qiao},
  title = {{Functional Linear Regression: Dependence and Error Contamination}},
  journal = {Journal of Business and Economic Statistics},
  year = {2020},
  doi = {10.1080/07350015.2020.1832503},
}
Figure