Regression with Functional Errors-in-Predictors: A Generalized Method-of-Moments Approach

Xinghao Qiao, Cheng Chen, Shaojun GuoarXiv 2018 · arXiv

TL;DR

This paper introduces a novel method for functional linear regression when predictors are both contaminated by complex measurement errors and exhibit serial dependence (i.e., they are a time series of curves). By leveraging the autocovariance of observed curves, the authors "filter out" unobservable noise without needing restrictive parametric assumptions, significantly improving estimation accuracy in time-series contexts.

What Problem Does This Paper Address?

In Functional Data Analysis (FDA), we often try to predict a response (like a stock's daily return) using a predictor that is a continuous curve (like the intraday price movement of the entire market).

Traditional models hit two major roadblocks in the real world:

  1. Measurement Error: We rarely observe the "true" underlying curve. Instead, we see data points contaminated by noise. Standard Least Squares (LS) estimators become biased when the predictor itself is noisy.
  2. Serial Dependence: Most models assume each curve is an independent sample. However, data like daily financial returns or weather patterns are "curve time series"—today's curve is likely related to yesterday's.

Existing solutions often require a "low noise" setting or very specific mathematical assumptions about the shape of the noise. This paper tackles the "genuinely functional" error—noise with a fully non-parametric structure—in the presence of dependent data.

Methods

The authors propose an Autocovariance-based Generalized Method-of-Moments (AGMM) approach.

The "secret sauce" lies in a simple mathematical property: if you look at the covariance between an observed curve at time $t$ and another at time $t+k$ (where $k \neq 0$), the white-noise measurement error term effectively vanishes. This is because the noise at time $t$ is uncorrelated with the noise at time $t+k$.

Key steps include: * Instrumental Variables: They use lagged versions of the functional predictors as "instruments" to handle the errors-in-predictors problem. * GMM Framework: They define population moment conditions based on these lags and minimize a quadratic form to find the best-fitting slope parameter. * Basis Expansion: To prevent "wiggly" or overfitted results (under-smoothing), they implement a version of the model using orthonormal basis functions (like B-splines or Fourier series) to ensure the estimated slope function is smooth and physically meaningful. * Generalization: The method is designed to work whether the response variable is a single number (scalar) or another curve (functional).

Datasets

The authors validated their approach using:

  • Synthetic Simulations: Two complex scenarios where signal and noise components overlap in ways that usually confuse standard covariance-based methods.
  • Public Financial Dataset: One-minute resolution price data for the S&P 100 index and its constituent stocks (Amazon, GE, and Johnson & Johnson) from the year 2017. They specifically modeled Cumulative Intraday Return (CIDR) trajectories.

Key Insights and Ideas

  • The Autocovariance Filter: The most elegant insight is that the autocovariance operator of observed curves automatically "cleans" the data of measurement error. You don't need to model the noise; you just need to look at the relationship between curves at different time steps.
  • Bridge to Dimension Reduction: The mathematical operator ($K$) derived for this regression is identical to one used in previous literature for dimension reduction in curve time series. This suggests a deep link between "finding the signal" for dimension reduction and "finding the signal" for regression.
  • Non-Parametric Flexibility: Unlike previous "errors-in-variables" models that require the noise to follow a specific pattern (like a "banded" covariance), AGMM makes no such assumptions, making it much more robust for real-world sensors or financial markets.

Key Results

  • Superior Accuracy: In simulations, AGMM consistently achieved lower Mean Integrated Squared Error (MISE) than standard Least Squares or traditional GMM, especially as the sample size ($n$) increased.
  • Stability in Finance: When applied to S&P 100 data, AGMM produced smoother, more stable "sensitivity" curves compared to standard methods.
  • Predictive Pattern: The real-world analysis revealed that a stock's return is most sensitive to the market's behavior toward the end of the trading day, a finding that was much clearer using the AGMM estimator than the noisy results from traditional competitors.

Limitations and Future Work

  • Convergence Rates: The authors note that their theoretical convergence rate is slightly slower than the minimax rate for "clean" data. It remains an open question whether a faster rate is mathematically possible under these noisier conditions.
  • High-Dimensional Predictors: The current model focuses on a single functional predictor. Future research could extend this to "Big Data" scenarios involving many different functional predictors simultaneously.
  • Tuning Parameters: While the method is robust, it still requires choosing the number of lags ($L$) and the number of basis functions ($J$). The authors suggest $L=5$ works well empirically, but a more automated selection process could be developed.

Summary updated

Abstract

Functional regression is an important topic in data analysis. Traditionally, one often assumes that samples of the predictor are independent realizations of an underlying stochastic process, and are observed over a grid of points contaminated by independent and identically distributed measurement errors. In practice, however, the dynamical dependence across different curves may exist and the parametric assumption on the measurement error covariance structure could be unrealistic. In this paper, we consider linear regression with serially dependent predictors, when the contamination of predictors by the measurement error is genuinely functional with fully nonparametric covariance structure. Inspired by the fact that the autocovariance operator of observed predictors automatically filters out the impact from the unobservable measurement error, we propose a novel autocovariance-based generalized method-of-moments estimate of the slope parameter. The asymptotic properties of the resulting estimators under different scenarios are established. We also demonstrate that our proposed method significantly outperforms possible competitors through intensive simulation studies. Finally, the proposed method is applied to a public financial dataset, revealing some interesting findings.
BibTeX
@misc{qiao2018regression,
  author = {Xinghao Qiao and Cheng Chen and Shaojun Guo},
  title = {{Regression with Functional Errors-in-Predictors: A Generalized Method-of-Moments Approach}},
  year = {2018},
  eprint = {1806.05471},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1806.05471},
}

Loading related content...

Figure