Functional Regression on Manifold with Contamination

Zhenhua Lin, Fang Yao

Abstract

We propose a new method for functional nonparametric regression with a predictor that resides on a finite-dimensional manifold but is only observable in an infinite-dimensional space. Contamination of the predictor due to discrete/noisy measurements is also accounted for. By using functional local linear manifold smoothing, the proposed estimator enjoys a polynomial rate of convergence that adapts to the intrinsic manifold dimension and the contamination level. This is in contrast to the logarithmic convergence rate in the literature of functional nonparametric regression. We also observe a phase transition phenomenon regarding the interplay of the manifold dimension and the contamination level. We demonstrate that the proposed method has favorable numerical performance relative to commonly used methods via simulated and real data examples.

Introduction

Functional Data Analysis (FDA) treats data points as entire curves or surfaces, such as a patient’s blood pressure over time or satellite temperature readings. While powerful, FDA faces a daunting "curse of dimensionality." Because these curves technically live in infinite-dimensional space, traditional nonparametric regression methods are notoriously slow, often requiring astronomical amounts of data to achieve accuracy.

A new framework by Zhenhua Lin and Fang Yao, published in Biometrika, offers a solution. By recognizing that functional data often resides on low-dimensional "manifolds" (curved surfaces within the higher-dimensional space), they have developed a method that achieves much faster polynomial convergence rates, even when the data is contaminated by noise.

Methodology

The central challenge in functional nonparametric regression is the "small ball probability" problem: in infinite-dimensional spaces, data points are so spread out that finding "neighbors" for prediction is nearly impossible. The authors propose a three-step procedure to overcome this:

  1. Recovery: Reconstructing true functional predictors $X$ from discrete, noisy measurements using ridged local linear smoothing to produce estimated curves $\hat{X}$.
  2. Geometry Estimation: Identifying the intrinsic dimension ($d$) and the local tangent space—a flat approximation of the manifold—using a localized version of Principal Component Analysis (PCA).
  3. Regression: Projecting the functional data onto these estimated tangent spaces and performing local linear regression on the resulting coordinates.
The estimation process involves projecting observed functional data onto an estimated tangent space $T_x M$ of the manifold $M$ to perform local linear smoothing (LLS).

Theoretical Insights: The Phase Transition

The most significant contribution is the theoretical proof that manifold structure allows the convergence rate to jump from logarithmic to polynomial ($n^{-\alpha}$). This means the estimator becomes accurate much faster as sample sizes increase.

Geometric decomposition of the estimated tangent space basis $\hat{\varphi}_1$ and the associated error terms relative to the true tangent space $T_x M$.

The authors discovered a "phase transition" phenomenon: the speed of convergence depends on the interplay between the manifold's dimension ($d$) and the level of contamination ($m$, the number of points per curve). If $m$ is large, the manifold's complexity is the bottleneck; if data is sparse, contamination dominates.

Empirical Results

The proposed method (FREM) was validated using both complex simulations (e.g., SO(3) rotation groups and Klein bottles) and real-world datasets:

  • Tecator Meat Data: Predicting fat content from spectrometry curves. FREM achieved a root mean square error (RMSE) of 1.06, compared to 2.56 for standard functional linear models.
  • DTI Data: Predicting cognitive function from brain white matter microstructure profiles.
  • SBP Data: Predicting future average blood pressure from irregularly sampled longitudinal readings.

In all cases, FREM significantly outperformed standard functional Nadaraya-Watson and linear models, demonstrating that the low-dimensional manifold assumption holds true even in complex biological processes.

Conclusion

By bridging manifold learning and functional data analysis, this research provides a robust framework for high-dimensional regression. While the current focus is on "dense" functional data, the identified phase transitions offer a roadmap for future research into sparse longitudinal designs, making functional nonparametric regression a more viable tool for practical engineering and medical applications.

Summary updated

Abstract

We propose a new method for functional nonparametric regression with a predictor that resides on a finite-dimensional manifold but is only observable in an infinite-dimensional space. Contamination of the predictor due to discrete/noisy measurements is also accounted for. By using functional local linear manifold smoothing, the proposed estimator enjoys a polynomial rate of convergence that adapts to the intrinsic manifold dimension and the contamination level. This is in contrast to the logarithmic convergence rate in the literature of functional nonparametric regression. We also observe a phase transition phenomenon regarding the interplay of the manifold dimension and the contamination level. We demonstrate that the proposed method has favorable numerical performance relative to commonly used methods via simulated and real data examples.
BibTeX
@misc{lin2017functional,
  author = {Zhenhua Lin and Fang Yao},
  title = {{Functional Regression on Manifold with Contamination}},
  year = {2017},
  eprint = {1704.03005},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1704.03005},
}
Figure