Theory in functional principal components of discretely observed data

Hang Zhou, Dongyi Wei, Fang YaoOpenReview 2023 · arXiv · OpenReview

TL;DR

This paper develops a unified theoretical framework for Functional Principal Component Analysis (FPCA) when data is observed discretely and with noise. The authors provide optimal convergence rates for estimating eigenfunctions, even when considering an increasing number of them, which is crucial for many functional data models. They introduce a novel "double truncation" technique to achieve uniform convergence, a significant advancement for both sparse and dense data scenarios.

What Problem Does This Paper Address?

Functional data analysis (FDA) treats data as random functions, and FPCA is a cornerstone for dimensionality reduction and representation. A key challenge in FDA is analyzing data that is not observed continuously but rather at discrete time points with measurement errors. While significant progress has been made in estimating the mean and covariance functions, understanding the behavior of eigenfunctions, especially when estimating an increasing number of them (diverging indices), has remained a difficult problem. Existing methods often struggle to provide optimal convergence rates for these diverging eigenfunctions, which is essential for the theoretical underpinnings of many FDA models, such as functional linear regression. This paper aims to bridge this gap by providing a comprehensive theory for estimating eigenfunctions from discretely observed functional data, particularly focusing on the challenging case of diverging indices.

Methods

The paper introduces a novel "double truncation" technique to handle the complexities of discretely observed functional data. This technique involves two stages of truncation: one on the observed data and another on the summation of random quantities. This approach allows for a more effective management of bias and variance, leading to improved convergence rates. The authors also leverage perturbation series expansions, a standard tool in functional data analysis, but adapt them to the discrete setting. They establish theoretical bounds for the L² convergence of eigenfunctions and the asymptotic normality of eigenvalues. The paper also introduces a new technical route for analyzing the perturbation series of the functional covariance operator, bridging the gap between idealized continuous data and realistic discrete observations.

Datasets

Not discussed in the paper. The paper focuses on theoretical development and uses synthetic data for numerical experiments.

Key Insights and Ideas

  • Optimal Rates for Diverging Eigenfunctions: The paper demonstrates that it's possible to achieve optimal convergence rates for estimating eigenfunctions with diverging indices, even with discretely observed data. This is a significant theoretical breakthrough.
  • The Power of Double Truncation: The novel "double truncation" technique is key to overcoming the limitations of previous methods, enabling uniform convergence for both covariance and eigenfunction estimates across various sampling schemes (sparse to dense).
  • Phase Transitions: The research highlights "phase transitions" in the convergence rates, showing how the data sampling scheme (sparse vs. dense) influences the estimation performance. This provides valuable insights for experimental design.
  • Bridging Theory and Practice: The theoretical framework developed here has direct implications for improving downstream FPCA-based analyses, such as functional linear regression, by providing better theoretical guarantees.

Key Results

The paper establishes theoretical results for the L² convergence of estimated eigenfunctions with diverging indices. Under mild assumptions and for various sampling schemes (random and fixed regular designs), they derive bounds that are optimal in the minimax sense, comparable to what is achievable with fully observed data. They also provide asymptotic normality results for eigenvalues. The paper's Corollary 1 and 2 offer a systematic partition of sampling schemes into "sparse" and "dense" categories, detailing how convergence rates change. For instance, they show that when the sampling rate $N_2$ is sufficiently large (dense case), the optimal rate $j^2/n$ can be achieved.

Limitations and Future Work

The paper acknowledges that while optimal rates are achieved under certain conditions, the exact conditions for these optimal rates depend on the smoothness of the underlying functions and the sampling scheme. The authors suggest that their techniques can be applied to more complex regression models, such as functional generalized linear models or functional Cox models, which would be a natural direction for future research. The paper also notes that the theoretical framework is primarily focused on the L² norm and uniform convergence, and further investigation into other convergence types or specific applications might be warranted. The numerical experiments, while illustrative, are based on synthetic data, and applying these methods to real-world, complex functional datasets would be a valuable next step.

Summary updated

Abstract

Convergence of eigenfunctions with diverging index is essential in nearly all methods based on functional principal components analysis. The main goal of this work is to establish the unified theory for such eigencomponents in different types of convergence based on discretely observed functional data. We obtain the moment bounds for eigenfunctions and eigenvalues for a wide range of the sampling rate and show that under some mild assumptions, the 2 bound of eigenfunctions estimator with diverging indices is optimal in the minimax sense as if the curves are fully observed. This is the first attempt at obtaining an optimal rate for eigenfunctions with diverging index for discretely observed functional data. We propose a double truncation technique in handling the uniform convergence of function data and establish the uniform convergence of covariance function as well as the eigenfunctions for all sampling scheme under mild assumptions. The technique route proposed in this work provides a new tool in handling the perturbation series with discretely observed functional data and can be applied in most problems based on functional principal components analysis and models involving inverse issue.
BibTeX
@misc{zhou2023theory,
  author = {Hang Zhou and Dongyi Wei and Fang Yao},
  title = {{Theory in functional principal components of discretely observed data}},
  year = {2023},
  eprint = {2209.08768},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2209.08768},
  howpublished = {OpenReview},
}
Figure