Variational Bayes for Functional Data Registration, Smoothing, and Prediction

Cecilia Earls, Giles HookerBayesian 2016 · DOI

TL;DR

This paper introduces a novel Bayesian hierarchical model for functional data registration that integrates prediction and smoothing within a single framework. To tackle the computational challenges of high-dimensional Bayesian models, they propose an Adapted Variational Bayes (AVB) algorithm, which offers a computationally efficient alternative to MCMC sampling while achieving comparable results. This work is significant because it extends the capabilities of functional data analysis by enabling prediction of future functional data and providing a more robust inferential framework.

What Problem Does This Paper Address?

Functional data analysis often involves dealing with curves or functions that are not perfectly aligned in time or phase. This misalignment, known as "functional data registration," is crucial for accurate analysis, as it can obscure underlying patterns and lead to misinterpretations. Traditional methods often treat registration as a separate pre-processing step, which can lead to an underestimation of variability and a loss of information. Furthermore, existing methods often lack the ability to predict future functional data or to simultaneously handle registration, smoothing, and prediction within a unified probabilistic framework. This paper aims to bridge this gap by proposing a flexible Bayesian model that addresses these limitations.

Methods

The core of the proposed method is a Bayesian hierarchical model where registered functions are modeled as Gaussian processes. The key innovation lies in the development of an Adapted Variational Bayes (AVB) algorithm. Traditional Variational Bayes requires that all posterior distributions be of a known parametric form, which is not always the case in complex hierarchical models, especially with Gaussian process priors. AVB extends this by allowing for some parameters (like the base functions in their model) to have distributions that are known only up to a constant of proportionality, while others follow known parametric forms. This allows for approximate inference in models where standard Variational Bayes would fail. The AVB algorithm iteratively updates these distributions, offering a computationally efficient alternative to MCMC sampling, especially for high-dimensional problems. The model also incorporates a mechanism for functional prediction, allowing for the estimation of future values of registered, warping, and unregistered functions, even from partially observed data.

Datasets

The authors evaluate their methods on:

  • Simulated Data Set: Consisting of 20 unregistered scaled mixtures of three Gaussian probability density functions.
  • Berkeley Boys Growth Velocity Data: A dataset of 39 velocity growth functions.
  • El-Niño Data: Weekly sea surface temperature readings from June 1950 to May 2004, restructured to define "years" based on consecutive lowest temperatures.

Key Insights and Ideas

  • Unified Framework: The ability to perform registration, smoothing, and prediction within a single Bayesian hierarchical model is a significant advancement, avoiding the pitfalls of sequential pre-processing.
  • Functional Prediction from Partial Data: The model's probabilistic nature allows for the prediction of future functional behavior even when only a portion of a curve is observed, a capability not readily available in many existing registration methods.
  • Computational Efficiency with AVB: The development of the Adapted Variational Bayes algorithm provides a practical solution for inference in complex, high-dimensional functional data models, making sophisticated analyses more accessible.
  • Variability Estimation: By integrating registration and smoothing within a probabilistic framework, the model can more accurately capture and represent the variability in the data, avoiding the underestimation that can occur with pre-processing steps.

Key Results

  • The proposed model achieves registration results comparable to state-of-the-art methods like the Fisher-Rao (F-R) and Sobolev Least Squares (SLS) criteria.
  • For simulated data, the GP model achieved the lowest SLS score, indicating better function alignment.
  • The AVB algorithm demonstrated significant computational savings compared to MCMC sampling, particularly for "real" data like the Berkeley Boys Growth Velocity Data, while yielding comparable estimates. For instance, in one comparison, AVB was orders of magnitude faster than MCMC.
  • The AVB algorithm's estimates for registered functions were shown to be very close to MCMC estimates, with only minor discrepancies even in more complex scenarios.
  • For noisy data, the AVB algorithm's credible intervals for the target function were shown to underrepresent variability compared to MCMC, highlighting the importance of accounting for noise within the model.

Limitations and Future Work

  • Convergence to Global Maximum: While the AVB algorithm is guaranteed to converge, it may converge to a local maximum. The authors suggest initial warping or adjusting registration/warping penalties over time to mitigate this.
  • MCMC for Posterior Characterization: For some models, especially those involving noise, MCMC sampling may still be preferred for a complete characterization of posterior distributions, with AVB serving as a useful initializer.
  • Optimal Sampler Selection: Determining the most efficient sampling method for joint posterior distributions remains an area for future work.
  • Prior Choice: The choice of priors for variance components can be sensitive, especially with small variance components or limited data.
  • Future Work: The authors suggest extending the models to other inferential areas for functional data, such as functional linear regression, and continuing to improve the proposed models. They also note that using a multivariate t-distribution for approximating posterior distributions might offer better confidence interval coverage properties with small sample sizes.

Summary updated

Abstract

We propose a model for functional data registration that extends current inferential capabilities for unregistered data by providing a flexible probabilistic framework that 1) allows for functional prediction in the context of registration and 2) can be adapted to include smoothing and registration in one model. The proposed inferential framework is a Bayesian hierarchical model where the registered functions are modeled as Gaussian processes. To address the computational demands of inference in high-dimensional Bayesian models, we propose an adapted form of the variational Bayes algorithm for approximate inference that performs similarly to Markov Chain Monte Carlo (MCMC) sampling methods for well-defined problems. The efficiency of the adapted variational Bayes (AVB) algorithm allows variability in a predicted registered, warping, and unregistered function to be depicted separately via bootstrapping. Temperature data related to the El-Niño phenomenon is used to demonstrate the unique inferential capabilities for prediction provided by this model.
BibTeX
@inproceedings{earls2016variational,
  author = {Cecilia Earls and Giles Hooker},
  title = {{Variational Bayes for Functional Data Registration, Smoothing, and Prediction}},
  booktitle = {Bayesian},
  year = {2016},
  doi = {10.1214/16-ba1013},
}
Figure