Bayesian Functional Generalized Additive Models with Sparsely Observed Covariates

Mathew W. McLean, Fabian Scheipl, Giles Hooker, Sonja Greven, David RuppertarXiv 2013 · DOI · arXiv

Abstract

The functional generalized additive model (FGAM) provides a flexible framework for regressing a scalar response on functional covariates. However, traditional estimation methods often require functional predictors to be observed noiselessly on a regular grid—a requirement rarely met in real-world longitudinal studies. This paper introduces a Bayesian framework for FGAM designed to handle sparsely observed, noisy functional data. By employing a simultaneous estimation approach for both the latent curves and the regression parameters, we account for the uncertainty inherent in sparse data. We present both Markov Chain Monte Carlo (MCMC) and Variational Bayes (VB) algorithms for model fitting, demonstrating significant improvements in predictive accuracy over traditional two-step frequentist methods, particularly in the context of forecasting eBay auction prices.

Introduction

In Functional Data Analysis (FDA), researchers treat observations as continuous trajectories or curves. A common goal is to predict a scalar outcome (e.g., a final price or a health metric) based on these functional predictors. While the Functional Linear Model (FLM) has been the standard tool for this task, it is often too restrictive, assuming a strictly linear relationship between the covariate and the response. The Functional Generalized Additive Model (FGAM) was introduced to offer more flexibility, allowing for non-linear effects that can vary across the domain of the function.

Despite its flexibility, the original FGAM formulation assumes "perfect" data: curves observed without noise on a dense grid. In practice, data is often "sparse longitudinal data"—a collection of noisy, irregularly timed snapshots. This paper addresses the gap between theoretical FGAM and the realities of messy data.

The Challenge of Sparsity and the "Two-Step" Fallacy

A common strategy for dealing with sparse functional data is the "two-step" approach: first, smooth the scattered data points into a continuous curve (often using techniques like Principal Analysis by Conditional Expectation, or PACE); second, use the recovered curves as inputs for a standard regression model.

This paper argues that this decoupling is fundamentally flawed. The two-step approach ignores the uncertainty of the first step during the second, leading to biased parameter estimates and overconfident predictions. Furthermore, the smoothing step occurs in a vacuum, ignoring the information contained in the response variable that might help clarify the shape of the underlying predictor curve.

A Simultaneous Bayesian Framework

To overcome these limitations, we propose a semiparametric Bayesian approach that treats the latent smooth trajectories as unknown parameters to be estimated alongside the regression surface.

Model Structure

We model the regression relationship using tensor product B-splines. This allows the effect of the functional covariate to change both over the time domain and based on the specific value of the covariate itself. By treating the "hidden" smooth trajectories as latent variables, the model updates its belief about the curve's shape while simultaneously learning the regression relationship. Crucially, this means the response variable $Y$ informs the estimation of the predictor $X$: if a specific curve shape better explains the observed outcome, the model is more likely to infer that shape from the noisy data points.

Inference Algorithms

Because the resulting posterior distributions are non-conjugate and complex, we developed two distinct computational strategies:

  1. MCMC (Markov Chain Monte Carlo): We utilize a Metropolis-within-Gibbs sampler. To handle smoothing parameters, we employ slice sampling, and we use a Metropolis-Hastings step for the principal component scores.
  2. Variational Bayes (VB): Recognizing that MCMC can be computationally prohibitive for large datasets, we developed a VB approximation. By using a Laplace approximation to handle non-conjugate components, we achieve a massive speedup—often 10 to 20 times faster than MCMC—with minimal loss in accuracy.

Empirical Results

Simulation Studies

In synthetic benchmarks with varying levels of sparsity (ranging from 10 to 40 observations per subject), our Bayesian FGAM consistently outperformed the frequentist two-step PACE approach. In many scenarios, our methods achieved predictive accuracy nearly identical to an "oracle" model that had access to the true, noiseless curves.

eBay Auction Data

We applied our methods to forecast the closing prices of Palm M515 PDAs on eBay. Using the "price path" (log-price ratio of bids) over the first six days of a seven-day auction, we predicted the final sale price. This data is notoriously irregular and sparse.

The results were stark: our Bayesian FGAM achieved a median Root Mean Square Error (RMSE) of approximately 0.09. In contrast, a linear model (FLM-PACE) yielded an RMSE of 0.59, and a frequentist two-step FGAM failed significantly with an RMSE of 4.91. This demonstrates that when data is sparse, accounting for uncertainty through a unified Bayesian framework is not just a theoretical preference but a practical necessity.

Conclusion

This work provides a robust roadmap for applying flexible functional models to real-world, sparse data. By moving away from two-step estimation and embracing simultaneous Bayesian inference, we can recover more accurate relationships and better predictive performance. While Variational Bayes offers a powerful tool for scaling these models, future work will focus on extending this framework to non-Gaussian responses, such as binary or count data, and further optimizing the computational overhead for high-dimensional functional predictors.

Summary updated

Abstract

The functional generalized additive model (FGAM) was recently proposed in McLean et al. (2013) as a more flexible alternative to the common functional linear model (FLM) for regressing a scalar on functional covariates. In this paper, we develop a Bayesian version of FGAM for the case of Gaussian errors with identity link function. Our approach allows the functional covariates to be sparsely observed and measured with error, whereas the estimation procedure of McLean et al. (2013) required that they be noiselessly observed on a regular grid. We consider both Monte Carlo and variational Bayes methods for fitting the FGAM with sparsely observed covariates. Due to the complicated form of the model posterior distribution and full conditional distributions, standard Monte Carlo and variational Bayes algorithms cannot be used. The strategies we use to handle the updating of parameters without closed-form full conditionals should be of independent interest to applied Bayesian statisticians working with nonconjugate models. Our numerical studies demonstrate the benefits of our algorithms over a two-step approach of first recovering the complete trajectories using standard techniques and then fitting a functional regression model. In a real data analysis, our methods are applied to forecasting closing price for items up for auction on the online auction website eBay.
BibTeX
@misc{mclean2013bayesian,
  author = {Mathew W. McLean and Fabian Scheipl and Giles Hooker and Sonja Greven and David Ruppert},
  title = {{Bayesian Functional Generalized Additive Models with Sparsely Observed Covariates}},
  year = {2013},
  eprint = {1305.3585},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1305.3585},
}
Figure