Generalized Gaussian Process Regression Model for Non-Gaussian Functional Data

Bo Wang, Jian Qing ShiJournal of the American Statistical Association 2014 · DOI

TL;DR

This paper introduces a flexible new model, the Generalized Gaussian Process Functional Regression (GGPFR) model, designed to handle complex functional data where the outcome variable isn't normally distributed. It offers a powerful way to simultaneously model the average trends and individual variations in data collected over time or curves, even with multiple types of predictor variables.

What Problem Does This Paper Address?

Traditional functional data analysis often assumes that the outcome variable follows a normal distribution. However, many real-world datasets, such as those involving counts, proportions, or binary outcomes (like disease presence/absence), violate this assumption. This paper tackles the challenge of modeling functional data when the response variable comes from a non-Gaussian distribution (specifically, from an exponential family like binomial or Poisson). Furthermore, it addresses the complexity of having both functional (e.g., time series) and scalar (e.g., age) predictor variables, and the need to model not just the average behavior but also the unique patterns of individual subjects or "batches" of data.

Methods

The core of the proposed method is the Generalized Gaussian Process Functional Regression (GGPFR) model. It builds upon existing Gaussian Process Regression (GPR) techniques, which are known for their flexibility in modeling complex relationships. The key innovations include:

  • Non-Gaussian Response: The model explicitly accommodates response variables from exponential families, moving beyond the Gaussian assumption.
  • Concurrent Regression: It models the relationship between functional predictors and the functional response at the same time points.
  • Simultaneous Mean and Covariance Modeling: The GGPFR model jointly estimates the common mean structure (representing overall trends) and the covariance structure (capturing individual variations and dependencies within batches).
  • Prior Specification: The use of covariance kernels allows for flexible modeling of nonlinear relationships.
  • Inference: The paper details inference procedures, including hyper-parameter estimation using empirical Bayesian methods and Laplace approximation for computational tractability.

Datasets

The paper uses two main types of data for evaluation:

  • Simulated Data: Various simulated datasets are generated to test the model's performance under controlled conditions, including scenarios with different sample sizes and covariance structures.
  • Paraplegia Data: Real-world data from paraplegic patients undergoing standing-up maneuvers is used to demonstrate the model's applicability to a practical problem involving ordinal response variables.

Key Insights and Ideas

  • Bridging the Gap: The GGPFR model effectively bridges the gap between standard Gaussian functional regression and the need to handle more complex, non-Gaussian outcomes.
  • Individualized Insights: By modeling the covariance structure, the GGPFR model can capture the unique characteristics of individual subjects or data batches, offering more than just an average trend. This is crucial for understanding individual variability.
  • Flexibility in Covariates: The model's ability to handle both functional and scalar covariates simultaneously makes it applicable to a wider range of real-world problems.
  • Computational Feasibility: Despite the complexity, the use of techniques like Laplace approximation makes the model computationally feasible for practical applications.

Key Results

The paper demonstrates the effectiveness of the GGPFR model through numerical examples:

  • Accurate Mean Estimation: In simulation studies, the estimated mean curves closely matched the true mean curves, even with relatively small sample sizes.
  • Accurate Individual Process Estimation: The model successfully estimated individual underlying processes, showing good agreement with the true processes and improving with increased data.
  • Robustness to Covariance Choice: The model showed reasonable performance even when the chosen covariance kernel was not perfectly matched to the true underlying structure, highlighting its flexibility.
  • Superior Performance on Real Data: When applied to the paraplegia data, the GGPFR model outperformed a generalized varying coefficient model in interpolation tasks, indicating its advantage in capturing complex functional relationships.
  • Mixed-Effects Extension: An extension to mixed-effects GGPFR (ME-GGPFR) was shown to further improve performance, particularly for interpolation, by explicitly accounting for cluster effects.

Limitations and Future Work

  • Computational Cost: While Laplace approximation aids computation, for very high-dimensional data or complex covariance structures, computational efficiency might still be a concern. The authors suggest developing more efficient computational methods.
  • Covariance Kernel Selection: Choosing the "best" covariance kernel remains an important and interesting topic, as it can influence results.
  • Mean Model Extension: The paper focuses on a specific form of the mean model. Extending the GGPFR framework to more complex mean models (e.g., varying coefficient models) is suggested.
  • Heterogeneity and Measurement Error: Addressing heterogeneity among subjects and dealing with functional data contaminated by measurement errors are identified as areas for further research.
  • Mean Function Consistency: The consistency of the mean function estimation for GGPFR is noted as an open problem requiring further investigation.

Summary updated

Abstract

In this article, we propose a generalized Gaussian process concurrent regression model for functional data, where the functional response variable has a binomial, Poisson, or other non-Gaussian distribution from an exponential family, while the covariates are mixed functional and scalar variables. The proposed model offers a nonparametric generalized concurrent regression method for functional data with multidimensional covariates, and provides a natural framework on modeling common mean structure and covariance structure simultaneously for repeatedly observed functional data. The mean structure provides overall information about the observations, while the covariance structure can be used to catch up the characteristic of each individual batch. The prior specification of covariance kernel enables us to accommodate a wide class of nonlinear models. The definition of the model, the inference, and the implementation as well as its asymptotic properties are discussed. Several numerical examples with different non-Gaussian response variables are presented. Some technical details and more numerical examples as well as an extension of the model are provided as supplementary materials.
BibTeX
@article{wang2014generalized,
  author = {Bo Wang and Jian Qing Shi},
  title = {{Generalized Gaussian Process Regression Model for Non-Gaussian Functional Data}},
  journal = {Journal of the American Statistical Association},
  year = {2014},
  doi = {10.1080/01621459.2014.889021},
}
Figure