Landmark-embedded Gaussian process with applications for functional data modeling

Jaesung Lee, Chao Wang, Xiaoyu Sui, Shiyu Zhou, Junhong ChenIISE 2021 · DOI

TL;DR

This paper introduces a novel "landmark-embedded Gaussian process" (LEGP) model to accurately infer target variables from complex functional data, like sensor readings. By simultaneously considering both the shape and location of these signals, LEGP significantly outperforms existing methods in both point and interval predictions, particularly in noisy and variable environments.

What Problem Does This Paper Address?

In many real-world applications, we need to infer a specific value (a "target variable") from data that changes over time or some other continuous dimension, forming a "functional signal." A prime example is inferring the concentration of a substance from a sensor's electrical output, which is a functional signal. The challenge is that these functional signals are often highly complex and unpredictable. They don't just vary in their overall shape (e.g., how high or low the signal goes) but also in their "location" – where key features like peaks or valleys appear. Furthermore, even for the same target variable value, the signals can exhibit significant random variations due to environmental factors or inherent sensor variability. Existing methods often struggle to capture all these variations simultaneously, leading to inaccurate predictions and unreliable estimates of uncertainty.

Methods

The core innovation is the Landmark-Embedded Gaussian Process (LEGP) model. Unlike traditional Gaussian processes that primarily focus on signal shape, LEGP explicitly incorporates "landmarks" – specific, identifiable features within the functional signal (like the bottom of a V-shape in the sensor data).

Here's how it works:

  • Unified Framework: LEGP models four types of variations simultaneously: same-level shape, same-level location, across-level shape, and across-level location.
  • Landmark Embedding: Landmarks are treated as parameters within the Gaussian process model itself. This allows the model to learn not only the signal's shape but also how its key features (landmarks) shift in relation to the target variable.
  • Two Latent Functions: The model uses two latent Gaussian processes: one to capture same-level variations (within a specific target variable value) and another to capture across-level variations (how shape and location change with the target variable).
  • Gibbs-Metropolis-Hasting Algorithm: A sophisticated MCMC algorithm is employed for both estimating model parameters (including the landmarks) and inferring the target variable for new, unseen signals.

This approach contrasts with simpler methods that might try to align signals first (using landmarks) and then fit a model, or those that ignore location variations altogether.

Datasets

The paper uses two main sources of data:

  • Simulated Data: Extensive numerical studies were conducted using synthetically generated functional data. These simulations were designed to mimic various scenarios of signal complexity, including different levels of shape and location variations, noise, and sample sizes. This allowed for controlled evaluation of the LEGP model against benchmarks.
  • GFET Sensor Data: A real-world dataset from Graphene-based Field-Effect Transistor (GFET) sensors measuring lead ion concentration in water was used for a case study. This data exhibits the complex functional signal characteristics the model is designed to handle.

The paper does not introduce a new benchmark dataset but rather uses existing simulation strategies and a real-world sensor dataset to validate its proposed method.

Key Insights and Ideas

  • Location Matters: The paper strongly emphasizes that ignoring the location of features within functional signals leads to significant inaccuracies. The LEGP model's ability to jointly model shape and location is a key differentiator.
  • Simultaneous Estimation is Crucial: Treating landmarks as model parameters and estimating them simultaneously with other model parameters (rather than in a separate pre-processing step) leads to more robust and accurate results. This "integrative modeling" avoids discrepancies and leverages the full information from the data.
  • Uncertainty Quantification: The LEGP model provides more accurate and reliable interval estimates (confidence intervals) for the target variable, which is critical for decision-making in applications like sensor calibration.
  • Flexibility of Landmarks: While the paper uses the "bottom of the V" as a landmark for GFET sensor data, the authors highlight that the landmark concept is generalizable to other identifiable features in functional data.

Key Results

  • Superior Accuracy: In both numerical studies and the GFET sensor case study, the LEGP model consistently outperformed benchmark methods (including conventional Gaussian processes, two-step landmark registration approaches, and functional linear regression) in terms of both point estimates (lower Mean Squared Prediction Error - MSPE) and interval estimates (lower Absolute Prediction Error - APE for interval coverage).
  • Improved Stability: The LEGP model demonstrated excellent performance stability, with a significantly lower percentage of failed calibrations (%Fail) compared to other methods, especially in challenging scenarios.
  • Effective with Less Data: LEGP showed strong performance even with smaller sample sizes (e.g., Case IV in numerical studies), where other methods struggled.
  • Accurate Interval Estimates: The APE for interval estimates from LEGP was consistently low (0-0.1), indicating highly reliable confidence intervals, whereas other methods showed much larger errors.

Limitations and Future Work

  • Landmark Pre-specification: A current limitation is that the LEGP model requires the user to pre-specify the landmarks. While the paper suggests that landmarks can often be identified based on physical understanding of the signal, there might be cases where selecting a meaningful landmark is difficult.
  • Algorithmic Landmark Specification: The authors suggest future work could focus on developing an algorithmic approach to automatically define good landmarks, potentially using feature selection methods. This would make the model more broadly applicable.
  • Censored Functional Data: Another promising direction is extending the framework to handle censored functional data, where parts of the signal are unobserved. This might involve inferring missing shapes and landmarks simultaneously.
  • Scalability: While the MCMC sampling can be parallelized, further improvements in scalability for very large datasets could be achieved using techniques like inducing points or GPU acceleration.

Summary updated

Abstract

In practice, we often need to infer the value of a target variable from functional observation data. A challenge in this task is that the relationship between the functional data and the target variable is very complex: the target variable not only influences the shape but also the location of the functional data. In addition, due to the uncertainties in the environment, the relationship is probabilistic, that is, for a given fixed target variable value, we still see variations in the shape and location of the functional data. To address this challenge, we present a landmark-embedded Gaussian process model that describes the relationship between the functional data and the target variable. A unique feature of the model is that landmark information is embedded in the Gaussian process model so that both the shape and location information of the functional data are considered simultaneously in a unified manner. Gibbs–Metropolis–Hasting algorithm is used for model parameters estimation and target variable inference. The performance of the proposed framework is evaluated by extensive numerical studies and a case study of nano-sensor calibration.
BibTeX
@inproceedings{lee2021landmark,
  author = {Jaesung Lee and Chao Wang and Xiaoyu Sui and Shiyu Zhou and Junhong Chen},
  title = {{Landmark-embedded Gaussian process with applications for functional data modeling}},
  booktitle = {IISE},
  year = {2021},
  doi = {10.1080/24725854.2021.1974129},
}
Figure