Abstract
Gaussian Process (GP) regression is a powerful tool for surrogate modeling, yet it often struggles with non-constant (heteroskedastic) noise and cubic scaling costs. hetGPy is a native Python library designed to overcome these hurdles by specifically modeling the noise surface and exploiting replicated observations. Through the use of the Woodbury matrix identity, hetGPy provides significant computational efficiency, often outperforming deep-learning-based GP libraries by an order of magnitude on standard CPU hardware.
Introduction
Gaussian Process Regression (GPR) serves as the "gold standard" for creating emulators of complex computer experiments. However, standard implementations face two primary limitations: the "scalability wall," where computational costs grow cubically ($O(N^3)$) with the number of data points, and the "homoskedasticity assumption," which assumes a constant noise level across the input space. In fields like climate modeling or epidemiology, stochastic simulations frequently exhibit noise that varies depending on input parameters. While researchers often use replicates to mitigate noise, standard tools treat these as independent points, leading to computational waste.
Methodology
hetGPy is a Python implementation of the R package hetGP, introducing two key innovations:
- Coupled Gaussian Processes: The library models both the response mean and the noise process as separate but linked GPs. This allows the model to learn a "noise surface," predicting volatility across the input domain.
- The Woodbury Trick: To address scalability,
hetGPyutilizes the Woodbury matrix identity. This allows matrix operations to scale with the number of unique locations ($n$) rather than the total number of observations ($N$). When $n \ll N$, the speedup is dramatic.
The library is built on a minimalist philosophy, utilizing NumPy and SciPy for ease of integration into scientific Python workflows without the overhead of deep learning frameworks.
Performance Evaluation
The authors evaluated hetGPy using classic benchmarks, including the Motorcycle Accident dataset and various synthetic test functions (Branin, Goldstein-Price, Hartmann, and Sphere).
In the 1D motorcycle example, hetGPy demonstrates its ability to capture the changing variance over time, a task where homoskedastic models typically fail by either over- or under-estimating uncertainty in different regions.
Key Results and Insights
The performance gains are most evident when datasets contain replicates. On the Branin test function with 1,000 unique locations and 5,000 total observations, hetGPy completed training in 2.96 seconds, compared to 98.45 seconds for a naive GPyTorch implementation—a 33x speedup.
These results highlight that algorithmic efficiency on a CPU can outperform raw GPU power for specific data structures. Furthermore, hetGPy provides a native Python alternative for researchers who previously relied on R-to-Python bridges for stochastic simulation modeling.
Limitations and Future Work
While highly efficient, hetGPy currently supports a narrower range of kernel functions compared to JAX or PyTorch-based libraries. Future development aims to integrate auto-differentiation frameworks to enable more complex model extensions and broader kernel flexibility. Additionally, the library's primary advantages are tied to the presence of replicated data; its benefits diminish for datasets consisting entirely of unique locations.
Conclusion
hetGPy offers a specialized, high-performance solution for heteroskedastic GP modeling in Python. By treating replication as an opportunity for efficiency rather than a burden, it provides a vital tool for the stochastic simulation community.
Summary updated