Multi-fidelity Gaussian process surrogate modeling for regression problems in physics

Kislaya Ravi, Vladyslav Fediukov, Felix Dietrich, Tobias Neckel, Fabian Buse, Michael Bergmann, Hans-Joachim Bungartz

Introduction

In fields ranging from aerospace engineering to plasma physics, high-fidelity (HF) simulations are essential for predicting complex system behaviors. However, the computational cost of a single HF run can be immense, often taking days to complete. This makes iterative tasks like design optimization or uncertainty quantification practically impossible. Surrogate models—machine learning "digital twins"—address this by mimicking simulations at a fraction of the cost. The primary challenge is the "data bottleneck": training these surrogates still requires a substantial amount of expensive HF data.

Multi-fidelity modeling offers a solution by combining a small set of HF data with a larger volume of "cheap" low-fidelity (LF) data (e.g., simulations with coarser grids or simplified physics). This paper presents advanced Gaussian Process (GP) techniques to bridge these data levels, specifically focusing on non-linear relationships and structured kernels.

Gaussian Process Framework

The authors utilize Gaussian Processes (GP) as the core probabilistic framework. Unlike standard regression, GPs provide a full distribution over possible functions, offering both a point prediction and a quantified measure of uncertainty.

Gaussian Process prior and posterior distributions. The prior (top) represents potential functions before data observation, while the posterior (bottom) incorporates training points to reduce uncertainty.

To handle the complexities of multi-fidelity data, the paper introduces several methodological enhancements:

  • Stacked NARGP Models: Extending Non-Linear Autoregressive (NARGP) models to handle more than two fidelity levels by cascading outputs.
  • GPDFC Kernel: A composite kernel designed for "Delay Fusion," which uses time-delayed LF values to approximate derivatives, effectively capturing how HF signals deviate from LF signals via a Taylor-like expansion.
  • Deep Gaussian Processes (DGP): Employed to learn hierarchical features when data points are not "nested" (i.e., LF and HF data are at different input locations).

The Manifold Advantage

A key insight of this work is why multi-fidelity modeling is so effective: the relationship between LF and HF data is often much simpler than the underlying physical function itself.

The manifold relationship between low-fidelity (blue) and high-fidelity (orange) functions. The smooth green manifold is significantly easier for a GP to learn than the raw oscillatory high-fidelity signal.

As illustrated above, while a high-fidelity function might be highly oscillatory and difficult to learn directly, the "manifold" connecting it to the low-fidelity approximation is often smooth. By learning this mapping, the GP can achieve high accuracy with far fewer HF samples.

Results and Applications

The proposed methods were validated across diverse domains:

  1. Terramechanics: In modeling rover wheel traction, the non-linear multi-fidelity model (NARDGP) achieved superior accuracy using only 104 HF points, compared to a single-fidelity GP requiring over 2,000 points.
  2. Plasma Physics: Using the GPDFC method, the researchers significantly improved predictions of temperature and density gradients in fusion reactor simulations, outperforming standard GP baselines.
  3. Academic Benchmarks: Tests on synthetic functions confirmed that while linear models (AR1) suffice for simple scaling, the new non-linear architectures are essential for complex, phase-shifted physics.

Conclusion and Calibration

While multi-fidelity GPs offer massive computational savings, the authors note a tendency for these models to be "overconfident." They demonstrate that applying Isotonic Regression as a post-processing calibration step is critical for ensuring that uncertainty bands remain reliable for engineering decisions. Future work will focus on adaptive "batch" sampling to further accelerate the modeling process in high-dimensional spaces.

Summary updated

BibTeX
@inproceedings{ravi2024multi,
  author = {Kislaya Ravi and Vladyslav Fediukov and Felix Dietrich and Tobias Neckel and Fabian Buse and Michael Bergmann and Hans-Joachim Bungartz},
  title = {{Multi-fidelity Gaussian process surrogate modeling for regression problems in physics}},
  booktitle = {Mach. Learn. Sci. Technol},
  year = {2024},
  url = {https://openreview.net/forum?id=ihJi5VjFSQ},
}

Loading related content...

Figure