Bayesian optimized deep ensemble for uncertainty quantification of deep neural networks: a system safety case study on sodium fast reactor thermal stratification modeling

Zaid Abulawi, Rui Hu, Prasanna Balaprakash, Yang Liu

Introduction

In safety-critical fields like nuclear engineering, the reliability of a predictive model is as vital as the prediction itself. Deep Neural Networks (DNNs) are increasingly utilized as surrogate models to replace slow, high-fidelity Computational Fluid Dynamics (CFD) simulations. However, for these models to be integrated into safety cases, they must provide robust Uncertainty Quantification (UQ). This paper introduces BODE (Bayesian Optimized Deep Ensemble), a method designed to significantly improve both the predictive accuracy and the reliability of uncertainty estimates in deep learning.

The Problem: "Lazy" Ensembles

The current "gold standard" for UQ in deep learning is the Deep Ensemble (DE), which involves training the same model architecture multiple times with different random initializations. However, standard ensembles are often "lazy." Because every member uses the same hyperparameters and architecture, the ensemble lacks the diversity needed to provide high-quality UQ. This lack of diversity often leads to models "hallucinating" high aleatoric uncertainty (inherent data noise) even when the training data is perfectly clean, simply because the models cannot fit the data well.

Methodology: The BODE Framework

BODE addresses the limitations of standard ensembles by integrating Bayesian Optimization (BO) into the ensemble construction process.

1. Hyperparameter Diversity

Instead of retraining the same model configuration, BODE uses BO to find a unique, optimal set of hyperparameters (such as learning rate, batch size, and dropout rate) for each member of the ensemble. This ensures that the ensemble is composed of diverse, high-performing models that approach the problem from different mathematical perspectives.

2. Optimization Strategy

The authors utilized the Ax-platform and BoTorch for the optimization process. The search begins with a Sobol sequence—a quasi-random sampling method—to explore the hyperparameter search space broadly, followed by Bayesian Optimization to exploit the most promising configurations.

3. Loss Function and UQ Decomposition

The models are trained using Negative Log-Likelihood (NLL), assuming a Gaussian distribution. This allows the network to output both a mean prediction and a variance. The total uncertainty is then mathematically decomposed into two distinct components:

  • Aleatoric Uncertainty: Representing inherent noise in the data.
  • Epistemic Uncertainty: Representing the model's lack of knowledge, measured by the variance (disagreement) between the different ensemble members.

Case Study: Sodium Fast Reactor Thermal Stratification

The study validated BODE using a Sodium Fast Reactor (SFR) case study, specifically modeling thermal stratification during a loss-of-flow event. High-fidelity CFD data was generated via STAR-CCM+, focusing on predicting eddy viscosity, a critical parameter for turbulence modeling. To test the robustness of the UQ, the authors injected 5% and 10% synthetic Gaussian noise into the clean CFD results to see if the model could correctly distinguish between model error and data noise.

Key Results and Insights

BODE outperformed the standard baseline ensemble across every metric:

  • Uncertainty Reduction: In a noise-free environment, BODE estimated a total uncertainty four times lower than the manually tuned baseline.
  • Accuracy Boost: BODE achieved an average RMSE of 0.77 Pa·s ($R^2$ of 0.997), significantly outperforming the baseline ensemble, which had an RMSE of 3.83 Pa·s ($R^2$ of 0.958).
  • Correcting Aleatoric Bias: A major finding was that the baseline ensemble overestimated aleatoric uncertainty in noise-free data. BODE correctly pushed this value toward zero, proving that standard ensembles often mistake a poor model fit for data noise. When noise was intentionally added (5% and 10%), BODE’s aleatoric estimates aligned almost perfectly with the injected levels.

Conclusion and Future Work

BODE demonstrates that diversity in ensemble construction is the key to reliable uncertainty quantification. By forcing ensemble members to have different architectures and training dynamics, BODE captures a much more realistic "model disagreement."

While the computational cost of running Bayesian Optimization for each ensemble member is higher than standard training, the authors argue this is a necessary one-time cost for safety-critical applications. Future work will involve testing BODE on experimental data where noise levels are unknown and exploring its application in more complex architectures beyond Deep Convolutional Neural Networks (DCNNs).

Summary updated

BibTeX
@article{abulawi2025bayesian,
  author = {Zaid Abulawi and Rui Hu and Prasanna Balaprakash and Yang Liu},
  title = {{Bayesian optimized deep ensemble for uncertainty quantification of deep neural networks: a system safety case study on sodium fast reactor thermal stratification modeling}},
  journal = {Reliab. Eng. Syst. Saf.},
  year = {2025},
  doi = {10.1016/j.ress.2025.111353},
}
Figure