TL;DR
This paper introduces Bayesian Treed Calibration (BTC), a novel method for calibrating complex computer models when experimental data is limited and the model's behavior is non-stationary. BTC effectively handles these challenges by partitioning the input space, leading to more accurate predictions and a better understanding of physical systems like carbon capture processes.
What Problem Does This Paper Address?
Accurate computer models are essential for understanding complex physical and engineering systems, especially when direct experimentation is costly or impossible. However, these models often require calibration with limited experimental data. A key challenge arises when the computer model's output or its discrepancy from reality isn't uniform across all possible inputs (i.e., it's non-stationary). Standard calibration methods, often relying on stationary Gaussian processes, struggle with such non-stationarity, leading to inaccurate predictions. This paper tackles this by developing a method that can adapt to these changing behaviors across the input space.
Methods
The core innovation is Bayesian Treed Calibration (BTC). It extends standard Gaussian process calibration by employing a Bayesian tree structure. This tree recursively partitions the input space (both observable and calibration parameters) into smaller, more manageable subregions. Within each subregion, a standard Gaussian process calibration model is applied. This approach allows BTC to model non-stationary behavior in the computer model's output, its discrepancy from reality, or even experimental error. Parameter estimation is performed using Markov Chain Monte Carlo (MCMC) techniques, with specific strategies implemented to improve computational efficiency and mixing.
Datasets
The paper uses two artificial case studies and a real-world application:
- Artificial Case Study 1: A three-dimensional input space with a known discontinuity in the computer model's output.
- Artificial Case Study 2: A six-dimensional input space with a complex, non-stationary function.
- Real-world Application: Data from experiments on carbon dioxide capture using an AX sorbent in a fluidized bed adsorber. This dataset involves gas velocity as the observable input and several physical parameters as calibration inputs.
Key Insights and Ideas
- Partitioning for Non-Stationarity: The fundamental idea is that by breaking down the input space into smaller regions, we can apply simpler, stationary models within each region, effectively capturing overall non-stationary behavior.
- Flexibility in Modeling: BTC's tree structure allows for flexible modeling of the computer model, discrepancy, and even experimental error, enabling them to vary across different parts of the input space.
- Improved Prediction Accuracy: By accounting for non-stationarity, BTC can provide more accurate predictions, especially in regions where standard methods struggle.
- Computational Efficiency: While MCMC is used, the partitioning approach can lead to smaller covariance matrices being inverted at each step, potentially reducing computational cost for large datasets.
Key Results
- In artificial case studies, BTC demonstrated superior performance compared to standard Bayesian Gaussian Process Calibration (SBGPC), particularly in scenarios with discontinuities or non-stationary behavior. For instance, in the first case study, BTC achieved a Mean Squared Error (MSE) of 0.0034, significantly lower than SBGPC's 0.0252.
- For the carbon capture application, BTC provided more accurate predictions with less variance than SBGPC. The Mean Squared Prediction Error (MSPE) for BTC was 0.021, compared to 0.086 for SBGPC, a four-fold improvement.
- BTC was able to capture the discontinuity in the computer model's response surface, which SBGPC failed to do.
- The method showed better representation of the discrepancy term's behavior, which varied with Gas Velocity in the carbon capture experiment.
Limitations and Future Work
- Parameter Sensitivity: The authors note that without the right combination of proposal distributions for calibration and GP parameters, the posterior distribution of calibration parameters might get trapped in local maxima. This is identified as an area for future work.
- Multivariate Extension: The paper suggests extending the model to a multivariate setting, where covariance functions for computer code and discrepancy can be modeled similarly to Konomi et al. (2014a).
- Robustness: While BTC generally shows more robust predictions, the authors mention that further investigation into the sensitivity of predictions is warranted.
Summary updated