Ensemble‐Based Experimental Design for Targeting Data Acquisition to Inform Climate Models

Oliver R. A. Dunbar, Michael F. Howland, Tapio Schneider, Andrew M. StuartJournal of Advances in Modeling Earth Systems 2022 · DOI

Abstract

Data required to calibrate uncertain general circulation model (GCM) parameterizations are often only available in limited regions or time periods, such as observational data from field campaigns or local high-resolution simulations. This study introduces a new ensemble-based parallel algorithm designed to automatically target data acquisition to regions and times that maximize uncertainty reduction, or information gain, about GCM parameters. The algorithm is embedded in the Calibrate-Emulate-Sample (CES) framework, which performs efficient model calibration and uncertainty quantification. In proof-of-concept applications using an idealized GCM, the calculated information gain reflects the reduction in parametric uncertainty obtained from Bayesian inference. The results demonstrate that the largest information gain typically, though not exclusively, results from regions near the intertropical convergence zone (ITCZ).

The Challenge of Subgrid-Scale Parameterization

Global Climate Models (GCMs) operate on grids that are too coarse to explicitly simulate individual clouds or turbulent eddies. Instead, they rely on "parameterizations"—mathematical shortcuts that represent these complex subgrid-scale processes. These schemes contain uncertain parameters that significantly influence climate predictions. While high-resolution Large Eddy Simulations (LES) or targeted field campaigns can provide the data needed to calibrate these parameters, such resources are extremely expensive. Researchers often face a strategic dilemma: given limited resources, where and when should data be collected to be most informative? Historically guided by intuition, this paper provides a rigorous, automated method to maximize information gain from every data point.

The Calibrate-Emulate-Sample (CES) Framework

The authors utilize the Calibrate-Emulate-Sample (CES) framework to handle the heavy computational requirements of Bayesian inference. This framework consists of three main stages:

  1. Calibrate: Uses Ensemble Kalman Inversion (EKI) to rapidly identify high-performing parameter sets.
  2. Emulate: Trains a Gaussian Process (GP) to act as a fast surrogate for the computationally expensive GCM.
  3. Sample: Employs Markov Chain Monte Carlo (MCMC) on the GP surrogate to produce a full posterior distribution of the parameters.
Latitudinal distribution of climate statistics (Relative Humidity, Mean Precipitation, and 90th Percentile Precipitation) used as targets for model calibration.

Automated Experimental Design

Built atop the CES framework is a novel "Experimental Design" layer. It employs a D-optimality criterion to calculate a "utility" score for various locations and times. This score identifies the "design point" (e.g., a specific latitude or season) that leads to the most concentrated posterior distribution, thereby minimizing uncertainty. Because the GP emulator is highly efficient, the algorithm can evaluate the potential information gain of hundreds of different locations in parallel without requiring additional GCM runs.

Key Insights: The ITCZ and Beyond

The algorithm's findings often align with physical intuition but occasionally offer surprises. In seasonally varying simulations, the most informative data consistently comes from the Intertropical Convergence Zone (ITCZ), where trade winds meet and convection is most active.

Seasonal variation of precipitation across latitudes, illustrating the migration of the ITCZ and the resulting shifts in optimal data acquisition windows.

However, the study also revealed that the most intense physical activity does not always yield the most information. In certain stationary tests, subtropical precipitation minima proved more informative than equatorial peaks. This suggests that the sensitivity of a model to its parameters is not always co-located with the highest magnitude of the observed phenomena.

Efficiency and Scalability

A major advantage of this approach is its massive efficiency gain. Traditional Bayesian calibration often requires $O(10^5)$ model evaluations, whereas the CES framework achieves comparable results with only $O(10^2)$ evaluations—a 1,000x speedup. Furthermore, the design stage is highly scalable; once the initial ensemble of GCM runs is complete, the utility of every possible sensor location can be calculated independently and simultaneously.

Conclusion and Future Directions

This research demonstrates a robust, automated method for targeting data acquisition in climate modeling. While successful in a "perfect-model" setting, future work must address "structural" model errors—where the underlying equations themselves may be slightly incorrect. The next step is to apply this framework to the full-scale, high-complexity climate models used in international climate assessments like the IPCC reports.

Summary updated

Abstract

Abstract Data required to calibrate uncertain general circulation model (GCM) parameterizations are often only available in limited regions or time periods, for example, observational data from field campaigns, or data generated in local high‐resolution simulations. This raises the question of where and when to acquire additional data to be maximally informative about parameterizations in a GCM. Here we construct a new ensemble‐based parallel algorithm to automatically target data acquisition to regions and times that maximize the uncertainty reduction, or information gain, about GCM parameters. The algorithm uses a Bayesian framework that exploits a quantified distribution of GCM parameters as a measure of uncertainty. This distribution is informed by time‐averaged climate statistics restricted to local regions and times. The algorithm is embedded in the recently developed calibrate‐emulate‐sample framework, which performs efficient model calibration and uncertainty quantification with only model evaluations, compared with evaluations typically needed for traditional approaches to Bayesian calibration. We demonstrate the algorithm with an idealized GCM, with which we generate surrogates of local data. In this perfect‐model setting, we calibrate parameters and quantify uncertainties in a quasi‐equilibrium convection scheme in the GCM. We consider targeted data that are (a) localized in space for statistically stationary simulations, and (b) localized in space and time for seasonally varying simulations. In these proof‐of‐concept applications, the calculated information gain reflects the reduction in parametric uncertainty obtained from Bayesian inference when harnessing a targeted sample of data. The largest information gain typically, but not always, results from regions near the intertropical convergence zone.
BibTeX
@article{dunbar2022ensemblebased,
  author = {Oliver R. A. Dunbar and Michael F. Howland and Tapio Schneider and Andrew M. Stuart},
  title = {{Ensemble‐Based Experimental Design for Targeting Data Acquisition to Inform Climate Models}},
  journal = {Journal of Advances in Modeling Earth Systems},
  year = {2022},
  doi = {10.1029/2022ms002997},
}
Figure