LB-SCAM: a learning-based method for efficient large-scale sensitivity analysis and tuning of the Single Column Atmosphere Model (SCAM)

Jiaxu Guo, Juepeng Zheng, Yidan Xu, Haohuan Fu, Wei Xue, Lanning Wang, Lin Gan, Ping Gao, Wubing Wan, Xianwei Wu, Zhitao Zhang, Liang Hu, Gaochao Xu, Xilong CheGeoscientific 2024 · DOI

Abstract

The single-column model, with its advantages of low computational cost and fast execution speed, can assist users in gaining a more intuitive understanding of the impact of parameters on the simulated results of climate models. It plays a crucial role in the study of parameterization schemes, allowing for a more direct exploration of the influence of parameters on climate model simulations. In this paper, we employed various methods to conduct sensitivity analyses on the 11 parameters of the Single Column Atmospheric Model (SCAM). We explored their impact on output variables such as precipitation, temperature, humidity, and cloud cover, among others, across five test cases. To further expedite experimentation, we utilized machine learning methods to train surrogate models for the aforementioned cases. Additionally, three-parameter joint perturbation experiments were conducted based on these surrogate models to validate the combined parameter effects on the results. Subsequently, targeting the sensitive parameter combinations identified from the aforementioned experiments, we further conducted parameter tuning for the corresponding test cases to minimize the discrepancy between the results of SCAM and observational data. Our proposed method not only enhances model performance but also expedites parameter tuning speed, demonstrating good generality at the same time.

Introduction

Climate models, such as the Community Atmosphere Model (CAM), are essential for understanding our planet's future. However, they are incredibly complex. Much of the small-scale physics, such as cloud formation or turbulence, cannot be simulated directly and is instead represented by "parameterization schemes"—mathematical shortcuts with tunable parameters. Finding the optimal values for these parameters is a massive challenge; tuning a full 3D model is computationally prohibitive. While the Single Column Atmosphere Model (SCAM) serves as a cheaper proxy, it still requires thousands of simulation runs to understand parameter interactions and find optimal settings. This paper introduces LB-SCAM, an automated, efficient, and large-scale approach to this "knob-turning" process.

Methodology

The authors propose a three-part workflow called LB-SCAM (Learning-Based SCAM), designed to replace expensive physical simulations with efficient machine learning surrogates.

The LB-SCAM workflow overview.

The workflow consists of three primary stages:

  1. Sampling and Data Collection: Utilizing Morris and Sobol sampling sequences to generate a representative dataset of parameter combinations across 11 key physical parameters related to deep convection, shallow convection, and cloud fraction.
  2. Surrogate Model Training: Machine learning surrogates are trained to predict model outputs. Among architectures like Linear Regression, Random Forest, and XGBoost, ResNet-18 emerged as the top performer, accurately capturing complex non-linear responses of the physical model.
  3. Sensitivity Analysis (SA) and Tuning: The fast surrogate enables multi-method SA (Sobol, Morris, HDMR) to identify critical parameters. Finally, the Whale Optimization Algorithm (WOA) combined with localized grid search finds parameter sets that best match observational data.

Experimental Setup

The framework was evaluated across five established SCAM test cases representing diverse climatic conditions, ranging from land-based to tropical oceanic convection.

Summary of SCAM test cases.

The models were trained to predict four primary output variables: total precipitation (PRECT), humidity at 850 mbar (Q850), temperature at 850 mbar (T850), and total cloud cover (CLDTOT).

Results and Discussion

The LB-SCAM framework demonstrated significant improvements in both accuracy and efficiency:

  • Accuracy Boost: Optimized parameters led to a 24.4% increase in precipitation (PRECT) accuracy compared to default settings.
  • Efficiency Gain: The framework achieved a 50% reduction in computing cost compared to traditional optimization methods.
  • Parameter Sensitivity: The parameter zmconv_tau (convective consumption timescale) was identified as the most sensitive across nearly all cases, highlighting its critical role in atmospheric energy release.
  • Geographic Variation: Sensitivity varied significantly by geography. Land-based cases (ARM95/97) showed similar responses but differed sharply from tropical ocean cases, underscoring the limitations of global "one-size-fits-all" parameters.

Conclusion

LB-SCAM effectively leverages deep learning to accelerate climate model tuning. By replacing expensive physical simulations with a ResNet-based surrogate, the researchers achieved substantial accuracy gains while cutting computational costs in half. While currently applied to 1D single-column models, the ultimate goal is to translate these insights into the full 3D Community Earth System Model (CESM). Future work will focus on regional tuning maps, expanded parameter sets, and applying the workflow to other Earth system components like land and ocean modules.

Summary updated

Abstract

The single-column model, with its advantages of low computational cost and fast execution speed, can assist users in gaining a more intuitive understanding of the impact of parameters on the simulated results of climate models. It plays a crucial role in the study of parameterization schemes, allowing for a more direct exploration of the influence of parameters on climate model simulations. In this paper, we employed various methods to conduct sensitivity analyses on the 11 parameters of the Single Column Atmospheric Model (SCAM). We explored their impact on output variables such as precipitation, temperature, humidity, and cloud cover, among others, across five test cases. To further expedite experimentation, we utilized machine learning methods to train surrogate models for the aforementioned cases. Additionally, three-parameter joint perturbation experiments were conducted based on these surrogate models to validate the combined parameter effects on the results. Subsequently, targeting the sensitive parameter combinations identified from the aforementioned experiments, we further conducted parameter tuning for the corresponding test cases to minimize the discrepancy between the results of SCAM and observational data. Our proposed method not only enhances model performance but also expedites parameter tuning speed, demonstrating good generality at the same time.
BibTeX
@inproceedings{guo2024lb,
  author = {Jiaxu Guo and Juepeng Zheng and Yidan Xu and Haohuan Fu and Wei Xue and Lanning Wang and Lin Gan and Ping Gao and Wubing Wan and Xianwei Wu and Zhitao Zhang and Liang Hu and Gaochao Xu and Xilong Che},
  title = {{LB-SCAM: a learning-based method for efficient large-scale sensitivity analysis and tuning of the Single Column Atmosphere Model (SCAM)}},
  booktitle = {Geoscientific},
  year = {2024},
  doi = {10.5194/gmd-17-3975-2024},
}
Figure