Toward stable, general machine-learned models of the atmospheric chemical system

Makoto Kelp, Daniel J. Jacob, J. Nathan Kutz, Julian Marshall, Christopher W. TessumZenodo 2020 · DOI

Abstract

Atmospheric chemistry models—used as components in models that simulate air pollution and climate change—are computationally expensive. Previous studies have shown that machine-learned atmospheric chemical solvers can be orders of magnitude faster than traditional integration methods but tend to suffer from numerical instability. Here, we present a modeling framework that reduces error accumulation compared to previous work while maintaining computational efficiency. Our approach is novel in that it: 1) uses a recurrent training regime that results in extended (>1 week) simulations without runaway error accumulation, and 2) can reversibly compress the number of modeled chemical species by >80% without further decreasing accuracy. We observe a ~260× reduction in computation time (~1900× when run on specialized hardware) compared to the traditional solver. We use random initial conditions in training to promote general applicability across a wide range of atmospheric conditions. For ozone (with an initial concentration range of 0–70 ppb), our model predictions over a 24-hour simulation period match those of the traditional solvers with median error of 2.7 ppb and less than 19 ppb error across 99% of simulations initialized with random noise. Error can be significantly higher in the remaining 1% of simulations, which include among the most extreme concentration fluctuations simulated by the reference model. Results are similar for total particulate matter (median error of 16 ug/m3 and <32 ug/m3 across 99% of simulations with concentrations ranging from 0-150 ug/m3). Finally, we discuss practical implications of our choice of modeling framework and next steps for improving performance. The machine learning models described here are not yet suitable replacements for traditional chemistry solvers but represent a step toward that goal.

Introduction

Simulating the Earth’s atmosphere is a significant computational challenge. Earth System Models (ESMs) and Air Quality Models (AQMs) must track hundreds of chemical species interacting across vast timescales. The chemical solver portion of these models is often the primary computational bottleneck, frequently doubling the total simulation cost. While machine learning (ML) surrogates offer potential speedups, they have historically struggled with numerical instability—where small errors grow exponentially—and poor generalization across diverse atmospheric conditions.

Methodology

The proposed framework addresses these challenges through a specialized architecture and training regime designed for stability and dimensionality reduction.

Encoder-Operator-Decoder Architecture

To handle the high dimensionality of chemical systems (e.g., 101 species in the CBM-Z/MOSAIC mechanism), the model employs an autoencoder-like structure. An Encoder network compresses the input species into a lower-dimensional "latent space" (as few as 16 features). An Operator network then simulates the chemical evolution within this compressed space over discrete time steps ($\Delta t = 1$). Finally, a Decoder translates these latent features back into the original chemical species concentrations.

Schematic of the Encoder-Operator-Decoder architecture used for dimensionality reduction and temporal evolution.

Recurrent Training and Optimization

A key innovation is the Recurrent Training Regime. Instead of training on single-step predictions, the model is trained to predict 24-hour sequences recursively. This approach forces the model to learn self-correction, preventing the runaway error accumulation typical of previous ML solvers. Furthermore, a weighted loss function allows for "specialized" models that prioritize accuracy for critical pollutants like ozone ($O_3$) or particulate matter (PM).

Results

The model was evaluated against the CBM-Z/MOSAIC reference model using 40 million training samples and 1 million independent test samples.

Computational Efficiency

The ML surrogate achieved a ~260x speedup over traditional solvers on a standard CPU. When deployed on specialized hardware (NVIDIA Tesla V100 GPU), the speedup increased to approximately 1,900x.

Accuracy and Stability

For ozone, the model demonstrated high accuracy with a median absolute error of 2.7 ppb over 24 hours. In 99% of simulations, error remained below 18.7 ppb. Crucially, the recurrent training enabled the model to remain stable for simulations exceeding 1.5 weeks, a significant improvement over prior methods that often failed within hours.

Dimensionality Reduction

The model successfully compressed 101 chemical species into just 16 latent features (an 84% reduction) without significant loss of accuracy. Interestingly, some latent features corresponded to known chemical regulators like $NO_x$ and $HO_x$, suggesting the model captured underlying physical principles.

Discussion and Limitations

While promising, several hurdles remain for full integration into 3D global climate models. The "long tail" of error in the most extreme 1% of chemical fluctuations remains a concern. Additionally, while stability is greatly improved, long-term drift still occurs after approximately two weeks. Future work will focus on integrating this surrogate into 3D models where it must interact with transport processes and complex emissions.

Summary updated

Abstract

Neural network model code for Kelp et al. 2020 "Toward stable, general machine-learned models of the atmospheric chemical system"
BibTeX
@inproceedings{kelp2020toward,
  author = {Makoto Kelp and Daniel J. Jacob and J. Nathan Kutz and Julian Marshall and Christopher W. Tessum},
  title = {{Toward stable, general machine-learned models of the atmospheric chemical system}},
  booktitle = {Zenodo},
  year = {2020},
  doi = {10.5281/zenodo.3711059},
}

Loading related content...

Figure