Climate Modeling with Neural Diffusion Equations

Jeehyun Hwang, Jeongwhan Choi, Hwangyong Choi, Kookjin Lee, Dongeun Lee, Noseong Park

Abstract

Climate modeling requires balancing deterministic physical laws with the chaotic uncertainties of real-world data. We present Neural Diffusion Equations (NDE), a hybrid framework that integrates the physical rigor of the diffusion equation with the flexibility of Neural Ordinary Differential Equations (NODEs). By modeling both known heat spread physics and latent uncertainties, NDE significantly reduces forecasting errors—achieving up to a 55% improvement over state-of-the-art baselines in synthetic tests and 41% in real-world urban datasets.

Introduction

Climate modeling sits at the intersection of deterministic physics and chaotic uncertainty. Traditional models rely on differential equations, such as the diffusion equation, to describe the spread of temperature or humidity. However, these models often struggle with "real-world" noise—including human activity, local geography, and sensor errors—that defies simple mathematical descriptions. Conversely, standard deep learning models like Recurrent Neural Networks (RNNs) or Graph Neural Networks (GNNs) excel at pattern recognition but often ignore underlying physical laws, potentially leading to physically nonsensical predictions.

NDE bridges this gap by incorporating physical priors into a neural framework, allowing the model to "know" the physics of diffusion while using a neural network to learn the complex dynamics that physics alone cannot explain.

Methods

The core innovation is the Neural Diffusion Equation (NDE), which modifies the standard time-derivative of a climate state $h(t)$:

$$\frac{dh(t)}{dt} = -kLh(t) + f(h(t), t; \theta_f)$$

Overview of the NDE framework, showing the transition from current climate states to predicted values through the NDE-based climate model.

The framework consists of three primary components:

  1. The Physics Term ($-kLh(t)$): This represents the classic diffusion equation, where $L$ is the graph Laplacian (defining node connectivity) and $k$ is the heat capacity coefficient.
  2. The Neural Term ($f(...)$): A neural network tasked with modeling uncertainty. It captures residuals—the non-linear dynamics and noise that the simple diffusion term misses.
  3. Heat Capacity Generation: Rather than assuming a fixed constant for $k$, NDE learns it from data. Learning $k$ based on Edge Classes (e.g., land usage between stations) proved most effective.

The workflow follows a four-step process: Encode raw data into a latent space $\rightarrow$ Generate heat capacity coefficients $\rightarrow$ Evolve the state using the NDE solver $\rightarrow$ Decode the result back into climate variables.

Datasets and Evaluation

The authors evaluated NDE across three distinct data regimes:

  • Synthetic Data: Simulated heat diffusion on grid and small-world networks to test performance under "perfect" physical conditions.
  • LA & SD Datasets: Hourly observations from hundreds of stations in Los Angeles and San Diego, encompassing 10 variables including temperature, albedo, and wind vectors.
  • NOAA Dataset: Hourly average temperatures from 188 stations across the Eastern United States.
Visualization of climate variables across Los Angeles (LA) and San Diego (SD) datasets over multiple time steps.

Key Insights

  • Physics as a Prior: By using the diffusion equation as a starting point, the neural network focuses solely on the "hard" parts of the problem (noise and uncertainty), leading to more efficient learning.
  • Non-Uniform Heat Capacity: Realizing that heat capacity varies by terrain (e.g., concrete vs. forest) and learning $k$ based on edge attributes was a critical breakthrough.
  • Spatio-Temporal Synergy: The graph-based Laplacian ($L$) captures spatial relationships, while the NODE framework treats time as continuous, ensuring stable and flexible forecasting.

Results

NDE consistently outperformed eleven baselines, including LSTMs, GNNs, and physics-informed models like DPGN.

  • Accuracy: In the LA dataset, NDE's Mean Squared Error (MSE) was 41% lower than the best-performing baseline.
  • Robustness: On noisy synthetic data, NDE achieved an error rate 55% smaller than Neural Diffusion Convolutional Networks (NDCN).
  • Efficiency: NDE achieved top-tier results with significantly fewer parameters than large RNN-GNN hybrids, making it suitable for resource-constrained environments.

Conclusion and Future Work

While NDE is highly effective, the uncertainty modeling network $f$ remains somewhat of a "black box." Future research may explore Stochastic Differential Equations (SDEs) for better probabilistic forecasting. Additionally, while NDE excels at short-to-medium-term forecasting, its performance on decadal climate projections remains an open question.

Summary updated

BibTeX
@inproceedings{hwang2021climate,
  author = {Jeehyun Hwang and Jeongwhan Choi and Hwangyong Choi and Kookjin Lee and Dongeun Lee and Noseong Park},
  title = {{Climate Modeling with Neural Diffusion Equations}},
  booktitle = {ICDM},
  year = {2021},
  url = {https://openreview.net/forum?id=TVTmGUQSmUT},
}
Figure