Uncertainty-aware segmentation for rainfall prediction post processing

Simone Monaco, Luca Monaco, Daniele ApilettiarXiv 2024 · DOI · arXiv

Abstract

Accurate rainfall prediction is critical for disaster management, yet traditional Numerical Weather Prediction (NWP) models often struggle with local spatial details and lack reliable uncertainty estimates. This paper introduces SDE U-Net, a novel deep-learning architecture that treats rainfall post-processing as an image segmentation task. By integrating Stochastic Differential Equations (SDEs) into a U-Net backbone, the model captures epistemic uncertainty more effectively than standard ensemble methods. Evaluated on complex topographical data from northwestern Italy, SDE U-Net demonstrates superior performance in both accuracy and reliability, particularly for extreme precipitation events.

Introduction

Numerical Weather Prediction (NWP) models are the foundation of modern meteorology, but they are highly sensitive to initial conditions and often fail to capture high-resolution spatial details. This leads to biases and deterministic outputs that do not convey the inherent uncertainty of the forecast. In high-stakes scenarios—such as flood warnings issued by the Italian Civil Protection—knowing the confidence level of a forecast is as vital as the prediction itself.

Traditional statistical post-processing often fails to model the complex, non-linear patterns of precipitation. This research addresses the need for a tool that can integrate multiple NWP models and provide a reliable "prediction interval" without sacrificing spatial accuracy.

Methodology

The authors reframe rainfall prediction as an image segmentation task, utilizing a U-Net architecture (an encoder-decoder model common in medical imaging) as the backbone. They compare three methods for injecting uncertainty awareness:

  1. Monte Carlo Dropout (MCD) U-Net: Randomly omits neurons during inference to generate a distribution of predictions.
  2. Deep Ensemble (Ens) U-Net: Trains multiple U-Nets with different initializations and averages their outputs.
  3. SDE U-Net: The primary contribution of this work, which views neural network transformations as a stochastic dynamical system governed by Stochastic Differential Equations. "Diffusion blocks" are placed at each skip connection to capture epistemic uncertainty (uncertainty stemming from model or data limitations).

To evaluate these models, the researchers employed the Coverage-Length-based Criterion (CLC). This metric balances the width of the prediction interval (precision) against the frequency with which actual rainfall falls within that interval (reliability).

Experimental Setup

The study focused on the Piemonte and Valle d’Aosta regions in northwestern Italy, areas where complex topography makes forecasting exceptionally difficult. The dataset included:

  • Ground Truth: Gridded daily cumulative precipitation observations from ARPA Piemonte (1957–present).
  • Model Inputs: Forecasts from four major NWP models: BOLAM-CNR, ECMWF IFS, COSMO-2I, and COSMO-5M.
  • Event Classification: Data was split into "Typical" events and "Intense" events (exceeding the 99th percentile of seasonal precipitation).

Key Results and Discussion

The deep learning models significantly outperformed the "PoorMan’s Ensemble" (PME), a simple average of NWP models. As shown in the performance summary below, the SDE U-Net achieved the highest accuracy for intense events.

Quantitative comparison of model performance across various metrics for typical and intense rainfall events.

Accuracy and Precision

For intense events, the SDE U-Net achieved the lowest root Mean Squared Error (rMSE) of $2.637 \times 10^{-2}$. While the baseline PME had high coverage (PICP), its prediction intervals were excessively wide, making them less useful for decision-making. The SDE U-Net reduced the Normalized Mean Prediction Interval Length (NMPIL) by nearly 10x for intense events compared to the baseline (0.345 vs 3.605).

The CLC Metric

The CLC metric highlights the trade-off between accuracy and reliability. A lower CLC score indicates a better balance. The SDE U-Net emerged as the clear winner for intense weather events, maintaining a lower CLC even as the penalty for wide intervals ($\eta$) increased.

CLC performance across different penalty levels (η) for typical and intense events, showing SDE U-Net's advantage in intense scenarios.

Conclusion

The SDE U-Net provides a principled way to model the uncertainty inherent in precipitation forecasting. By treating the weather map as an image and applying stochastic differential equations, the model preserves fine-grained local details while providing narrow, meaningful prediction intervals. Future work will focus on strictly distinguishing between aleatoric and epistemic uncertainty and exploring newer architectures like Transformers or Diffusion Models to further refine these forecasts.

Summary updated

Abstract

Accurate precipitation forecasts are crucial for applications such as flood management, agricultural planning, water resource allocation, and weather warnings. Despite advances in numerical weather prediction (NWP) models, they still exhibit significant biases and uncertainties, especially at high spatial and temporal resolutions. To address these limitations, we explore uncertainty-aware deep learning models for post-processing daily cumulative quantitative precipitation forecasts to obtain forecast uncertainties that lead to a better trade-off between accuracy and reliability. Our study compares different state-of-the-art models, and we propose a variant of the well-known SDE-Net, called SDE U-Net, tailored to segmentation problems like ours. We evaluate its performance for both typical and intense precipitation events. Our results show that all deep learning models significantly outperform the average baseline NWP solution, with our implementation of the SDE U-Net showing the best trade-off between accuracy and reliability. Integrating these models, which account for uncertainty, into operational forecasting systems can improve decision-making and preparedness for weather-related events.
BibTeX
@misc{monaco2024uncertainty,
  author = {Simone Monaco and Luca Monaco and Daniele Apiletti},
  title = {{Uncertainty-aware segmentation for rainfall prediction post processing}},
  year = {2024},
  eprint = {2408.16792},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.16792},
}
Figure