Resolution-Agnostic Transformer-based Climate Downscaling

Declan J. Curran, Hira Saleem, Sanaa Hobeichi, Flora D. SalimarXiv 2024 · arXiv · OpenReview

TL;DR

This paper introduces an "Earth Vision Transformer" (Earth ViT) model capable of downscaling climate data to higher resolutions without requiring retraining for different input resolutions. This resolution-agnostic approach significantly reduces computational costs, paving the way for generating larger ensembles of climate projections for better extreme weather event planning and climate change adaptation.

What Problem Does This Paper Address?

Global Climate Models (GCMs) are essential for understanding future climate change, but they operate at coarse resolutions (tens to hundreds of kilometers). This coarseness is insufficient for capturing fine-scale atmospheric processes that drive extreme weather events and for detailed regional planning. Downscaling GCMs to finer resolutions using traditional methods like Regional Climate Models (RCMs) is computationally extremely expensive, requiring months of computation and millions of dollars for a single ensemble. This cost barrier limits the number of climate scenarios that can be generated, hindering comprehensive risk assessment and adaptation strategies. The paper aims to bridge this gap by developing a more efficient and flexible downscaling method.

Methods

The core of the proposed solution is a pretrained Earth Vision Transformer (Earth ViT) model, adapted from the Pangu-Weather model. The key innovation lies in its "resolution-agnostic" nature, meaning it can generalize to different input resolutions without needing to be retrained.

The Earth ViT model was initially trained on ERA5 data to downscale from 50 km to 25 km resolution. Crucially, it was then tested on the much higher resolution BARRA-SY dataset (3 km) without any additional fine-tuning.

A novel aspect is the introduction of a modified loss function. In addition to the standard Mean Squared Error (MSE) loss, a "Mass Conservation Loss" was incorporated. This loss penalizes predictions that violate the physical law of mass conservation, ensuring that the downscaled output accurately reflects the cumulative properties of the input region. This is achieved by comparing the total mass (sum of pixel values) in the low-resolution input with that of the super-resolved output.

Datasets

  • ERA5: Used for initial training of the Earth ViT model. It provided climate variables at a 50 km resolution, downscaled to 25 km.
  • BARRA-SY: An ultra-high-resolution dataset (3 km resolution) covering the Sydney and coastal New South Wales region. This dataset was used for testing the pretrained Earth ViT model's generalization capabilities without further training.

Key Insights and Ideas

  • Resolution Agnosticism is Achievable: The most significant insight is that a deep learning model, specifically a transformer architecture, can generalize across vastly different spatial resolutions for climate downscaling without explicit retraining. This breaks a major bottleneck in current downscaling practices.
  • Pretraining is Powerful: Leveraging a pretrained model (Earth ViT) trained on a large dataset (ERA5) allows for effective downscaling on a completely different, higher-resolution dataset (BARRA-SY) with minimal effort.
  • Physical Consistency Matters: Incorporating a mass conservation loss function demonstrably improves the model's performance, highlighting the importance of enforcing physical laws in climate modeling, even with data-driven approaches.
  • Cost-Effectiveness: The proposed method offers a substantial reduction in computational cost compared to traditional RCMs, making large ensemble climate projections economically feasible.

Key Results

  • Impressive Generalization: The Earth ViT model, trained on 50km ERA5 data, performed remarkably well on the 3km BARRA-SY dataset without any fine-tuning. This is evidenced by strong agreement in magnitude and spatial consistency for temperature predictions.
  • Improved Accuracy: In downscaling ERA5 temperature data, the Earth ViT with the modified loss function achieved an RMSE of 0.019, significantly outperforming Bilinear Interpolation (0.279) and ResNET (8.541).
  • Enhanced Physical Consistency: The mass conservation loss improved performance across various metrics, particularly in the BARRA-R results where the Earth ViT with modified loss achieved an RMSE of 0.029 for temperature.
  • Computational Savings: The paper emphasizes that this approach offers "large carbon savings over equivalent dynamical downscaling models," implying a significant reduction in computational resources and energy consumption.
  • Comparable Performance: For BARRA-R testing, the ERA5-pretrained Earth ViT performed comparably to an Earth ViT model specifically trained on BARRA data, further underscoring its generalization capabilities.

Limitations and Future Work

  • Limited Geographic Scope for Testing: While the model was tested on BARRA-SY (Australia), its performance across a wider range of geographical regions and climate zones is not extensively demonstrated.
  • Future Research Directions: The authors suggest refining Earth ViT to enhance its robustness across different regions and scenarios. They also highlight the potential for transfer learning to adapt the model to new regions or datasets, further expanding its applicability. The focus on improving physical consistency and exploring its use for predicting extreme weather events is also a key future direction.

Summary updated

Abstract

Understanding future weather changes at regional and local scales is crucial for planning and decision-making, particularly in the context of extreme weather events, as well as for broader applications in agriculture, insurance, and infrastructure development. However, the computational cost of downscaling Global Climate Models (GCMs) to the fine resolutions needed for such applications presents a significant barrier. Drawing on advancements in weather forecasting models, this study introduces a cost-efficient downscaling method using a pretrained Earth Vision Transformer (Earth ViT) model. Initially trained on ERA5 data to downscale from 50 km to 25 km resolution, the model is then tested on the higher resolution BARRA-SY dataset at a 3 km resolution. Remarkably, it performs well without additional training, demonstrating its ability to generalize across different resolutions. This approach holds promise for generating large ensembles of regional climate simulations by downscaling GCMs with varying input resolutions without incurring additional training costs. Ultimately, this method could provide more comprehensive estimates of potential future changes in key climate variables, aiding in effective planning for extreme weather events and climate change adaptation strategies.
BibTeX
@misc{curran2024resolution,
  author = {Declan J. Curran and Hira Saleem and Sanaa Hobeichi and Flora D. Salim},
  title = {{Resolution-Agnostic Transformer-based Climate Downscaling}},
  year = {2024},
  eprint = {2411.14774},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.14774},
}
Figure