A test case for application of convolutional neural networks to spatio-temporal climate data: Re-identifying clustered weather patterns

Ashesh Chattopadhyay, Pedram Hassanzadeh, Saba PashaarXiv 2018 · DOI · OpenReview

Abstract

Convolutional neural networks (CNNs) can potentially provide powerful tools for classifying and identifying patterns in climate and environmental data. However, because of the inherent complexities of such data, which are often spatio-temporal, chaotic, and non-stationary, the CNN algorithms must be designed/evaluated for each specific dataset and application. Yet to start, CNN, a supervised technique, requires a large labeled dataset. Labeling demands (human) expert time, which combined with the limited number of relevant examples in this area, can discourage using CNNs for new problems. To address these challenges, here we (1) Propose an effective auto-labeling strategy based on using an unsupervised clustering algorithm and evaluating the performance of CNNs in re-identifying these clusters; (2) Use this approach to label thousands of daily large-scale weather patterns over North America in the outputs of a fully-coupled climate model and show the capabilities of CNNs in re-identifying the 4 clustered regimes. The deep CNN trained with 1000 samples or more per cluster has an accuracy of 90% or better. Accuracy scales monotonically but nonlinearly with the size of the training set, e.g. reaching 94% with 3000 training samples per cluster. Effects of architecture and hyperparameters on the performance of CNNs are examined and discussed.

Introduction

Deep learning, specifically Convolutional Neural Networks (CNNs), has revolutionized image recognition. However, applying these tools to climate science is notoriously difficult. Climate data is spatio-temporal, chaotic, and "non-stationary," meaning its statistical properties evolve over time due to factors like anthropogenic climate change.

The primary bottleneck for applying supervised learning in this domain is the "cold start" problem: labeling. Identifying complex phenomena like atmospheric blocking events or specific circulation regimes requires significant expert time. This study proposes an "auto-labeling" strategy to bypass manual annotation by using unsupervised clustering to generate training data from raw climate model outputs.

Methodology and Data

The researchers implemented a two-step framework. First, unsupervised labeling was performed using K-means clustering to group daily weather patterns into four distinct regimes (C1-C4). Second, supervised re-identification was tested using two CNN architectures: a shallow 2-layer model (CNN2) and a deeper 4-layer model (CNN4).

The study utilized data from the Large Ensemble (LENS) Community Project, specifically focusing on geopotential height at 500 hPa (Z500) over North America from 1980 to 2005. A critical preprocessing step involved converting scalar data into RGB contour images and down-sampling them to $28 \times 28$ pixels. This down-sampling acts as a noise filter, removing transient small-scale fluctuations and forcing the CNN to focus on large-scale circulation patterns.

Results and Performance

The deeper CNN4 model achieved high test accuracy, reaching approximately 93.3% for summer and 93.8% for winter patterns. The confusion matrices demonstrate that the model consistently identifies the correct cluster across all four regimes, with high diagonal values indicating successful re-identification.

Confusion matrix for summer weather patterns (C1-C4) showing high re-identification accuracy.

The study found that accuracy scales nonlinearly with training set size. A "sweet spot" was identified at approximately 1,000 samples per cluster (yielding >90% accuracy), with diminishing returns observed after 3,000 samples. Furthermore, the 4-layer architecture consistently outperformed the 2-layer version by 4–7%, suggesting that depth is necessary to capture the hierarchical features of atmospheric flow.

Diagnostic Analysis

To understand the nature of classification errors, the authors analyzed "silhouette values," which measure how well a data point fits its assigned cluster. The results indicate that CNN performance is directly tied to the clarity of the initial clustering.

CNN identification accuracy as a function of cluster silhouette value.

As shown in the table above, accuracy increases from 89.0% for samples with low silhouette values ($s < 0$) to 95.2% for samples with high silhouette values ($s > 0.4$). This suggests that the CNN primarily struggles with "borderline" cases that were also ambiguous for the unsupervised K-means algorithm, rather than failing randomly.

Conclusion

This research demonstrates that CNNs can effectively navigate the complexities of spatio-temporal climate data when paired with an auto-labeling strategy. While the required data volume may be problem-dependent, the success in re-identifying North American weather regimes provides a roadmap for future applications. Future work may explore Recurrent Neural Networks (RNNs) or LSTMs to incorporate temporal "memory" into the classification of evolving weather patterns.

Summary updated

Abstract

Convolutional neural networks (CNNs) can potentially provide powerful tools for classifying and identifying patterns in climate and environmental data. However, because of the inherent complexities of such data, which are often spatio-temporal, chaotic, and non-stationary, the CNN algorithms must be designed/evaluated for each specific dataset and application. Yet to start, CNN, a supervised technique, requires a large labeled dataset. Labeling demands (human) expert time, which combined with the limited number of relevant examples in this area, can discourage using CNNs for new problems. To address these challenges, here we (1) Propose an effective auto-labeling strategy based on using an unsupervised clustering algorithm and evaluating the performance of CNNs in re-identifying these clusters; (2) Use this approach to label thousands of daily large-scale weather patterns over North America in the outputs of a fully-coupled climate model and show the capabilities of CNNs in re-identifying the 4 clustered regimes. The deep CNN trained with $1000$ samples or more per cluster has an accuracy of $90\%$ or better. Accuracy scales monotonically but nonlinearly with the size of the training set, e.g. reaching $94\%$ with $3000$ training samples per cluster. Effects of architecture and hyperparameters on the performance of CNNs are examined and discussed.
BibTeX
@misc{chattopadhyay2018test,
  author = {Ashesh Chattopadhyay and Pedram Hassanzadeh and Saba Pasha},
  title = {{A test case for application of convolutional neural networks to spatio-temporal climate data: Re-identifying clustered weather patterns}},
  year = {2018},
  eprint = {1811.04817},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1811.04817},
}
Figure