Distance Learning for Analog Methods

Paul Platzer, Arthur Avenas, Bertrand Chapron, Lucas Drumetz, Alexis Mouche, Pierre Tandeo, Léo VinourMonthly 2025 · DOI

Abstract

Analogs are similar states of a system, occurring at remote times within independent numerical simulations or previous observations. This concept has emerged in atmospheric sciences and was further used in ocean sciences for forecasting, downscaling, upscaling, and extreme event attribution. The distance used to find and rate analogs is a key feature of analog methods. Most studies are based on the Euclidean distance or other pre-defined metrics. In this investigation, we adapt distance learning algorithms originally designed for classification and regression to statistical forecasting objectives, using the continuous-ranked probability score as a loss function. Our algorithm allows to jointly optimize three key hyperparameters of analog methods: the feature space, distance, the number of analogs used. In particular, this algorithm allows to reduce the feature space dimension while preserving high performances, a key requirement for small-size datasets. We test our algorithm on an idealized chaotic system and a tropical cyclone dataset. These experiments suggest that the optimal distance depends on the forecast horizon and the number of available data, and that our algorithm allows for reasonable performances of analog ensemble methods even for small-size datasets. Our algorithm runs faster then existing grid-search-like analog distances optimization algorithms. This allows to test and optimize a wider class of distances, for instance weighting a large number of predictive variables. Our approach is not limited to forecasting and can assist the search for optimal hyperparameters of any analog method, enhancing exploration possibilities and improving overall performances.

Introduction

In meteorology and oceanography, "analog methods" are a classic tool based on a simple intuition: if today’s atmospheric state looks like a specific day in the past, the weather tomorrow will likely follow a similar path. While intuitive and interpretable, these methods face a major technical hurdle: the distance metric. To find an "analog," one must calculate the distance between the current state and every state in a historical catalog. Most researchers default to standard Euclidean distance, treating every variable (temperature, pressure, wind speed) as equally important. However, in complex chaotic systems, some variables are much better predictors than others, and their importance changes depending on the forecast horizon. Manually tuning these weights is tedious and often impossible for high-dimensional data.

Methodology: Learning the Metric

This research bridges the gap between Metric Learning and atmospheric science by introducing a distance learning algorithm that automatically optimizes the similarity metric for probabilistic forecasting.

Mahalanobis-type Distance

Instead of a fixed distance, the algorithm learns a transformation matrix A. This matrix rotates and scales the input data so that the "distance" in the transformed space better reflects the actual predictive relationship.

CRPS as a Loss Function

Most metric learning algorithms are designed for classification. This paper adapts the approach for probabilistic forecasting by using the Continuous Ranked Probability Score (CRPS) as the loss function. CRPS measures how well a predicted probability distribution matches the actual outcome, rewarding both accuracy and reliable uncertainty estimates.

Joint Optimization and Regularization

The algorithm simultaneously tunes three things: the feature space (which variables to use), the distance metric (how to weight them), and the "bandwidth" (how many analogs to include in the forecast). By adding an $l_1/l_2$ regularization term, the algorithm can perform automatic variable selection, effectively "zeroing out" irrelevant data to prevent overfitting—a critical feature when working with small datasets.

Experimental Evaluation

The authors evaluated their approach on two distinct systems:

  1. Lorenz-63 System: A famous three-variable chaotic toy model that mimics atmospheric convection. This allowed for testing the algorithm's behavior across different forecast horizons and catalog sizes in a controlled environment.
  2. IBTrACS (Tropical Cyclone Data): A real-world dataset of North Atlantic tropical cyclones from 2003–2022. The focus was on predicting intensity changes (wind speed) using 15 input variables, including storm size, location, and translation speed.

Key Insights

  • Dynamic Similarity: The "optimal" distance is not static. The best metric for a 1-hour forecast differs significantly from the best metric for a 24-hour forecast. For short horizons, the current state of the variable itself is the dominant predictor; for longer horizons, dynamical indicators like translation speed become more important.
  • Efficiency with Small Data: The algorithm is particularly effective for small datasets. By being more "meticulous" in how it selects analogs, the optimized method can achieve performance on a small catalog that would normally require a much larger one.
  • Automatic Physics Discovery: In the tropical cyclone experiments, the algorithm naturally identified which variables were redundant or irrelevant without any prior physical guidance, correctly prioritizing maximum wind speed and Coriolis frequency.

Results and Conclusion

On the Lorenz system, the optimized distance consistently outperformed the standard Euclidean distance across various forecast horizons. For tropical cyclones, the method achieved a CRPS gain of 7% to 20% compared to unoptimized analog methods. The optimized analog ensembles also significantly outperformed standard baselines like persistence and climatology.

While the current implementation uses batch gradient descent, future work could involve scaling to massive gridded datasets using stochastic gradient descent. Additionally, a "weighted-CRPS" could be used to specifically target extreme events. Overall, this approach enhances the search for optimal hyperparameters of any analog method, improving both exploration possibilities and overall forecasting performance.

Summary updated

Abstract

Abstract Analogs are similar states of a system, occurring at remote times within independent numerical simulations or previous observations. This concept has emerged in atmospheric sciences and was further used in ocean sciences for forecasting, downscaling, upscaling, and extreme event attribution. The distance used to find and rate analogs is a key feature of analog methods. Most studies are based on the Euclidean distance or other pre-defined metrics. In this investigation, we adapt distance learning algorithms originally designed for classification and regression to statistical forecasting objectives, using the continuous-ranked probability score as a loss function. Our algorithm allows to jointly optimize three key hyperparameters of analog methods: the feature space, distance, the number of analogs used. In particular, this algorithm allows to reduce the feature space dimension while preserving high performances, a key requirement for small-size datasets. We test our algorithm on an idealized chaotic system and a tropical cyclone dataset. These experiments suggest that the optimal distance depends on the forecast horizon and the number of available data, and that our algorithm allows for reasonable performances of analog ensemble methods even for small-size datasets. Our algorithm runs faster then existing grid-search-like analog distances optimization algorithms. This allows to test and optimize a wider class of distances, for instance weighting a large number of predictive variables. Our approach is not limited to forecasting and can assist the search for optimal hyperparameters of any analog method, enhancing exploration possibilities and improving overall performances.
BibTeX
@inproceedings{platzer2025distance,
  author = {Paul Platzer and Arthur Avenas and Bertrand Chapron and Lucas Drumetz and Alexis Mouche and Pierre Tandeo and Léo Vinour},
  title = {{Distance Learning for Analog Methods}},
  booktitle = {Monthly},
  year = {2025},
  doi = {10.1175/mwr-d-24-0204.1},
}
Figure