An integrated framework that combines machine learning and numerical models to improve wave-condition forecasts

Fearghal O’Donncha, Yushan Zhang, Bei Chen, Scott JamesJournal of Marine Systems 2018 · DOI

TL;DR

This paper introduces a novel framework that integrates physics-based wave models with machine learning to create more accurate wave condition forecasts. By combining ensemble predictions from multiple models and learning optimal weights based on historical performance, the system significantly outperforms individual models and traditional aggregation methods.

What Problem Does This Paper Address?

Accurate forecasting of near-shore wave conditions is crucial for various applications, from maritime operations to coastal management. Traditional physics-based numerical models, while powerful, are subject to uncertainties arising from their physical formulations, numerical discretizations, and input data. Ensemble forecasting aims to address this by running multiple simulations with perturbed inputs or physics, but aggregating these ensembles into a single, reliable forecast remains a challenge. Existing machine learning approaches often struggle with extreme events, and purely data-driven methods can lack the physical grounding needed for robust predictions. This paper tackles the problem of effectively combining the strengths of physics-based modeling and machine learning to improve the accuracy and reliability of wave condition forecasts, particularly in dynamic coastal environments.

Methods

The core of this research lies in an integrated framework that combines a physics-based wave model with a machine-learning aggregation technique.

  • Physics-Based Model: The Simulating WAves Nearshore (SWAN) model, a third-generation wave model, is used to simulate wave conditions.
  • Ensemble Generation: To create an ensemble of forecasts, the SWAN model's inputs and physics are perturbed. This includes:
  • Perturbing lateral boundary conditions using statistical bounds derived from observational data and WAVEWATCH III forecasts.
  • Varying model physics by running SWAN in its first-, second-, and third-generation modes. This results in a set of 15 unique ensemble members.

  • Machine Learning Aggregation: Two machine learning techniques are employed to aggregate the ensemble predictions into a single "best-estimate" forecast:

  • Ridge Regression (RR): A linear predictor that learns a weight vector to minimize the average mean square error (MSE) between predictions and observations, incorporating a penalty function to keep weights small and reduce variation.
  • Exponentiated Gradient (EG): Another linear predictor that also learns a weight vector but constrains the weights to a convex combination (summing to one), which can limit rapid fluctuations and potentially improve extensibility.

The aggregation process is "on-line," meaning it uses all available past data (real-time) to update the weight coefficients for each ensemble element.

Datasets

The study utilizes several datasets for forcing the SWAN model and for verification:

  • Wave Condition Forecasts: Data from the NOAA WAVEWATCH III Eastern North Pacific model.
  • Ocean Current Forecasts: Data from the Monterey Bay ocean forecasting system (ROMS model).
  • Wind Forecasts: Data from The Weather Company (TWC) API.
  • Observational Data: Measurements from three National Data Buoy Center (NDBC) buoys located in Monterey Bay (stations 46042, 46114, and 46240). These buoys provide wave conditions, along with other ocean and meteorological data.

The paper does not introduce a new benchmark dataset but rather uses existing observational data for model verification.

Key Insights and Ideas

  • Synergy of Physics and ML: The research demonstrates that combining physics-based models with machine learning aggregation can overcome the limitations of each approach in isolation. Physics models provide the underlying dynamics, while ML learns to optimally combine their outputs, accounting for their historical performance and current conditions.
  • Importance of Dynamic Weighting: The "on-line" aggregation approach, where weights are continuously updated based on recent performance and observations, is crucial for capturing the volatile nature of ocean waves. This allows the system to adapt to changing conditions and identify the best-performing ensemble members dynamically.
  • Ensemble Spread as Uncertainty Indicator: The study highlights that periods of large spread in ensemble model forecasts often correspond to periods of highest uncertainty and can be effectively managed by assigning larger weights to models that perform well during these times.
  • Non-Invasive Integration: A significant advantage of this framework is its non-invasive nature. The machine learning aggregation acts solely on model outputs, meaning it does not require modifications to the underlying physics-based models, making it easier to integrate into existing operational systems.

Key Results

  • The integrated framework significantly improved wave condition forecasts compared to traditional methods and the best-performing individual ensemble member.
  • The Exponentiated Gradient (EG) aggregation technique achieved the lowest Mean Absolute Percentage Error (MAPE) of 9.26%, a substantial reduction from the best individual model's MAPE of 20.8%.
  • The aggregated predictions consistently outperformed the best individual model, except for brief periods where the best model's error was already near zero.
  • The computed weights showed that the aggregation approach leveraged most members of the ensemble, with many weights having magnitudes notably different from zero, indicating a balanced contribution.

Limitations and Future Work

  • Computational Cost: The paper notes that due to the relatively high computational cost of the SWAN model, there were limits to the number of ensemble elements that could be created.
  • Ensemble Design: The experimental design for building ensembles was relatively simple, considering only statistical deviations of observational data. Future work could explore more sophisticated ensemble designs.
  • Model Diversity: The study primarily focused on perturbing a single wave model (SWAN). The authors suggest that adding other wave-forecasting models to the ensemble would be beneficial to counter potential biases of the SWAN model.
  • Prior Information: Future work could investigate how to incorporate prior information about model complexity and physics into the aggregating technique and parameterization.
  • Extensibility: The framework is designed to be non-invasive and easily integrated into operational systems. Future work could explore extending this to combine models with different complexities and physics, such as rule-based models with data-driven models.

Summary updated

BibTeX
@article{odonncha2018integrated,
  author = {Fearghal O’Donncha and Yushan Zhang and Bei Chen and Scott James},
  title = {{An integrated framework that combines machine learning and numerical models to improve wave-condition forecasts}},
  journal = {Journal of Marine Systems},
  year = {2018},
  doi = {10.1016/j.jmarsys.2018.05.006},
}
Figure