Abstract
Solar Energetic Particle (SEP) events and their major subclass, Solar Proton Events (SPEs), can have unfavorable consequences on numerous aspects of life and technology, making them one of the most harmful effects of solar activity. Garnering knowledge preceding such events by studying operational data flows is essential for their forecasting. Considering only Solar Cycle (SC) 24 in our previous study, Sadykov et al. 2021, we found that it may be sufficient to utilize only proton and soft X-ray (SXR) parameters for SPE forecasts. Here, we report a catalog recording $\geq$ 10 MeV $\geq$ 10 particle flux unit SPEs with their properties, spanning SCs 22-24, using NOAA's Geostationary Operational Environmental Satellite flux data. We report an additional catalog of daily proton and SXR flux statistics for this period, employing it to test the application of machine learning (ML) on the prediction of SPEs using a Support Vector Machine (SVM) and eXtreme Gradient Boosting (XGBoost). We explore the effects of training models with data from one and two SCs, evaluating how transferable a model can be across different time periods. XGBoost proved to be more accurate than SVMs for almost every test considered, while outperforming operational SWPC NOAA predictions and a persistence forecast. Interestingly, training done with SC 24 produces weaker TSS and HSS2, even when paired with SC 22 or SC 23, indicating transferability issues. This work contributes towards validating forecasts using long-spanning data -- an understudied area in SEP research that should be considered to verify the cross-cycle robustness of ML-driven forecasts.
Introduction: The Challenge of Solar Proton Events
Predicting Solar Proton Events (SPEs)—bursts of high-energy particles from the Sun—is critical for protecting satellite electronics, GPS accuracy, and terrestrial power grids. However, forecasting these events is a classic "needle in a haystack" problem. SPEs are rare, occurring only a few hundred times over several decades, which creates a significant class imbalance for predictive modeling.
In a dataset spanning roughly 12,500 days across three solar cycles, only 538 days contained an SPE. Standard machine learning models often struggle with such skewed data, tending to ignore the rare "positive" cases. Furthermore, solar activity follows an ~11-year cycle, but no two cycles are identical. This study investigates whether models trained on one cycle can effectively transfer their predictive power to another.
Methodology and Data
The researchers utilized data from the Geostationary Operational Environmental Satellite (GOES) series, spanning 1986 to 2019. This period covers Solar Cycles 22, 23, and 24. Two primary machine learning architectures were compared: Support Vector Machines (SVM) and eXtreme Gradient Boosting (XGBoost).
Key technical steps included:
- Feature Engineering: 24 daily statistical features derived from proton and soft X-ray (SXR) flux were narrowed down to the 9 most impactful features using Gini Importance and Fisher-scoring.
- Imbalance Handling: Techniques such as standard oversampling, SMOTE (Synthetic Minority Over-Sampling Technique), and ADASYN (Adaptive Synthetic sampling) were tested to ensure the models could identify rare events.
- Validation: Performance was evaluated using the True Skill Statistic (TSS) and Heidke Skill Score (HSS2), which are robust metrics for imbalanced datasets.
Key Insights and Results
XGBoost Superiority
XGBoost consistently outperformed SVMs and traditional operational forecasts. The models achieved TSS scores between 0.72 and 0.75, significantly higher than the operational NOAA SWPC probabilistic forecasts, which scored approximately 0.49 for Solar Cycle 22. The best XGBoost configurations reached a recall of 0.90, successfully identifying 90% of the SPE events in the test set.
The Transferability Gap
A major finding of the study is the "transferability gap." Models trained on Solar Cycle 24—a historically "weak" and quiet cycle—performed poorly when tested on the more active Cycles 22 and 23. This suggests that models trained on recent, quieter data may be under-prepared for more intense solar activity in the future. Training on two solar cycles instead of one generally led to more stable and reliable models, though the weak-cycle training issue remained a hurdle.
Feature Importance
Proton flux remains the most critical predictor. While soft X-ray data improves model performance, the "persistence" of proton levels is the primary driver for predicting an SPE on the following day.
Limitations and Future Work
While machine learning models excel at identifying patterns in raw flux data, they currently lack the contextual knowledge used by human forecasters, such as identifying specific active regions on the Sun. Future research aims to integrate spatial data, including information about solar Active Regions (ARs), coronal mass ejections (CMEs), and magnetic field data, to provide the physical context needed to reduce false positives and improve cross-cycle robustness.
Summary updated