TL;DR
The Continual Resilient (CoRe) optimizer is a new "all-in-one" optimization algorithm that bridges the gap between stochastic mini-batch learning (where Adam excels) and deterministic batch learning (where RPROP shines). By blending Adam-like adaptive learning rates with RPROP-style step size updates and a unique "stability-plasticity" mechanism, CoRe achieves state-of-the-art performance across diverse tasks—from computer vision to reinforcement learning and molecular physics—often requiring only a single hyperparameter adjustment.
What Problem Does This Paper Address?
In the "zoo of optimizers," practitioners often face a dilemma. Adam is the industry standard for mini-batch stochastic gradient descent, but it can struggle with stability in certain regimes. RPROP (Resilient Backpropagation) is incredibly robust for batch learning because it relies on the sign of the gradient rather than its magnitude, but it fails miserably in noisy, mini-batch environments.
Furthermore, most optimizers require tedious hyperparameter tuning for every new task. The authors, Marco Eckhoff and Markus Reiher, aimed to create a "universal" optimizer that is robust, fast-converging, and generally applicable across the entire spectrum of machine learning tasks without requiring exhaustive search for the "perfect" settings.
Methods
The CoRe optimizer is a hybrid first-order gradient-based algorithm. Its novelty lies in how it synthesizes three distinct concepts:
- Adam-like Adaptation: It uses exponential moving averages of the gradient and its square to scale learning rates, including bias correction for the initial steps.
- RPROP-like Step Sizes: It incorporates a step-size adaptation logic where the step size increases if the gradient sign is constant (speeding up convergence) and decreases if the sign flips (signaling a jump over a minimum).
- Stability-Plasticity Balance: This is perhaps the most unique feature. CoRe calculates an "importance score" for each weight based on its contribution to reducing the loss. It can then "freeze" the most important weights (the "plasticity factor") to prevent "catastrophic forgetting" during lifelong learning or to stabilize training in noisy environments.
- Weight Decay: Unlike standard L2 regularization, CoRe’s weight decay is proportional to the actual weight update magnitude, preventing weights from exploding or collapsing too aggressively.
Datasets
The authors benchmarked CoRe against nine other optimizers (including Adam, SGD, RMSprop, and AdaGrad) using a diverse battery of tests:
- Image Tasks: MNIST (digits) and Fashion-MNIST for variational auto-encoding and classification.
- Reinforcement Learning: The "Cart-pole" problem using both naive and actor-critic algorithms.
- Super-Resolution: BSD300 dataset for single-image upscaling.
- Graph Learning: The Cora dataset for semi-supervised classification using Graph Convolutional Networks (GCNs).
- Time Series: Sine wave prediction using LSTMs.
- Physics/Chemistry: A specialized "lifelong Machine Learning Potential" (lMLP) task involving 8,600 $S_N2$ chemical reaction systems.
Key Insights and Ideas
- The Best of Both Worlds: CoRe successfully handles the "stochastic fluctuation" problem that kills RPROP in mini-batch settings, while maintaining the "robustness to gradient magnitude" that makes RPROP great for batch settings.
- The "One Hyperparameter" Rule: The authors found that for almost all tasks, a set of "generally applicable" hyperparameters works out-of-the-box. The only value that truly needs to be toggled is $s_{max}$ (the maximum step size), which changes depending on whether you are doing mini-batch ($10^{-3}$) or batch ($1$) learning.
- Importance Matters: The stability-plasticity mechanism (weight freezing) proved particularly powerful for "lifelong learning" tasks (like the lMLP potential), where the model must learn new data without destroying previously acquired knowledge.
Key Results
- Broad Superiority: CoRe achieved the highest final accuracy in 6 out of 9 PyTorch benchmark tasks. In the remaining three, it was always within the uncertainty interval of the top performer.
- Reinforcement Learning: CoRe was significantly more reliable in RL tasks where RPROP failed to learn entirely.
- Molecular Modeling: In the lMLP (Machine Learning Potential) task, CoRe outperformed Adam and RPROP significantly, achieving lower Root Mean Square Errors (RMSE) for both energy and atomic force predictions.
- Efficiency: Despite its complexity, the computational overhead of CoRe is negligible. An optimizer step takes less than 0.2% of the time required for a single loss function gradient calculation.
Limitations and Future Work
- Hyperparameter Complexity: While the authors provide "general" values, CoRe has more hyperparameters than SGD or Adam. This could be intimidating for users who want to tune beyond the suggested defaults.
- Task-Specific Freezing: The "stability-plasticity" feature (weight freezing) is highly beneficial for lifelong learning but can slightly degrade performance in standard image classification (ICD/ICF) if the freezing fraction ($p_{frozen}$) isn't tuned carefully.
- Future Directions: The authors suggest that future work could explore more automated ways to determine the importance scores or adapt the freezing mechanism dynamically during training.
Reference: Eckhoff, M., & Reiher, M. (2024). CoRe Optimizer: An All-in-One Solution for Machine Learning. arXiv:2307.15663v2 [cs.LG].
Summary updated