Abstract
Standard deep learning optimization typically employs a single global learning rate or simple decay schedules, often failing to capture the nuance of biological learning. This paper introduces Dual Variable Learning Rates (DVLR), a training technique inspired by behavioral psychology that utilizes separate learning rates for correct and incorrect responses. By updating these rates on a "variable ratio" schedule—analogous to reinforcement schedules in animal training—DVLR demonstrates consistent accuracy improvements across various neural network architectures.
Introduction
In current artificial neural network (ANN) paradigms, models are often treated as "black boxes," making it difficult to determine specific weight adjustments required for accuracy improvements. Standard optimizers apply updates uniformly, regardless of whether a network's prediction was nearly correct or entirely erroneous. To bridge this gap, we look toward behavioral psychology, specifically the reinforcement learning principles established by Edward Thorndike and B.F. Skinner. We hypothesize that providing psychologically motivated feedback—treating correct and incorrect responses as distinct signals—can enhance learning efficiency.
Methodology
The DVLR framework introduces two primary innovations:
- Dual Learning Rates ($\eta_C$ and $\eta_I$): The system maintains two distinct learning rates: $\eta_C$ for correct classifications and $\eta_I$ for incorrect ones. To maintain computational efficiency on modern hardware, a "majority rule" is applied to batches: if the majority of samples in a batch are classified correctly, the optimizer utilizes $\eta_C$; otherwise, it defaults to $\eta_I$.
- Variable Threshold (VT) Updates: Inspired by Skinner’s variable ratio reinforcement schedules, learning rates are not updated at fixed intervals. Instead, they are adjusted after a stochastic number of responses.
As shown in the figure above, the network tracks correct responses and triggers a learning rate adjustment once a randomly selected threshold (e.g., between 45 and 55) is reached. This prevents the model from falling into rigid, predictable update cycles.
Experimental Setup
The researchers evaluated DVLR on two benchmark computer vision datasets:
- MNIST: 70,000 handwritten digits.
- CIFAR-10: 60,000 color images across 10 distinct classes.
These datasets were tested across three architectures: Feedforward (FFNN), Convolutional (CNN), and Residual (ResNet18) networks.
Key Insights
- Learning Asymmetry: Optimal performance was frequently achieved by maintaining a stable "correct" learning rate ($\eta_C$) while aggressively adjusting the "incorrect" learning rate ($\eta_I$). This suggests that fine-tuning the reaction to errors is more beneficial than constantly altering the response to known data.
- Stochasticity and Nuance: The randomness introduced by the VT method appears to help the network avoid stagnation and discover subtle data nuances that standard schedulers might overlook.
- Architectural Scaling: The benefits of DVLR were more pronounced in complex models like CNNs and ResNets compared to simple FFNNs, indicating that deeper architectures require more nuanced feedback mechanisms.
Results
DVLR consistently outperformed both standard single-learning-rate baselines and "static-dual" baselines (where two rates are used without dynamic updates).
As detailed in the table above, the MNIST CNN accuracy improved from a baseline of 98.32% to 98.77% ($p < 0.05$). Similar trends were observed in more complex tasks:
- CIFAR-10 CNN: Accuracy rose from 61.14% to 63.28%.
- CIFAR-10 ResNet18: Accuracy improved from 72.73% to 74.96%.
Limitations and Future Work
The primary challenge for DVLR is the requirement for extensive hyperparameter tuning to identify optimal initial rates and thresholds. Future research will focus on developing a theoretical framework for threshold discovery and testing DVLR on larger, state-of-the-art models. Additionally, we aim to explore deeper biological parallels, such as varying learning rates across different simulated brain regions, to further refine the variable schedule approach.
Summary updated