Abstract
Maintaining the performance of neural networks in production environments is challenging due to concept drift. Traditional retraining methods often suffer from high computational costs or catastrophic forgetting. This paper presents a novel framework for continuous neural network retraining that integrates neuron importance regularization with Multi-Armed Bandit (MAB) algorithms. By intelligently selecting training data and weight clusters for updates, the proposed method achieves superior accuracy and generalization while remaining computationally efficient for real-time model serving.
Introduction
In real-world model serving, data distributions are rarely static. Models trained on historical data frequently experience performance degradation as underlying patterns shift—a phenomenon known as concept drift. Engineers typically face a trade-off between retraining on the entire accumulated dataset, which is computationally prohibitive, or fine-tuning on only new data, which often leads to catastrophic forgetting of previously learned edge cases. This work introduces a middle ground: an incremental retraining strategy that is both fast enough for production and robust enough to preserve historical knowledge.
Methodology
The proposed framework optimizes the retraining process through three key innovations:
- Synapse and Neuron Importance Regularization: Unlike traditional methods that only penalize changes to critical weights, this approach introduces a loss term that penalizes significant shifts in neuron activity. This ensures the model's structural behavior remains consistent, effectively mitigating forgetting.
- MAB-based Memory Replay: To optimize the use of historical data, the framework employs a Multi-Armed Bandit (MAB) algorithm. Instead of random sampling, the MAB treats data mini-batches as "arms" and learns to select those that provide the highest "reward," defined by the decrease in loss or gradient norm.
- MAB-based Weight Optimization: Observing that many weights in a network are highly correlated, the authors cluster these weights. A second MAB then decides which cluster to update at each training step. This reduces the number of active parameters per iteration, acting as a form of regularization that improves generalization.
Analysis of Weight Dynamics
A core motivation for the weight clustering approach is the observation that weights within different layers often move in tandem during training.
As shown in Figure 1, many weight pairs maintain high positive or negative correlations throughout the training process. By identifying and optimizing these correlated weights as single units, the retraining process becomes more efficient and less prone to overfitting.
Experimental Results
The framework was evaluated across six benchmarks, including image classification (MNIST, CIFAR-10), concept drift datasets (SEA, ELEC), and text classification (IMDB, REUTERS).
The MAB-based approach consistently outperformed state-of-the-art reservoir sampling, yielding an average accuracy boost of 0.48%. Notably, the "Clustering + MAB" strategy significantly improved generalization on augmented or distorted test data by an average of 4.9% compared to standard Batch Normalization.
In terms of efficiency, the MAB-driven selection process allows for rapid updates. For instance, on the REUTERS dataset, the proposed method was nearly 20 times faster than retraining on the full union of old and new data, while maintaining comparable or better performance.
Conclusion and Future Work
This paper demonstrates that combining neuron-level regularization with active resource allocation via MABs provides a powerful solution for life-long model retraining. While the current implementation achieves significant accuracy gains, further performance improvements could be realized through custom sparse tensor kernels optimized for the framework's selective weight updates. Future research will also explore integrating this method with automated drift detection systems and extending it to handle class-incremental learning scenarios.
Summary updated