Elastic Multi-Gradient Descent for Parallel Continual Learning

Fan Lyu, Wei Feng, Yuepan Li, Qing Sun, Fanhua Shang, Liang Wan, Liang WangarXiv 2024 · DOI · arXiv

Introduction

Traditional machine learning paradigms typically categorize learning into two distinct frameworks: Multi-Task Learning (MTL) and Serial Continual Learning (SCL). In MTL, a model is trained on a fixed set of tasks simultaneously, assuming all data is available at once. In SCL, tasks arrive sequentially, requiring the model to learn new information without forgetting previously acquired knowledge.

However, real-world scenarios—such as autonomous driving or personalized AI assistants—rarely fit these neat categories. Data streams for various tasks often arrive at different times, overlap in duration, and vary in complexity. This paper introduces Parallel Continual Learning (PCL), a more realistic and challenging paradigm that bridges the gap between MTL and SCL.

Comparison of Multi-Task Learning, Serial Continual Learning, and the proposed Parallel Continual Learning paradigm.

The Dual Challenges of PCL

PCL presents a unique set of difficulties because it inherits the core problems of both its predecessors:

  1. Task Conflict: Similar to MTL, when multiple tasks are trained simultaneously, their respective gradients may point in conflicting directions, leading to suboptimal convergence or "interference."
  2. Catastrophic Forgetting: Similar to SCL, as the model continues to learn from new streams, it tends to overwrite the weights optimized for older tasks that are no longer actively streaming.

Existing methods struggle in the PCL setting. Standard Multi-Gradient Descent (MGDA), for instance, often prioritizes tasks with smaller gradients to ensure a common descent direction. In PCL, this can be detrimental because brand-new tasks typically have large, informative gradients that need aggressive optimization, while older, nearly-converged tasks have small gradients that can stall the overall learning process.

Proposed Methodology: EMGD and Memory Editing

To address these issues, the authors propose a unified framework consisting of Elastic Multi-Gradient Descent (EMGD) and Gradient-Guided Memory Editing (ME).

Overview of the EMGD framework, including gradient computation, elastic factor integration, and memory editing.

Elastic Multi-Gradient Descent (EMGD)

The core innovation of EMGD is the introduction of elastic factors ($\sigma$). Unlike traditional MGDA, which treats all task gradients with equal priority constraints, EMGD relaxes these constraints based on the current state of each task. This "elasticity" allows the model to find a Pareto descent direction that accommodates the high-velocity learning required for new tasks without compromising the stability of older ones.

The authors propose two primary strategies for calculating these elastic factors:

  • Gradient Magnitude Change (GMC): This strategy monitors the convergence state of a task. If a task's gradient norm is rapidly decreasing, it is considered to be in an active learning phase and is given more "elastic" room to influence the model update.
  • Gradient Similarity (GS): This strategy measures the alignment between different task gradients. If a task's gradient significantly conflicts with the majority, its constraints are adjusted to prevent it from derailing the collective progress.

By solving the resulting optimization problem via quadratic programming, EMGD efficiently determines an update direction that balances the needs of all active data streams.

Gradient-Guided Memory Editing (ME)

In rehearsal-based continual learning, a small buffer of samples from previous tasks (memory) is maintained to prevent forgetting. Typically, these samples are treated as static ground truth. However, as the model's internal representations evolve, these static samples can become sources of interference.

The authors introduce a novel Memory Editing mechanism that treats the stored data as dynamic. By calculating second-order derivatives, the system "edits" the pixels of stored images. This process nudges the memory samples so that their gradients align more closely with the current optimal descent direction. This proactive alignment reduces future task conflicts and significantly mitigates catastrophic forgetting.

Experimental Evaluation

To evaluate PCL, the authors developed "Parallel Split" (PS) versions of standard benchmarks, including PS-EMNIST, PS-CIFAR-100, and PS-ImageNet-TINY. These benchmarks simulate realistic environments with overlapping training windows and randomized task "debut" times.

Key Results

The EMGD framework demonstrated superior performance across all tested scenarios:

  • Accuracy: On the PS-CIFAR-100 benchmark (task-incremental setting), EMGD combined with Memory Editing achieved an average accuracy of 56.61%, outperforming standard SCL baselines like Experience Replay (ER, 47.32%) and Dark Experience Replay (DER++, 49.27%).
  • Forgetting Mitigation: In class-incremental settings on PS-ImageNet-TINY, the Memory Editing component reduced the performance drop due to forgetting to just -9.09%, a significant improvement over the -15.82% drop seen with average gradient baselines.
  • Computational Efficiency: Despite the use of quadratic programming and second-order derivatives for memory editing, the approach remains computationally feasible. The optimization for elastic factors scales with the number of active tasks rather than the total number of model parameters.

Conclusion and Future Directions

This research highlights the importance of moving beyond idealized learning paradigms toward the more complex reality of Parallel Continual Learning. By introducing elasticity into multi-gradient descent and treating memory as a dynamic, editable resource, the authors provide a robust framework for managing the trade-offs between learning new tasks and preserving old knowledge.

Future work may explore the application of PCL in distributed or federated learning environments, where data streams are naturally decentralized. Additionally, further optimization of the memory editing process could make it even more scalable for high-resolution imagery and massive-scale architectures.

Summary updated

Abstract

The goal of Continual Learning (CL) is to continuously learn from new data streams and accomplish the corresponding tasks. Previously studied CL assumes that data are given in sequence nose-to-tail for different tasks, thus indeed belonging to Serial Continual Learning (SCL). This paper studies the novel paradigm of Parallel Continual Learning (PCL) in dynamic multi-task scenarios, where a diverse set of tasks is encountered at different time points. PCL presents challenges due to the training of an unspecified number of tasks with varying learning progress, leading to the difficulty of guaranteeing effective model updates for all encountered tasks. In our previous conference work, we focused on measuring and reducing the discrepancy among gradients in a multi-objective optimization problem, which, however, may still contain negative transfers in every model update. To address this issue, in the dynamic multi-objective optimization problem, we introduce task-specific elastic factors to adjust the descent direction towards the Pareto front. The proposed method, called Elastic Multi-Gradient Descent (EMGD), ensures that each update follows an appropriate Pareto descent direction, minimizing any negative impact on previously learned tasks. To balance the training between old and new tasks, we also propose a memory editing mechanism guided by the gradient computed using EMGD. This editing process updates the stored data points, reducing interference in the Pareto descent direction from previous tasks. Experiments on public datasets validate the effectiveness of our EMGD in the PCL setting.
BibTeX
@misc{lyu2024elastic,
  author = {Fan Lyu and Wei Feng and Yuepan Li and Qing Sun and Fanhua Shang and Liang Wan and Liang Wang},
  title = {{Elastic Multi-Gradient Descent for Parallel Continual Learning}},
  year = {2024},
  eprint = {2401.01054},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2401.01054},
}
Figure