Towards stable training of parallel continual learning

Yuepan Li, Fan Lyu, Yuyang Li, Wei Feng, Guangcan Liu, Fanhua ShangarXiv 2024 · DOI · OpenReview

TL;DR

This paper tackles the critical issue of training instability in Parallel Continual Learning (PCL), where multiple tasks are learned simultaneously. They introduce SPCL, a novel method that uses orthogonalization techniques in both forward and backward propagation to stabilize training, leading to improved performance and robustness.

What Problem Does This Paper Address?

Imagine an autonomous vehicle that needs to learn from various sensors (cameras, lidar, radar) simultaneously. This is the essence of Parallel Continual Learning (PCL): training a model on multiple, often dynamic, data streams concurrently. While efficient, this setup suffers from severe training instability. This instability arises because:

  1. Forward Propagation Issues: Features from different tasks can become entangled in shared network layers, hindering the model's ability to learn task-specific knowledge and leading to inconsistent signal propagation.
  2. Backward Propagation Issues: Gradients from different tasks can conflict with each other. This means the updates intended to improve one task might degrade performance on another, making the overall training process erratic and prone to divergence.

Existing methods have tried to mitigate these issues, but they often struggle with the inherent complexities of balancing multiple, potentially conflicting, learning objectives in a dynamic, multi-task environment.

Methods

The proposed solution, Stable Parallel Continual Learning (SPCL), employs a two-pronged orthogonalization strategy:

  • Stable Forward Propagation (Doubly-block Toeplitz - DBT Matrix Orthogonality): For the network's convolutional layers, SPCL enforces orthogonality constraints using a DBT matrix representation. This technique aims to ensure that feature representations remain distinct and consistent across tasks, preventing entanglement and promoting stable signal flow during forward passes.
  • Stable Backward Propagation (Orthogonal Decomposition for Gradient Management): To tackle conflicting gradients, SPCL utilizes orthogonal decomposition. This method aims to align gradients in a way that minimizes conflicts and ensures that updates for one task don't negatively impact others. A key aspect here is optimizing the gradient matrix to have a low condition number, which is a measure of how sensitive the system is to input changes and a proxy for stability. They achieve this by minimizing the difference between the current gradient matrix and an orthogonalized version, while also controlling the diagonal elements of the gradient matrix.

Both strategies leverage "soft" orthogonal regularization, which adds a penalty to the loss function, offering a balance between computational efficiency and achieving the desired stability.

Datasets

The paper evaluates SPCL on three standard PCL benchmarks:

  • Parallel Split EMNIST (PS-EMNIST): 62 classes, split into 5 tasks.
  • Parallel Split CIFAR-100 (PS-CIFAR-100): 100 classes, split into 20 tasks.
  • Parallel Split ImageNet-TINY (PS-ImageNet-TINY): 200 classes, split into 20 tasks.

These datasets are used in both task-incremental and class-incremental learning scenarios, providing a comprehensive evaluation.

Key Insights and Ideas

  • Orthogonality is Key to PCL Stability: The paper strongly argues that the instability in PCL stems from a lack of orthogonality in both feature representations (forward pass) and gradient updates (backward pass).
  • Condition Number as a Stability Metric: They highlight the condition number of the gradient matrix as a crucial indicator of training stability in multi-task learning. A low condition number signifies a more well-behaved optimization landscape.
  • DBT Matrices for Efficient Orthogonality: The use of Doubly-block Toeplitz matrices is presented as an efficient way to enforce orthogonality in convolutional filters, reducing computational overhead.
  • Dual Orthogonalization Strategy: The paper's core innovation is the combination of orthogonalization for both forward and backward passes, addressing the problem holistically.
  • Balancing Orthogonality and Task Specificity: The proposed method aims for "approximate orthogonality" rather than strict orthogonality to avoid stifling learning by reducing natural variations between task gradients.

Key Results

  • Significant Performance Gains: SPCL consistently outperforms state-of-the-art methods on the evaluated PCL benchmarks (PS-EMNIST, PS-CIFAR-100, PS-ImageNet-TINY) in terms of average accuracy and reduced forgetting.
  • Improved Training Stability: Empirical analysis shows that SPCL significantly reduces gradient conflicts and the condition number of the gradient matrix, leading to more stable training dynamics. For instance, the condition number of gradients obtained by their method is shown to be very close to 1.
  • Robustness Across Datasets: The method demonstrates strong performance across different datasets and task configurations, indicating its generalizability.
  • Ablation Studies Validate Components: Experiments show that both the DBT-based orthogonality for filters and the gradient management strategy contribute to the overall performance improvement.

Limitations and Future Work

  • Hyperparameter Sensitivity: The authors acknowledge that the orthogonality regularization parameter requires careful tuning, and incorrect settings can lead to excessive or insufficient orthogonalization, impacting adaptability and potentially increasing training time.
  • Future Work: They suggest exploring adaptive methods for dynamically adjusting the regularization parameter to improve model flexibility and efficacy in response to varying task complexities and data distributions.

Summary updated

Abstract

Parallel Continual Learning (PCL) tasks investigate the training methods for continual learning with multi-source input, where data from different tasks are learned as they arrive. PCL offers high training efficiency and is well-suited for complex multi-source data systems, such as autonomous vehicles equipped with multiple sensors. However, at any time, multiple tasks need to be trained simultaneously, leading to severe training instability in PCL. This instability manifests during both forward and backward propagation, where features are entangled and gradients are conflict. This paper introduces Stable Parallel Continual Learning (SPCL), a novel approach that enhances the training stability of PCL for both forward and backward propagation. For the forward propagation, we apply Doubly-block Toeplit (DBT) Matrix based orthogonality constraints to network parameters to ensure stable and consistent propagation. For the backward propagation, we employ orthogonal decomposition for gradient management stabilizes backpropagation and mitigates gradient conflicts across tasks. By optimizing gradients by ensuring orthogonality and minimizing the condition number, SPCL effectively stabilizing the gradient descent in complex optimization tasks. Experimental results demonstrate that SPCL outperforms state-of-the-art methjods and achieve better training stability.
BibTeX
@misc{li2024towards,
  author = {Yuepan Li and Fan Lyu and Yuyang Li and Wei Feng and Guangcan Liu and Fanhua Shang},
  title = {{Towards stable training of parallel continual learning}},
  year = {2024},
  eprint = {2407.08214},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2407.08214},
}
Figure