TL;DR
This paper tackles the "catastrophic forgetting" problem in continual learning for Vision-Language Models (VLMs). They propose a novel approach called PAM (Parameter Alignment for Merging) that merges newly learned task parameters with a global model, guided by an alignment mechanism during training, to achieve better knowledge retention and plasticity.
What Problem Does This Paper Address?
Continual learning (CL) aims to train models on a sequence of tasks without forgetting previously learned knowledge. A major challenge in CL, especially for large models like VLMs, is "catastrophic forgetting," where learning a new task drastically degrades performance on older tasks. Traditional sequential fine-tuning exacerbates this by favoring recent tasks. While parameter-efficient fine-tuning (PEFT) methods like LoRA help manage model size, they still struggle with forgetting. Existing solutions often involve complex regularization, replay mechanisms, or post-hoc merging, which can be inefficient or suboptimal. This paper proposes a simpler, more effective merging strategy that addresses both the stability-plasticity dilemma and the computational burden of CL.
Methods
The core of the proposed method, PAM (Parameter Alignment for Merging), is a novel approach to model merging for continual learning. Instead of simply fine-tuning a model sequentially, PAM:
- Model Merging: It fine-tunes a temporary LoRA module on a new task and then merges its learned parameters into a global LoRA module that represents all previously learned tasks. This is done via element-wise averaging.
- Parameter Alignment (During Training): To prevent interference between newly learned and existing parameters during the merging process, PAM introduces a "during-training" alignment mechanism. Periodically during the fine-tuning of the temporary LoRA, it checks for misalignment between its weights and the global LoRA weights. If weights have conflicting signs and the global weight is significant, the temporary weight is re-initialized. This alignment is controlled by a percentage parameter, balancing stability and plasticity.
- LoRA Integration: The entire process is built upon LoRA (Low-Rank Adaptation), a PEFT method, making it computationally efficient for large VLMs.
Datasets
The paper evaluates their approach on several benchmarks:
- CoIN: A benchmark for continual instruction tuning with six image QA tasks (VQAV2, TextVQA, OCR-VQA, ScienceQA, VizWizVQA, GQA).
- Extended CoIN: An expanded version of CoIN with four additional image-based tasks (AI2D, AOKVQA-MC, OKVQA, RSVQA-lr), creating a longer sequence of ten tasks.
- Video QA tasks: A sequence of three video QA tasks (MSVD-QA, MSRVTT-QA, ActivityNet-QA).
Key Insights and Ideas
- Merging as a Solution to Forgetting: The paper posits that model merging, rather than sequential fine-tuning, offers a more fundamental solution to the stability-plasticity dilemma in CL.
- "During-Training" Alignment is Crucial: Simply merging weights post-training can be suboptimal. PAM's key innovation is aligning parameters during the fine-tuning of a new task, which prevents interference and leads to more effective merging.
- Sign-Based Alignment is Effective: The paper leverages the observation that weight sign is a simple yet effective indicator for alignment, guiding the re-initialization process.
- Robustness to Task Order and Similarity: PAM demonstrates significantly better robustness to different task orders and varying degrees of task similarity compared to traditional fine-tuning.
- PAM is Complementary: The method can be effectively combined with existing CL techniques like regularization (LWF) and replay (ER) to further boost performance.
Key Results
- Reduced Forgetting: PAM significantly reduces catastrophic forgetting, as evidenced by higher Backward Transfer (BWT) scores compared to fine-tuning and other merging baselines. For instance, on the CoIN benchmark, PAM achieved a BWT of -19.45±0.95, a substantial improvement over fine-tuning (-39.51±9.87).
- Improved Accuracy: PAM achieves competitive or superior accuracy (ACC) across tasks. On CoIN, PAM achieved an ACC of 49.89±1.66, outperforming fine-tuning (43.36±8.18) and many other merging methods.
- Enhanced Generalization: The method shows improved Forward Transfer (FWT), indicating better generalization to unseen tasks. PAM achieved an FWT of 11.11±0.09 on CoIN.
- Robustness: PAM exhibits greater robustness to task ordering compared to fine-tuning, with less performance fluctuation.
- Efficiency: By leveraging LoRA, PAM offers a computationally efficient approach for large VLMs.
Limitations and Future Work
- Scenario Specificity: The paper acknowledges that while merging into a single LoRA is efficient, it might not fully address scenarios where new data drastically differs from previously learned tasks.
- Selective Fine-tuning: The authors suggest that selectively fine-tuning subsets of weights could further mitigate forgetting and enhance forward transfer.
- Further Exploration of Alignment: While PAM demonstrates the effectiveness of during-training alignment, further research could explore more sophisticated alignment strategies.
Summary updated