One-stage Prompt-based Continual Learning

Youngeun Kim, Yuhang Li, Priyadarshini PandaSubmission to ICLR 2024 · Rejected

TL;DR

Existing prompt-based continual learning (PCL) methods are computationally expensive because they require two separate passes through a Vision Transformer (ViT)—one to pick a prompt and another to process the image. This paper introduces OS-Prompt, a "one-stage" framework that uses intermediate layer embeddings as queries, slashing computational costs by ~50% while maintaining or even improving state-of-the-art accuracy.

The Challenge: The Two-Stage Bottleneck in Continual Learning

Continual Learning (CL) is the challenge of training a model on a stream of new data without it "forgetting" what it learned previously—a phenomenon known as catastrophic forgetting. Recently, Prompt-based Continual Learning (PCL) has emerged as a top-tier solution. Instead of retraining the whole model, PCL keeps a "pool" of learnable tokens (prompts) that instruct a frozen, pre-trained Vision Transformer (ViT) on how to handle specific tasks.

However, current PCL methods have a major efficiency bottleneck. To know which prompt to use for a given image, they first run the image through a "query ViT" to generate a feature vector. Then, they run the image again through the "backbone ViT" along with the selected prompts. This two-stage process effectively doubles the computational load (GFLOPs), making it difficult to deploy these models on edge devices or in real-time scenarios.

Comparison of Two-Stage vs. One-Stage PCL architectures and their performance trade-offs.

The Solution: OS-Prompt

To address this redundancy, the authors propose a streamlined, one-stage approach called OS-Prompt (One-Stage Prompt).

1. Intermediate Querying

Instead of a separate query network, OS-Prompt uses the [CLS] token embedding from an intermediate layer (typically layer 5) of the backbone ViT itself. This embedding acts as the query to select prompts from the pool for the subsequent layers.

2. The Stability Observation

The core justification for this architectural shift is the authors' finding that early layers in a ViT are remarkably stable during continual learning. Their feature space doesn't shift much as new tasks are added, making them reliable "anchors" for prompt selection.

Feature drift across ViT layers after training on subsequent tasks. Early layers remain highly stable.

3. Query-Pool Regularization (QR) Loss

Using intermediate features can sometimes lead to a slight drop in representation power compared to using final-layer features. To fix this, the authors introduce the OS-Prompt++ variant. It uses a Query-Pool Regularization (QR) Loss during training to align the relationship between the intermediate query and the prompt pool with the relationship seen in a frozen "reference" ViT's final layer.

Crucially, the QR loss and the reference ViT are only used during training. At inference time, the reference ViT is discarded, maintaining the 50% speedup.

Datasets Evaluated

The authors evaluated their method on standard class-incremental continual learning benchmarks:

  • Split CIFAR-100: 100 classes split into 10 tasks.
  • Split ImageNet-R: A challenging dataset with diverse styles (art, cartoons, etc.), split into 5, 10, and 20 tasks.
  • DomainNet: A large-scale dataset used for domain-incremental learning across five different domains (Clipart, Real, Sketch, etc.).

Key Insights and Results

  • The "Early Layer" Advantage: The most significant insight is that you don't need the "perfect" final-layer representation to pick a good prompt. Early-to-mid-layer features are "good enough" and, more importantly, more resistant to the feature drift that causes forgetting.
  • Eliminating Redundancy: By restructuring the architecture to be one-stage, the authors prove that the separate "query" pass used in prior SOTA methods like L2P and CodaPrompt was largely redundant.
  • Efficiency: OS-Prompt reduces GFLOPs and GPU latency by approximately 50% during inference compared to prior PCL methods.
  • Accuracy: Despite the massive speedup, OS-Prompt++ actually outperforms the previous state-of-the-art (CodaPrompt) by about 1.4% on ImageNet-R.
  • Robustness: The method holds up across different task lengths (5, 10, or 20 tasks) and even when using unsupervised pre-trained weights (DINO) instead of standard supervised ImageNet-1k weights.

Limitations and Future Work

  • Training Overhead: While inference is 50% faster, the OS-Prompt++ version (with QR loss) requires a reference ViT during training. This means its training-time computational cost is roughly equal to previous two-stage methods. The "light" OS-Prompt (without QR loss) is faster to train but has a marginal (~1%) accuracy drop.
  • Pre-training Dependency: Like all PCL methods, this approach relies heavily on a strong pre-trained backbone. If the initial pre-training is poor or irrelevant to the target stream, the prompts may struggle to compensate.
  • Future Directions: The authors suggest exploring how to integrate task-specific information even more deeply into the prompt selection process to further boost representation power in one-stage architectures.

Summary updated

Abstract

Prompt-based Continual Learning (PCL) has gained considerable attention as a promising continual learning solution as it achieves state-of-the-art performance while preventing privacy violation and memory overhead issues. Nonetheless, existing PCL approaches face significant computational burdens because of two Vision Transformer (ViT) feed-forward stages; one is for the query ViT that generates a prompt query to select prompts inside a prompt pool; the other one is a backbone ViT that mixes information between selected prompts and image tokens. To address this, we introduce a one-stage PCL framework by directly using the intermediate layer's token embedding as a prompt query. This design removes the need for an additional feed-forward stage for query ViT, resulting in $\sim 50\%$ computational cost reduction for both training and inference with marginal accuracy drop ($\le 1\%$). We further introduce a Query-Pool Regularization (QR) loss that regulates the relationship between the prompt query and the prompt pool to improve representation power. The QR loss is only applied during training time, so there is no computational overhead at inference from the QR loss. With the QR loss, our approach maintains $\sim 50\%$ computational cost reduction during inference as well as outperforms the prior two-stage PCL methods by $\sim 1.4\%$ on public class-incremental continual learning benchmarks including CIFAR-100 and ImageNet-R.
BibTeX
@misc{kim2024one,
  author = {Youngeun Kim and Yuhang Li and Priyadarshini Panda},
  title = {{One-stage Prompt-based Continual Learning}},
  year = {2024},
  eprint = {2402.16189},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.16189},
}
Figure