Mitigating Classifier Bias and Representation Drift in Incremental Learning

311 concepts264 papers

Imagine training a neural network to recognize dogs and cats. Once it achieves perfect accuracy, you decide to teach it a new category: birds. You feed the network a stream of bird images, updating its weights to minimize classification error.

If you then test the network on dogs and cats, you will find its performance has collapsed. This is catastrophic forgetting, the fundamental challenge of Class-Incremental Learning (CIL). When a model learns new classes sequentially without continuous access to past data, the optimization process aggressively overwrites previously learned parameters to accommodate the new data distribution.

To understand why this happens—and how to fix it—we must look closely at how neural networks update their internal representations and final decision boundaries.


The Mechanics of Forgetting: Why Networks Fail

Catastrophic forgetting is not a random loss of information; it is a structural consequence of standard deep learning optimization. Two primary phenomena drive this failure: representation drift (the erosion of internal feature extractors) and classifier bias (the distortion of the final decision layer).

The Softmax Tyranny

In a standard multi-class network, the final layer uses a global softmax function to compute probabilities across all known categories. As discovered by the authors of SS-IL: Separated Softmax for Incremental Learning, this architecture introduces a severe mathematical flaw during incremental updates.

When the network trains exclusively on new classes, the target probability for all historical classes is zero. Consequently, the gradients computed for the old classes are consistently positive. In gradient descent, a positive gradient forces the network to systematically decrease the corresponding weights. The global softmax layer actively forces the model to suppress the prediction scores of historical classes, creating an overwhelming bias toward the newly introduced data.

Feature Erasure vs. Classifier Drift

Does the network actually erase its visual understanding of past concepts, or does it simply forget how to label them?

Junhao Zheng argues that a major misconception in CIL is conflating representation loss with classifier drift. Often, the internal backbone retains the ability to extract distinct features for old classes, but the final linear classifier tilts entirely toward the new classes due to the data imbalance. Misidentifying the source of forgetting leads to overly complex preservation techniques when the underlying feature representations may actually remain intact.


Strategy 1: Rebalancing the Decision Boundaries

If the primary culprit is a biased final layer, we can mitigate forgetting by structurally altering how the classifier handles new categories.

Weight Imprinting

Instead of allowing gradient descent to slowly tilt the classifier, we can calculate the ideal weights mathematically. Qi et al. introduced weight imprinting, a technique that bypasses gradient training for new classes. When a new category is introduced, the model extracts its feature embeddings using the existing backbone. The average embedding vector (the prototype) is then directly "imprinted" as the weight vector for that class in the final layer. This provides an immediate, stable classification boundary without disturbing existing weights.

Normalization and Margin Ranking

When new classes are trained alongside a tiny fraction of old data, the weights associated with new classes grow disproportionately large in magnitude. To counteract this, Hou et al. proposed replacing standard dot-product layers with cosine normalization. By normalizing both the feature extractors and the class weights, the model eliminates magnitude imbalances, forcing old and new classes to compete solely on the angular similarity of their features.

One-Class Classifiers

Another way to eliminate cross-class interference is to abandon the global multi-class classifier altogether. Sun et al. developed a framework (ILCOC) that utilizes parallel, one-class classifiers. Each class learns its own tight "hypersphere" boundary independently. When a new category arrives, a new hypersphere is trained without shifting or compressing the boundaries of historical classes, completely neutralizing the global softmax problem.

Standard Global Softmax:
[Old Class 1] \ 
[Old Class 2]  -->  [Shared Softmax Layer]  --> (New data forces old boundaries to collapse)
[New Class 3] /

Parallel One-Class Classifiers (ILCOC):
[Old Class 1 Hypersphere]  --> (Untouched)
[Old Class 2 Hypersphere]  --> (Untouched)
[New Class 3 Hypersphere]  --> (Added independently)

Strategy 2: Replaying the Past (Real and Imagined)

A straightforward intuition for preventing forgetting is to remind the model of past data. However, storing every historical image violates memory constraints and privacy requirements. Modern methods solve this by strategically selecting or generating data.

Experience Replay via Game Theory

If we can only store a small buffer of old images, which ones should we keep? Shim et al. introduced ASER, a method that uses game-theoretic Shapley Values to evaluate memory samples. Instead of keeping random images, ASER selects "sentinel" data points that actively protect the decision boundaries of old knowledge while forcing the model to learn sharp boundaries for new information. Similarly, Gu proposed retrieving samples that suffer from the maximum gradient interference, targeting the specific memories most vulnerable to being overwritten.

Generative Replay

When real data cannot be stored due to privacy constraints, models can "hallucinate" their own training data. The authors of RGR and SDDGR leverage text-to-image foundation models like Stable Diffusion to generate synthetic examples of past classes. By replaying these generated images alongside real new data, the model maintains a balanced distribution, ensuring that the optimization path honors historical classes without retaining a single byte of original user data.


Strategy 3: Parameter Isolation and Prompting

In the era of large, pre-trained foundation models, the strategy has shifted from preventing representation drift to managing pre-existing knowledge.

Zhou et al. demonstrated a surprising insight: a completely frozen pre-trained backbone already possesses enough generalizability to distinguish between incremental classes. The challenge is not changing the features, but adapting them.

Instead of updating the entire network, parameter-isolation methods freeze the core weights and introduce lightweight, modular components for new tasks:

  • Prompts: Liu et al. introduced attention-aware self-adaptive prompts that guide a frozen Vision Transformer. The prompts absorb task-specific nuances while the underlying model remains completely unchanged, ensuring zero representation drift.
  • Enhancers: In decentralized settings, Liu et al. utilize tiny, trainable "Enhancer" modules alongside a frozen backbone, drastically reducing communication costs while preserving historical knowledge.

By separating the network into a stable base (which retains general knowledge) and a plastic adapter (which absorbs new categories), modern incremental learning architectures achieve a reliable balance between stability and adaptability.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_catastrophic_forgetting_incremental_lear_c186e3b3,
  title = {catastrophic forgetting (incremental learning interference)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/catastrophic-forgetting-incremental-learning-interference/rdr_c186e3b31e2e4ae7b31ce4057363ea9d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_catastrophic_forgetting_incremental_lear_c186e3b3_v1,
  title = {catastrophic forgetting (incremental learning interference)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/catastrophic-forgetting-incremental-learning-interference/rdr_c186e3b31e2e4ae7b31ce4057363ea9d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}

See something wrong? Report a mistake