Representation Drift in Continual Learning

829 concepts598 papers1 proposal

When a neural network learns a sequence of tasks, we typically diagnose its failure through the lens of catastrophic forgetting—the sudden drop in performance on previously learned tasks. It is tempting to visualize this as the network explicitly overwriting its old decision boundaries. However, the underlying mechanism is often more subtle and destructive: representation drift.

Instead of the decision boundaries changing, the coordinate system of the latent space itself warps under the feet of the classifier. When a model updates its weights to accommodate new data, the feature extractor alters its mapping function. Consequently, if old data points are passed through this updated extractor, they project into entirely unfamiliar regions of the latent space, rendering previously accurate classifiers obsolete.

Initial State:
[ Class A Cluster ] ----> ( Fixed Classifier Boundary ) <---- [ Class B Cluster ]

After Learning Class C (Representation Drift):
          [ Class A Shifts ] 
                 |
                 v
[ Class C Cluster ] ----> ( Fixed Classifier Boundary ) <---- [ Class B Shifts ]

Why Output-Level Constraints Fail

A common baseline for preserving old knowledge is functional regularization, such as logit-based knowledge distillation. The intuition is straightforward: force the new model to mimic the output probabilities of the old model. However, as Hou et al. demonstrate, matching output probabilities alone fails to preserve the underlying geometric structure of the feature space. If the orientation of internal representations shifts, the fixed classification heads for older categories lose their predictive power.

To understand why this happens, we must look inside the hidden layers. Kim et al. introduced an analytical metric called representation discrepancy to measure this internal shift. Their theoretical framework reveals that representational drift is not uniform; deeper layers of a network experience significantly faster and more severe drift than shallower layers. This explains why output-level constraints are often too weak: they attempt to patch the symptoms at the final layer while the underlying feature foundation collapses.

This instability is not unique to supervised class-incremental learning. Zhaolong Du demonstrates that feature drift also severely degrades weakly supervised settings like Multiple-Instance Learning (MIL). Without explicit instance-level labels to anchor the representations, the feature extractor's mapping process easily drifts toward irrelevant patterns, eventually causing the model's performance to degrade to random guessing.

Paradigm 1: Latent Feature Realignment

If representation drift is inevitable when weights update, how can a model continue to leverage historical data? One approach is to accept the drift and actively correct it.

When models store historical features instead of raw images to save memory, those stored features become "stale" the moment the backbone updates. To solve this, Iscen et al. proposed a Feature Adaptation Network. Instead of keeping features static, this module learns to "translate" historical embeddings into the newly updated feature space.

Stale Feature Vector -------> [ Feature Adaptation Network ] -------> Aligned Feature Vector
                                                                             |
Updated Feature Extractor ---------------------------------------------------+---> [ Current Classifier ]

Other alignment strategies focus on modeling the transformation mathematically or statistically:

  • Geometric Rotations: Baek et al. utilize category-specific rotation matrices to dynamically shift historical features, ensuring they remain aligned with the evolving feature extractor without the massive memory overhead of storing raw images.
  • Energy Manifolds: Joseph et al. introduced an Energy-based Latent Aligner (ELI). By learning the energy manifold of the latent space, the model can identify exactly how far an old data point has drifted and mathematically "undo" the representational shift during inference.

Paradigm 2: Prototype Evolution and Augmentation

In exemplar-free continual learning, storing individual features or images is prohibited. Instead, models represent entire classes using a single central vector called a prototype. Representation drift causes these static prototypes to become disconnected from the current state of the network (De Lange et al.).

To counter this, Xiang et al. developed a method to mathematically estimate how old class features drift during incremental updates. This allows the model to shift the stored prototypes along the estimated drift vector, maintaining alignment with the updated feature extractor.

When prototypes cannot be perfectly tracked, researchers use augmentation to safeguard decision boundaries. Zhu et al. show that without original data, new class clusters tend to encroach on the territory previously occupied by old classes. By applying Prototype Augmentation—injecting structured noise into old prototypes—they create virtual decision boundaries that prevent new features from crowding out old representations.

Paradigm 3: Geometric Anchoring

Rather than correcting drift after it occurs, a third paradigm prevents drift entirely by constraining the geometry of the latent space during training.

One way to achieve this is through explicit spatial partitioning. Lyu et al. introduced a Two-level Angular Margin (TAM) loss that enforces strict angular boundaries between classes. By structuring the latent space into rigid, pre-allocated geometric sectors, the model guarantees that representations for new tasks cannot bleed into the space reserved for old ones.

Alternatively, optimization can be steered toward regions of the parameter space that are naturally resistant to drift. Shi et al. discovered that forcing the model into "flat minima" during the initial training phase creates a robust parameter buffer. In a flat minimum, subsequent weight updates for new tasks cause minimal changes to the network's underlying activation patterns, fundamentally stabilizing the mathematical representations of original classes against semantic drift.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 575 papers
  2. Version 1 · 2026-10-04 · 583 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_representation_drift_d5e0f432,
  title = {representation drift},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/representation-drift/rdr_d5e0f43203ba4b63988a2932a3813679},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_representation_drift_d5e0f432_v2,
  title = {representation drift},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/representation-drift/rdr_d5e0f43203ba4b63988a2932a3813679/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake