Latent Space Evolution under Incremental Learning with Concept Drift (Student Abstract)

Charles Bourbeau, Audrey DurandAAAI 2023 · DOI

Introduction

In real-world machine learning deployments, data distributions are rarely static. A common phenomenon is virtual concept drift, where the underlying relationship between inputs and labels remains constant, but the frequency of specific classes changes over time. While the impact of this drift on model performance is known, its effect on the internal representations—the latent space—is less understood. This research, presented at AAAI-23, argues that the latent space is a "fragile ecosystem" where the stability of one class depends on the presence of others.

Methodology: Visualizing the Latent Space

To investigate these internal dynamics, the researchers utilized a two-stage training process on the MNIST dataset. First, a model was pre-trained on a balanced dataset ($D_1$). Second, it underwent incremental learning on a dataset ($D_2$) where one specific digit ($y_-$) was under-sampled by a factor of $\mu$.

To observe the model's internal state, the authors developed a novel decoder-based visualization method. A decoder was trained to reconstruct images from the model's latent representations after the first stage. By freezing this decoder and passing latent vectors from the second stage through it, they could visually track how the model's internal prototypes for different digits evolved or distorted as drift occurred.

Visualizing latent space evolution using a decoder-based projection method. As the under-sampling factor $\mu$ increases, the reconstructed prototypes for digits like '4' and '8' become distorted or shift toward other classes.

Results: The Deception of Global Metrics

A key finding of this study is that standard performance metrics can be highly deceptive. In experiments where a single digit was under-sampled, the overall accuracy of the model remained remarkably stable, often dropping by only about 1%. However, this masked a catastrophic failure in class-wise accuracy for the drifted class, which plummeted by as much as 20%.

Comparison of overall vs. class-wise accuracy evolution. While overall accuracy remains stable, the accuracy of the under-sampled class ($y_-$) drops significantly.

For example, when the digit '4' was under-sampled ($\mu=10$), its class-wise accuracy dropped by 20.66%, while the overall accuracy only dipped by 1.14%. This suggests that relying on global metrics in non-stationary environments can provide a false sense of security while the model fails on specific, rare cases.

Discussion: Generalized Confusion

The study also revealed a phenomenon of generalized confusion. Drift in one class does not just affect that class; it destabilizes the entire latent space. The authors observed that the features used to define one class could be "hijacked" by others. For instance, when digit '4' was under-sampled, the internal representation of digit '8' began to shift, eventually resembling a '3' in the latent space. This indicates that the presence of all classes acts as a form of implicit regularization that maintains the organization of the latent space.

Conclusion and Future Work

This research highlights the need for robust monitoring tools that look beyond global accuracy. While the current study focused on the relatively simple MNIST dataset and MLP architectures, it provides a crucial foundation for understanding latent space fragility. Future work will involve scaling these experiments to high-dimensional datasets like ImageNet and developing algorithmic "anchors" to stabilize the latent space against virtual concept drift.

Summary updated

BibTeX
@inproceedings{bourbeau2023latent,
  author = {Charles Bourbeau and Audrey Durand},
  title = {{Latent Space Evolution under Incremental Learning with Concept Drift (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i13.26943},
}
Figure