Introduction
In real-world machine learning deployments, data distributions are rarely static. A common phenomenon is virtual concept drift, where the underlying relationship between inputs and labels remains constant, but the frequency of specific classes changes over time. While the impact of this drift on model performance is known, its effect on the internal representations—the latent space—is less understood. This research, presented at AAAI-23, argues that the latent space is a "fragile ecosystem" where the stability of one class depends on the presence of others.
Methodology: Visualizing the Latent Space
To investigate these internal dynamics, the researchers utilized a two-stage training process on the MNIST dataset. First, a model was pre-trained on a balanced dataset ($D_1$). Second, it underwent incremental learning on a dataset ($D_2$) where one specific digit ($y_-$) was under-sampled by a factor of $\mu$.
To observe the model's internal state, the authors developed a novel decoder-based visualization method. A decoder was trained to reconstruct images from the model's latent representations after the first stage. By freezing this decoder and passing latent vectors from the second stage through it, they could visually track how the model's internal prototypes for different digits evolved or distorted as drift occurred.
Results: The Deception of Global Metrics
A key finding of this study is that standard performance metrics can be highly deceptive. In experiments where a single digit was under-sampled, the overall accuracy of the model remained remarkably stable, often dropping by only about 1%. However, this masked a catastrophic failure in class-wise accuracy for the drifted class, which plummeted by as much as 20%.
For example, when the digit '4' was under-sampled ($\mu=10$), its class-wise accuracy dropped by 20.66%, while the overall accuracy only dipped by 1.14%. This suggests that relying on global metrics in non-stationary environments can provide a false sense of security while the model fails on specific, rare cases.
Discussion: Generalized Confusion
The study also revealed a phenomenon of generalized confusion. Drift in one class does not just affect that class; it destabilizes the entire latent space. The authors observed that the features used to define one class could be "hijacked" by others. For instance, when digit '4' was under-sampled, the internal representation of digit '8' began to shift, eventually resembling a '3' in the latent space. This indicates that the presence of all classes acts as a form of implicit regularization that maintains the organization of the latent space.
Conclusion and Future Work
This research highlights the need for robust monitoring tools that look beyond global accuracy. While the current study focused on the relatively simple MNIST dataset and MLP architectures, it provides a crucial foundation for understanding latent space fragility. Future work will involve scaling these experiments to high-dimensional datasets like ImageNet and developing algorithmic "anchors" to stabilize the latent space against virtual concept drift.
Summary updated