ViRN: Variational Inference and Distribution Trilateration for Long-Tailed Continual Representation Learning

Hao Dai, Chong Tang, Jagmohan ChauhanarXiv 2025 · DOI · arXiv

Abstract

Continual learning (CL) in real-world scenarios often faces the dual challenge of catastrophic forgetting and extreme data imbalance, known as Long-Tailed Continual Learning (LTCL). Traditional methods struggle to represent rare "tail" classes with minimal samples. We present ViRN (Variational Inference and Redistribution via Neighbors), a framework that leverages pre-trained embedding spaces to reconstruct missing statistical information for tail classes. By combining Variational Inference (VI) with a novel distribution trilateration technique, ViRN achieves a significant performance leap, outperforming state-of-the-art methods by an average of 10.24% across multiple benchmarks.

Introduction

In practical applications, data distributions are rarely uniform. A voice assistant may encounter common keywords thousands of times ("head" classes) while rarely hearing critical but infrequent sounds like a window breaking ("tail" classes). Standard CL models suffer from catastrophic forgetting, where new tasks overwrite old knowledge, and long-tailed bias, where the model ignores rare classes in favor of frequent ones. While generative replay is a common solution, it fails when a class has only one or two samples, as the model cannot accurately estimate the underlying distribution. ViRN addresses this extreme sample scarcity by "borrowing" structural information from semantically similar head classes.

Methodology

ViRN operates within the embedding space of high-quality pre-trained models (e.g., Wav2Vec for audio, DINOv2 for images) and employs a two-stage approach:

1. Variational Inference (VI)

Instead of using static prototypes, ViRN utilizes a Variational Autoencoder (VAE) to model each class as a Gaussian distribution. This probabilistic approach captures the uncertainty and variance of a class even when data is sparse.

2. Distribution Trilateration (RD)

This is the core innovation of ViRN. Recognizing that semantically similar classes cluster together, the model performs the following:

  • Neighbor Identification: It calculates the 2-Wasserstein distance between the estimated tail distribution and all known classes to identify the $K$ nearest "head" neighbors.
  • Reconstruction: The tail class's mean and covariance are reconstructed by interpolating between these well-defined neighbors.
  • Adaptive Fusion: The initial VAE estimate is fused with the trilaterated distribution using a weighting factor that favors the neighbors when samples are scarce and shifts toward observed data as more samples arrive.
Conceptual overview of distribution trilateration using Wasserstein distance to align tail distributions with head neighbors.

Experimental Setup

ViRN was evaluated on six long-tailed benchmarks with an imbalance ratio of $\rho = 0.01$ (the smallest class has 1% of the data of the largest). The datasets include four acoustic benchmarks (Speechcommands-LT, AudioMNIST-LT, ESC50-LT, UrbanSound8K-LT) and two visual benchmarks (CIFAR-100-LT, TinyImageNet-LT). Each dataset was partitioned into five sequential tasks to simulate a continual learning environment.

Key Results

ViRN demonstrates superior performance across all tested domains. As shown in the results table, ViRN achieved an overall summed accuracy of 524.26, a substantial improvement over the previous state-of-the-art, LAE (462.82). Notably, ViRN's performance closely approaches the "Upper Bound" (538.52), which represents a model trained on balanced, non-sequential data.

Comparison of ViRN against state-of-the-art continual learning methods across six long-tailed benchmarks.

Ablation studies highlight the necessity of the Redistribution (RD) component. For instance, removing RD led to a 44.96% drop in new-class accuracy on the ESC50-LT dataset. Furthermore, ViRN maintains high stability; in the Speechcommands task, it kept forgetting rates as low as 2.94%.

Discussion and Conclusion

The success of ViRN stems from its ability to treat class distributions as geometric entities within a semantic manifold. By using a generative classifier ($P(x|y)$) rather than a discriminative one, the model naturally decouples feature learning from class frequency.

However, limitations remain. ViRN is dependent on the quality of the pre-trained backbone; if the embedding space lacks semantic coherence for a specific domain, trilateration may be less effective. Additionally, the computational cost of calculating Wasserstein distances for every class may pose challenges for real-time edge applications. Future work will focus on optimizing these geometric calculations and exploring the framework's robustness across even more diverse data modalities.

Summary updated

Abstract

Continual learning (CL) with long-tailed data distributions remains a critical challenge for real-world AI systems, where models must sequentially adapt to new classes while retaining knowledge of old ones, despite severe class imbalance. Existing methods struggle to balance stability and plasticity, often collapsing under extreme sample scarcity. To address this, we propose ViRN, a novel CL framework that integrates variational inference (VI) with distributional trilateration for robust long-tailed learning. First, we model class-conditional distributions via a Variational Autoencoder to mitigate bias toward head classes. Second, we reconstruct tail-class distributions via Wasserstein distance-based neighborhood retrieval and geometric fusion, enabling sample-efficient alignment of tail-class representations. Evaluated on six long-tailed classification benchmarks, including speech (e.g., rare acoustic events, accents) and image tasks, ViRN achieves a 10.24% average accuracy gain over state-of-the-art methods.
BibTeX
@misc{dai2025virn,
  author = {Hao Dai and Chong Tang and Jagmohan Chauhan},
  title = {{ViRN: Variational Inference and Distribution Trilateration for Long-Tailed Continual Representation Learning}},
  year = {2025},
  eprint = {2507.17368},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.17368},
}
Figure