Abstract
Deep Neural Networks face significant challenges in sequential learning, primarily due to catastrophic forgetting. We present a novel framework that utilizes Conditional Generative Adversarial Networks (cGANs) to mitigate this issue. Unlike traditional methods that store raw image exemplars, our approach preserves only the statistical distributions (mean and covariance) of class-specific feature embeddings. A generator then produces synthetic "pseudo-features" to facilitate adversarial rehearsal during incremental steps. Evaluations on CIFAR-100 and CUB-200 show that our method maintains higher accuracy and better plasticity than current state-of-the-art solutions while remaining memory-efficient.
Introduction
The ability to learn new information incrementally without forgetting previously acquired knowledge is a hallmark of biological intelligence but remains a hurdle for artificial neural networks. When trained on new data, standard models often experience "catastrophic forgetting," where weights optimized for previous tasks are overwritten. Existing solutions typically involve storing a subset of original images (exemplar-based) or freezing parts of the network (architecture-based). However, these methods either demand excessive memory or restrict the model's ability to learn complex new features.
Proposed Framework
The proposed architecture balances memory efficiency with model plasticity by operating in the feature space rather than the pixel space.
The system comprises three primary modules:
- Base Sub-net ($B$): A pre-trained feature extractor (e.g., ResNet) that remains fixed after the initial training phase to provide stable embeddings.
- Generator ($G$): This component generates synthetic feature maps ("pseudo-examples") conditioned on class-specific embeddings. These embeddings are sampled from Gaussian distributions defined by the stored mean and covariance of previous classes.
- Discriminator ($D$): A dual-headed network that performs both multi-class classification and adversarial discrimination (real vs. synthetic).
During incremental learning, the model combines real features from new classes with pseudo-features generated from the stored statistics of old classes. This "adversarial rehearsal" allows the discriminator to update its classification boundaries for both old and new categories simultaneously.
Experimental Results
The framework was evaluated on the CIFAR-100 and CUB-200 datasets, simulating incremental learning by introducing classes in varying batch sizes (steps of 2, 5, 10, or 25).
As shown in the performance plots, the proposed method consistently outperforms state-of-the-art baselines. While competitors like iCaRL and FearNet show rapid accuracy drops as the number of classes increases, our approach maintains a more robust performance profile.
Key insights from the experiments include:
- Efficiency: Storing second-order statistics (mean and covariance) requires constant memory per class, $O(d^2)$, which is far more scalable than storing raw images.
- Ablation Studies: Removing the adversarial loss led to a significant decrease in the retention of old knowledge, confirming that the "real vs. fake" discrimination is vital for high-quality pseudo-feature generation.
- Reconstruction Loss: Implementing a cosine-distance-based reconstruction loss prevents the generator from drifting, ensuring that synthetic features remain representative of their target classes.
Conclusion
This work demonstrates that generating perceptual features via cGANs is a powerful alternative to storing raw data for incremental learning. By leveraging statistical compression and adversarial rehearsal, the model achieves a superior balance between stability and plasticity. Future research will explore relaxing the Gaussian distribution assumption and adapting the base sub-net to further improve feature extraction for highly diverse sequential tasks.
Summary updated