Abstract
Low-shot learning aims to recognize novel categories from a limited number of training examples. This paper introduces Covariance-Preserving Adversarial Augmentation Networks (CP-AAN), a novel framework that addresses the data scarcity problem by generating synthetic feature vectors for novel classes. Unlike previous methods that assume uniform variability across classes, CP-AAN leverages semantically related base classes and explicitly preserves the covariance structure of the data distribution. Experimental results on ImageNet demonstrate that CP-AAN significantly outperforms existing state-of-the-art methods, particularly in the challenging one-shot setting.
Introduction
Deep learning models typically require vast amounts of labeled data to achieve high performance. In contrast, humans can learn new concepts from just a few examples. This gap is addressed by low-shot learning, which focuses on training models to recognize "novel" classes with minimal supervision. A common strategy is data augmentation—generating synthetic data to supplement the few available real samples. However, existing augmentation techniques often suffer from two main issues: the universal variance assumption, which incorrectly assumes all classes share the same distribution shape, and mode collapse, where generative models produce repetitive, low-diversity samples.
Methodology
The proposed CP-AAN framework operates at the feature level, generating high-dimensional vectors rather than raw pixels. It consists of three primary innovations:
Neighborhood Batch Sampling (NBS)
To avoid the pitfalls of universal variance, CP-AAN uses Neighborhood Batch Sampling (NBS). Instead of borrowing variability from random base classes, the model identifies semantically similar "base" classes (those with abundant data) to guide the generation of "novel" class features.
Cycle-Consistent GAN (cCyc-GAN)
To ensure the generated features remain semantically grounded, the authors employ a Cycle-Consistent GAN (cCyc-GAN). A generator $G_n$ translates base features into the novel class space, while an auxiliary generator $G_b$ attempts the reverse translation. This cyclic constraint stabilizes training and prevents the loss of class-specific "essence" during the augmentation process.
Covariance-Preserving Objective
The core intuition of CP-AAN is that while a novel class has few samples, its distribution's "shape" should mirror its semantic neighbors. The model uses a covariance-preserving loss based on the Ky Fan m-norm (the sum of singular values from a truncated SVD) to penalize differences between the covariance matrices of base classes and the generated novel features.
Experiments and Results
The model was evaluated on the ImageNet benchmark using a standard low-shot split (1,000 classes divided into base and novel sets). The performance was measured across various values of $K$ (number of training examples per novel class).
Key Findings
- Superiority in One-Shot Learning: At $K=1$, CP-AAN achieved 48.4% top-5 accuracy, representing a ~9% improvement over the baseline and outperforming previous generative approaches like Feature Imagination (45.0%).
- Diversity and Realism: Qualitative t-SNE analysis confirmed that CP-AAN generates a "cloud" of diverse features that closely match the true data distribution, whereas competing methods often produced tight "clumps" of nearly identical points.
- Consistency: The performance gains were consistent across all tested values of $K$ (1 to 20), demonstrating the effectiveness of covariance preservation as a robust signal for data augmentation.
Conclusion and Future Work
CP-AAN provides a mathematically grounded approach to feature-level data augmentation by focusing on the preservation of distribution "shape" via covariance. While the method introduces some computational complexity due to the required SVD calculations, its ability to generate diverse, class-specific variability sets a new standard for low-shot learning. Future research may explore higher-order feature matching beyond second-order covariance to further enhance the realism of synthesized data.
Summary updated