Abstract
Intra-class Part Swapping (InPS) is a specialized data augmentation strategy for fine-grained image classification. By swapping object parts between images of the same class using attention-guided spatial transformations, InPS avoids the "noisy label" problem common in methods like Mixup and CutMix. This approach preserves structural integrity and encourages the model to learn more discriminative features across the entire object. Experimental results on CUB-200-2011, Stanford-Cars, and FGVC-Aircraft demonstrate that InPS achieves state-of-the-art classification accuracy and significantly enhances weakly supervised object localization.
Introduction
Fine-grained image classification—distinguishing between highly similar categories such as specific bird species or car models—presents a unique challenge in computer vision. While standard data augmentation techniques like Mixup and CutMix have significantly improved general image classification, they often underperform in fine-grained tasks due to two primary issues: label noise and structural corruption.
As shown in the comparison above, methods like Mixup and CutMix often generate "noisy labels" by blending different classes, which can confuse models when inter-class differences are subtle. Furthermore, random cutting and pasting can destroy the holistic structure of an object, creating unrealistic training samples. InPS addresses these by restricting swaps to the same category and using guided transformations.
Methodology
The InPS pipeline consists of three main stages:
- Attention Priors: The model utilizes Class Activation Maps (CAM) to identify semantically meaningful regions of an object. These regions are stored in an "attention pool."
- Intra-class Selection: Unlike traditional methods that mix random images, InPS selects pairs from the same category. This ensures the ground-truth label remains 1.0 for that class, eliminating label noise.
- Guided Swapping: Using the attention pool, a part from a source image is swapped into a target image. An affine transformation (scaling and translation) is applied to ensure the swapped part fits naturally, maintaining the object's structural integrity.
Additionally, a simple mid-level feature learning sub-network (Conv1x1 followed by Max Pooling) is incorporated to capture granular details without significant computational overhead.
Experimental Results
The authors evaluated InPS on three standard benchmarks: CUB-200-2011 (birds), Stanford-Cars, and FGVC-Aircraft.
Classification Accuracy
InPS consistently outperformed baseline models. On the CUB-200-2011 dataset, InPS achieved a classification accuracy of 89.23% (using ResNet-101), surpassing CutMix (87.88%) and Mixup (87.80%). Similar improvements were observed across the Cars and Aircraft datasets.
Weakly Supervised Object Localization (WSOL)
The most significant gains were observed in localization. By forcing the network to recognize a class through various part combinations, the model learns to attend to the entire object rather than just the most discriminative feature.
As illustrated in the performance curves, InPS significantly improves localization accuracy across all tested datasets. On CUB-200-2011, it reached 72.28% accuracy, nearly doubling the performance of standard ResNet-50 + CAM (59.82%) and vastly outperforming Mixup (17.92%), which tends to blur object boundaries.
Conclusion
Intra-class Part Swapping (InPS) provides a robust solution for fine-grained classification by leveraging attention-guided, same-class augmentation. It effectively eliminates label noise and preserves object structure, leading to superior classification and localization performance. Future work may explore extending this mechanism to video recognition or more complex scene understanding tasks.
Summary updated