Abstract
Fine-grained visual categorization (FGVC) is a challenging task due to small inter-class variations and large intra-class variations. We propose a multi-branch and multi-scale learning network (MMAL-Net, also referred to as TBMSL-Net) that automatically identifies discriminative object and part regions without requiring manual bounding-box or part annotations. By leveraging an Attention Object Location Module (AOLM) and an Attention Part Proposal Module (APPM), the network processes images at multiple scales through a shared-parameter CNN backbone. Our approach achieves state-of-the-art performance on the CUB-200-2011, FGVC-Aircraft, and Stanford Cars datasets while maintaining high inference efficiency.
Introduction
General object recognition often focuses on broad categories (e.g., "bird" vs. "car"). In contrast, Fine-Grained Visual Categorization (FGVC) requires distinguishing between highly similar sub-categories, such as specific aircraft models or bird species. This task is complicated by low inter-class variation—where different species look nearly identical—and high intra-class variation caused by diverse poses, lighting, and backgrounds.
Traditional methods often relied on labor-intensive manual annotations (bounding boxes and part locations) to guide the model toward "discriminative regions" like the shape of a beak or the pattern on a wing. TBMSL-Net addresses this by using attention mechanisms to automatically locate these regions, enabling end-to-end training using only image-level labels.
Methodology
The proposed architecture utilizes a three-branch design built upon a shared ResNet-50 backbone. This allows the model to learn from global, object-level, and part-level features simultaneously without a significant increase in parameter count.
Attention-Driven Modules
- Raw Branch: Processes the full image to understand global characteristics.
- Object Branch (AOLM): The Attention Object Location Module aggregates feature maps from the CNN to create an activation map. By identifying where neurons fire most strongly across multiple layers, it generates a mask to crop the object from the background, providing a "finer scale" view of the target's structure.
- Part Branch (APPM): The Attention Part Proposal Module uses a sliding window approach on the feature maps to find regions with high information density. It employs Non-Maximum Suppression (NMS) to ensure the selected parts (e.g., a bird's head or wing) are distinct and informative.
Crucially, all three branches share the same parameters. This weight sharing ensures that learning from one scale (like a close-up of a part) benefits the others, improving overall robustness.
Experimental Results
The model was evaluated on three gold-standard FGVC datasets: CUB-200-2011 (200 bird species), FGVC-Aircraft (100 aircraft variants), and Stanford Cars (196 car models).
As shown in the results table, the proposed method consistently outperforms existing architectures. It achieved an accuracy of 89.6% on CUB-200-2011, 94.7% on FGVC-Aircraft, and 95.0% on Stanford Cars.
Beyond accuracy, the model demonstrates significant efficiency. During the testing phase, the "Part" branch can be discarded, as the "Object" branch alone provides optimal classification performance. This results in an inference time of just 1.80 ms per image on a Tesla P100, roughly three times faster than competing models like NTS-Net.
Conclusion
TBMSL-Net provides a robust and efficient solution for fine-grained recognition by automatically focusing on multi-scale discriminative regions. By eliminating the need for manual part annotations and optimizing for fast inference, it bridges the gap between high-accuracy research models and practical applications. Future work will explore adaptive window selection for the part proposal module to further enhance flexibility across diverse image types.
Summary updated