Weakly-Supervised Part-Attention and Mentored Networks for Vehicle Re-Identification

Lisha Tang, Yi Wang, Lap-Pui Chau

TL;DR

This paper introduces a novel weakly-supervised approach for vehicle re-identification (Re-ID) that learns to identify vehicle parts without requiring manual annotations. Their method, PANet+PMNet, uses an attention mechanism to locate parts and a teacher-student learning framework to extract robust features, outperforming existing methods that rely on expensive annotations.

What Problem Does This Paper Address?

Vehicle Re-ID is crucial for applications like intelligent transportation and public security, aiming to match vehicle images across different cameras. A major challenge is the significant intra-class variance (same car looking different due to lighting, viewpoint, etc.) and subtle inter-class differences (different cars looking similar). To tackle this, researchers have focused on learning discriminative features from vehicle parts. However, existing methods either divide images into arbitrary patches (which don't align well with actual parts), use external tools to detect parts (requiring extensive annotations and prone to errors), or employ attention models that can be complex to train. This paper addresses the need for an effective part-based Re-ID method that is weakly-supervised, meaning it doesn't need manual part annotations, and is robust to the inherent challenges of Re-ID.

Methods

The proposed solution consists of two main components:

  • Part-Attention Network (PANet): This network is designed for weakly-supervised vehicle part localization. It uses a part-relevant channel recalibration mechanism to highlight important channels in the feature maps and a cluster-based mask generation module. Crucially, it achieves this without any explicit vehicle part annotations, instead using a generated "pseudo-label" from GrabCut for initial segmentation. During inference, PANet can predict refined foreground masks and K vehicle part masks.
  • Part-Mentored Network (PMNet): This network leverages the part masks from PANet to learn discriminative features. It employs a teacher-student learning paradigm. For each identified vehicle part, there's a "Teacher" branch that learns from cropped part-specific features and a "Student" branch that learns from the full feature map. This teacher-student setup allows the Students to learn robust part representations by being guided by the Teachers. A key innovation is that during inference, the PMNet can operate without the part masks from PANet, relying solely on the learned Student branches, thus avoiding potential errors from unstable part localization. PMNet also incorporates a multi-scale attention module (MAM) within its branches to capture fine-grained local features. The entire system is trained using a multi-task learning approach, combining global and part-level Re-ID losses, with Homoscedastic Uncertainty Learning (HUL) to automatically balance these losses.

Datasets

The paper evaluates its method on two widely-used vehicle Re-ID datasets:

  • VeRi776: A popular dataset with over 51,000 images of 776 vehicles captured by 20 cameras. It's known for its diverse viewpoints and challenging conditions.
  • VehicleID: A larger dataset with over 220,000 images of 26,000 vehicles, often used for evaluating performance on different test set sizes (small, medium, large).

The authors also created an "occluded vehicle test set" by applying random color patches to existing datasets to specifically test robustness to occlusion.

Key Insights and Ideas

  • Weakly-supervised Part Localization is Feasible: The core idea that you can effectively learn to identify vehicle parts without manual annotations is a significant contribution. This bypasses the costly and error-prone process of creating labeled datasets.
  • Teacher-Student for Robustness: The teacher-student framework is crucial. It allows the model to learn from potentially noisy part masks (from PANet) while ensuring the student model is robust by learning from the holistic image. The ability for the student to operate independently during inference is a major advantage.
  • Attention for Fine-Grained Details: The use of attention mechanisms, both in PANet for localization and MAM in PMNet for feature extraction, is key to capturing subtle visual cues that distinguish similar vehicles.
  • Multi-task Learning for Generalization: Combining global and part-level features through multi-task learning, with intelligent weighting via HUL, leads to better overall performance and generalization, especially for challenging scenarios like occlusions.

Key Results

  • State-of-the-art Performance: The proposed PANet+PMNet method achieves state-of-the-art results on both VeRi776 and VehicleID benchmarks, outperforming recent methods that require no extra annotations.
  • On VeRi776, it achieves an average increase of 3.0% in CMC@5 and 1.4% in mAP compared to other non-annotated methods.
  • On VehicleID, it shows an average improvement of 0.8% in CMC@1.
  • Robustness to Occlusion: The method demonstrates significant robustness to occluded vehicles, outperforming baselines and other methods by over 25% in mAP and CMC@1 on occluded test sets.
  • Efficiency during Inference: A key advantage is that PMNet can bypass the part localization step during inference, making it more efficient and less prone to errors from potentially unstable part masks.

Limitations and Future Work

  • Number of Parts (K): The number of vehicle parts (K) is a hyperparameter that needs to be set. While they found K=3 to be effective, exploring optimal K selection or adaptive part discovery could be future work.
  • Generalization to Other Domains: While the method shows good generalization on occluded Re-ID, further testing on datasets with extreme variations in lighting, weather, or vehicle types could be beneficial.
  • Further Exploration of Weak Supervision: The paper demonstrates the power of weak supervision for part localization. Future work could explore even weaker forms of supervision or unsupervised part discovery for Re-ID.

Summary updated

Abstract

Vehicle re-identification (Re-ID) aims to retrieve images with the same vehicle ID across different cameras. Current part-level feature learning methods typically detect vehicle parts via uniform division, outside tools, or attention modeling. However, such part features often require expensive additional annotations and cause sub-optimal performance in case of unreliable part mask predictions. In this paper, we propose a weakly-supervised Part-Attention Network (PANet) and Part-Mentored Network (PMNet) for Vehicle Re-ID. Firstly, PANet localizes vehicle parts via part-relevant channel recalibration and cluster-based mask generation without vehicle part supervisory information. Secondly, PMNet leverages teacher-student guided learning to distill vehicle part-specific features from PANet and performs multi-scale global-part feature extraction. During inference, PMNet can adaptively extract discriminative part features without part localization by PANet, preventing unstable part mask predictions. We address this Re-ID issue as a multi-task problem and adopt Homoscedastic Uncertainty to learn the optimal weighing of ID losses. Experiments are conducted on two public benchmarks, showing that our approach outperforms recent methods, which require no extra annotations by an average increase of 3.0% in CMC@5 on VehicleID and over 1.4% in mAP on VeRi776. Moreover, our method can extend to the occluded vehicle Re-ID task and exhibits good generalization ability.
BibTeX
@article{tang2022weakly,
  author = {Lisha Tang and Yi Wang and Lap-Pui Chau},
  title = {{Weakly-Supervised Part-Attention and Mentored Networks for Vehicle Re-Identification}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2022},
  url = {https://openreview.net/forum?id=OoMTeraSeQkr},
}
Figure