Parallel Augmentation and Dual Enhancement for Occluded Person Re-Identification

Zi Wang, Huaibo Huang, Aihua Zheng, Chenglong Li, Ran He

TL;DR

This paper introduces PADE, a novel approach for person re-identification in scenarios with significant occlusions. PADE tackles the imbalance between occluded test data and less-occluded training data by employing a Parallel Augmentation Mechanism (PAM) and a Dual Enhancement Strategy (DES) to improve feature representation.

What Problem Does This Paper Address?

Person re-identification (Re-ID) aims to match a person across different camera views. A major challenge in this field is "occluded Re-ID," where a person in an image is partially hidden by objects or other people. This problem is exacerbated by a common data imbalance: training datasets often contain fewer occluded images, while real-world test scenarios (especially query images) are frequently occluded. Existing methods often rely on data augmentation or external models to handle occlusions, but they can overlook this critical data imbalance and fail to fully leverage both global context and fine-grained details.

Methods

The proposed PADE method consists of two main components:

  • Parallel Augmentation Mechanism (PAM): Instead of applying augmentations sequentially, PAM generates an image triplet from a single input image. This triplet includes:
  • Base Augmentation (BA): Standard resizing and normalization.
  • Erasing Augmentation (EA): Randomly erases parts of the image to simulate occlusions.
  • Cropping Augmentation (CA): Irregularly crops the image to simulate partial views. This parallel generation creates a more diverse set of occluded and non-occluded views for training, helping the network become more robust to various occlusion scenarios.

  • Dual Enhancement Strategy (DES): This strategy focuses on improving the quality of extracted features. It interactively enhances global and local features:

  • Local to Global Enhancement: Local features are first enhanced by incorporating information from the global feature.
  • Global to Local Enhancement: Subsequently, the global feature is refined by absorbing the detailed information from the enhanced local features. This process, implemented using Relation-based Enhancement Modules (REM), ensures that both contextual understanding and fine-grained details are effectively utilized for better discrimination.

Datasets

The paper evaluates PADE on the following datasets:

  • Occluded Re-ID Datasets:
    • Occluded-Duke
    • Partial-REID
    • Occluded-ReID
  • Non-occluded Re-ID Datasets:
    • Market-1501
    • DukeMTMC-reID

The authors do not introduce new benchmarks but focus on demonstrating the effectiveness of their method on existing, widely used datasets.

Key Insights and Ideas

  • The Power of Parallel Augmentation: Generating occluded and cropped versions of an image in parallel rather than sequentially is crucial for creating a more robust training signal that directly addresses the query-gallery occlusion imbalance.
  • Interactive Feature Enhancement: Forcing global and local features to mutually enhance each other is more effective than treating them independently. This ensures that the network learns to leverage both the "big picture" and the "fine details" simultaneously.
  • Auxiliary-Clue Free: PADE achieves strong performance without relying on external models for pose estimation or segmentation, making it more practical and easier to deploy.
  • Generalizability: The PAM component can be readily integrated into existing Re-ID frameworks, demonstrating its versatility.

Key Results

  • On the Occluded-Duke dataset, PADE achieved a Rank-1 accuracy of 63.0% and an mAP of 72.3%, outperforming existing state-of-the-art methods.
  • On Partial-REID, PADE reached 84.8% Rank-1 accuracy and 89.3% mAP.
  • On Occluded-ReID, PADE also showed superior performance compared to methods without auxiliary clues.
  • When tested on non-occluded datasets like Market-1501 and DukeMTMC-reID, PADE still achieved competitive results, demonstrating its general applicability.
  • An ablation study confirmed that both PAM and DES contribute significantly to the overall performance improvement. Replacing traditional augmentation with PAM in other models (OSNet, ViT, TransReID) also led to performance gains.

Limitations and Future Work

  • Limitations: While PADE performs well, the paper notes that PFD (which uses an additional keypoint detection model) slightly outperforms PADE on mAP for Occluded-ReID, suggesting that explicit pose information can still be beneficial. The paper also acknowledges that the "erase operation" in PAM is random, and more sophisticated occlusion simulation could be explored.
  • Future Work: The authors plan to focus on adaptive data augmentation and dynamic feature enhancement to handle even more complex occlusion scenarios. They also suggest exploring methods that can dynamically adjust feature extraction based on the level of occlusion.

Summary updated

Abstract

Occluded person re-identification (Re-ID), the task of searching for the same person’s images in occluded environments, has attracted lots of attention in the past decades. Recent approaches concentrate on improving performance on occluded data by data/feature augmentation or using extra models to predict occlusions. However, they ignore the imbalance problem in this task and can not fully utilize the information from the training data. To alleviate these two issues, we propose a simple yet effective method with Parallel Augmentation and Dual Enhancement (PADE), which is robust on both occluded and non-occluded data and does not require any auxiliary clues. First, we design a parallel augmentation mechanism (PAM) to generate more suitable occluded data to mitigate the negative effects of unbalanced data. Second, we propose the global and local dual enhancement strategy (DES) to promote the context information and details. Experimental results on three widely used occluded datasets and two non-occluded datasets validate the effectiveness of our method. The code is available at PADE (GitHub).
BibTeX
@inproceedings{wang2024parallel,
  author = {Zi Wang and Huaibo Huang and Aihua Zheng and Chenglong Li and Ran He},
  title = {{Parallel Augmentation and Dual Enhancement for Occluded Person Re-Identification}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  doi = {10.1109/icassp48485.2024.10446014},
}
Figure