Aerial-Ground Person Re-ID

Huy Nguyen, Kien Nguyen, Sridha Sridharan, Clinton FookesICME 2023 · DOI

TL;DR

This paper introduces AG-ReID, the first large-scale dataset for person re-identification across aerial and ground-based cameras, addressing a critical gap in surveillance research. They also propose a novel two-stream explainable model that leverages soft-biometric attributes to effectively bridge the significant appearance differences between aerial and ground views.

What Problem Does This Paper Address?

Person re-identification (Re-ID) aims to match individuals across different cameras. While significant progress has been made in matching individuals captured by multiple ground-based cameras (ground-ground Re-ID) and even by multiple aerial cameras (aerial-aerial Re-ID), a crucial gap exists in matching individuals between aerial and ground viewpoints. This is becoming increasingly relevant with the rise of drone-based surveillance. The challenge lies in the drastic changes in appearance due to different viewpoints, altitudes, resolutions, and poses captured by aerial versus ground cameras, making traditional Re-ID methods struggle. Existing datasets and methods largely overlook this aerial-ground scenario.

Methods

The authors propose a two-stream explainable person Re-ID model.

  • Stream 1: Transformer-based Re-ID Model: This stream uses a Vision Transformer (ViT) backbone, chosen for its effectiveness with low-resolution features, to learn discriminative image features. It extracts feature maps and aggregates them using Generalized Mean Pooling (GeM) to obtain global features.
  • Stream 2: Explainable Re-ID Model: This stream also uses a similar backbone and focuses on learning from soft-biometric attributes. A key component is the Attribute Decomposed Head (ADH), which generates attribute-guided attention maps (AAMs). These AAMs highlight salient regions related to specific attributes, allowing the model to understand why two individuals might be matched or mismatched. The model learns to decompose salient regions of attributes without being biased by imbalanced attribute distributions.

The two streams are trained jointly with a metric distillation loss to ensure consistency between the feature-based and attribute-based distances, and an attribute prior loss that explicitly learns differences based on exclusive attributes. This explainable approach aims to provide insights into the matching process by visualizing attribute importance.

Datasets

  • AG-ReID (Newly Introduced): This is the core contribution, a novel dataset for aerial-ground person Re-ID. It contains 21,983 images of 388 identities, collected from a DJI XT2 drone (flying at 15-45 meters) and a fixed ground-based CCTV camera on a university campus. The dataset features 15 soft-biometric attributes per identity. It presents significant challenges due to low resolution, illumination changes, occlusions, motion blur, and varying poses.
  • Existing Datasets (for comparison): Market-1501, DukeMTMC-reID (ground-ground), PRAI-1581, UAV-Human (aerial-aerial).

Key Insights and Ideas

  • The Aerial-Ground Gap is Significant: The paper highlights that the visual appearance difference between aerial and ground views is a major hurdle for person Re-ID, more so than differences within ground-ground or aerial-aerial settings.
  • Soft-Biometric Attributes are Crucial: Leveraging soft-biometric attributes provides a powerful way to bridge the appearance gap, as these attributes can remain relatively consistent or understandable across different viewpoints.
  • Explainability Enhances Performance: The proposed explainable stream, by focusing on attributes, not only provides insights but also demonstrably improves Re-ID performance, particularly in challenging aerial-ground scenarios.
  • ViT is Well-Suited for Aerial Imagery: The Vision Transformer backbone shows promise for handling the low-resolution and diverse features present in aerial surveillance images.

Key Results

  • The proposed AG-ReID dataset presents a statistically significant and challenging task for person Re-ID.
  • The two-stream explainable model, when applied to AG-ReID, achieved improved rank-1 accuracy and mAP compared to baselines. For example, the explainable SBS model with ResNet50 backbone achieved 60.93% mAP and 74.21% rank-1 accuracy for aerial-to-ground matching.
  • The explainable stream consistently improved rank-1 accuracy for various backbones (ResNet, ViT, OSNet) on the AG-ReID dataset.
  • The paper demonstrates qualitative improvements, showing how attribute attention maps help visualize factors impacting similarity distances and improve Rank-5 accuracy.

Limitations and Future Work

  • Dataset Size: While novel, the AG-ReID dataset has 388 identities, which is smaller than some established ground-ground datasets. The authors mention an initial split of 397 identities, with 9 removed due to irrelevant details.
  • Future Work: The authors plan to release the AG-ReID dataset and baseline codes. They also intend to release video tracks to further enrich the dataset, suggesting future research could leverage temporal information. The paper also hints at exploring other backbones and further refining the attribute learning mechanisms.

Summary updated

BibTeX
@inproceedings{nguyen2023aerial,
  author = {Huy Nguyen and Kien Nguyen and Sridha Sridharan and Clinton Fookes},
  title = {{Aerial-Ground Person Re-ID}},
  booktitle = {ICME},
  year = {2023},
  doi = {10.1109/icme55011.2023.00440},
}
Figure