Part Relational Mean Model for Group Re-Identification

Ping Hu, Hongwei Zheng, Wei-Shi ZhengIEEE Access 2021 · OpenReview

Introduction

Pedestrian re-identification (ReID) has traditionally focused on identifying single individuals across non-overlapping camera views. However, in urban environments, people frequently move in groups—families, friends, or colleagues. Group Re-Identification (G-ReID) addresses this reality but introduces significant complexities. Unlike single-person ReID, G-ReID must account for "deformable" group structures where members change relative positions or the group size itself fluctuates.

The G-ReID Challenge

The primary difficulties in G-ReID stem from two factors:

  1. Layout Variability: Group members often swap positions (e.g., walking side-by-side vs. in a line), making global image descriptors unreliable.
  2. Membership Dynamics: Individuals may join or leave a group between camera sightings, requiring a model that can handle varying numbers of features.

Traditional global approaches often fail to capture the internal relationships that define a group's unique visual signature.

The Part Relational Mean (PRM) Framework

To overcome these challenges, we propose the Part Relational Mean (PRM) model. This bottom-up approach shifts the focus from the group as a whole to the individuals within it and their mutual relationships.

Pipeline Overview

The PRM pipeline consists of three main stages:

  1. Detection: A Single Shot MultiBox Detector (SSD) identifies and crops each individual within a group image.
  2. Feature Extraction: A Part-based Convolutional Baseline (PCB) network extracts high-dimensional local features for each detected person.
  3. Descriptor Fusion: The core of the model uses two parallel descriptors to represent the group:
  4. Arithmetic Mean Descriptor ($h^M$): This averages the features of all group members, providing a fixed-dimension representation regardless of group size.
  5. Minus-Average Relational Descriptor ($h^R$): This calculates the average of absolute differences between all pairs of group members. This descriptor captures the "relational fingerprint" of the group—how members differ from each other—which remains more stable across different camera views than absolute appearance.

The BRT G-ReID Dataset

Recognizing the scarcity of large-scale G-ReID data, we introduced the Bus Rapid Transit (BRT) G-ReID dataset. This dataset contains 3,210 images of 200 groups captured across 53 different stations in five administrative areas.

The BRT G-ReID dataset captures groups across diverse urban locations, reflecting real-world surveillance challenges.

Unlike existing datasets often collected in controlled campus environments, the BRT dataset reflects the complexity of real-world urban travel, featuring diverse lighting, backgrounds, and group configurations.

Experimental Results

The PRM model was evaluated against state-of-the-art methods on three benchmarks:

  • i-LIDS MCTS: Achieved a Rank-1 accuracy of 38.6%, a 7.5% improvement over previous methods.
  • Road Group: Achieved a Rank-1 accuracy of 62.4%, outperforming existing algorithms by 19.4%.
  • BRT Dataset: Established a baseline Rank-1 accuracy of 50.6%, outperforming the baseline by 2.4%.

These results demonstrate that modeling the internal relationships of a group (the "minus-average" approach) is significantly more effective than treating the group as a monolithic entity.

Conclusion and Future Directions

The PRM model provides a robust framework for G-ReID by leveraging both the average appearance and the internal relational differences of group members. Future work will explore semi-supervised learning to reduce labeling costs and the integration of spatio-temporal data to further enhance tracking accuracy in large-scale camera networks.

Summary updated

Abstract

Most current research on pedestrian re-identification (ReID) is focusing on single-person ReID. However, people are rarely alone and often walk together in groups. Therefore, there is an urgent need to study the problem of group ReID (G-ReID). G-ReID is challenging because of the difficulties related to the differences in group appearance caused by changes in the group layout and membership. In this paper, we have proposed a part-based minus-average relational and arithmetic mean descriptor (PRM) algorithm to obtain a robust representation of groups. Based on local features, we have designed the arithmetic mean descriptor and the minus-average relational descriptor to solve the G-ReID problem caused by changes in the number of group members and their relative positions within the group. Moreover, the minus-average relational descriptor can also be used to describe the differences in the appearance of group members. Considering the rarity of G-ReID datasets and the need to improve the applicability of the G-ReID algorithm in real scenarios, we have collected a new dataset called the Bus Rapid Transit (BRT) G-ReID dataset. Extensive experimental results demonstrate the effectiveness of the PRM algorithm and indicate that it outperforms state-of-the-art algorithms by 7.5% for the cumulative matching feature (CMC-1) on the i-LIDS MCTS group dataset and by 19.4% for the CMC-1 on the Road Group dataset and it outperforms the baseline by 2.4% for the CMC-1 on the BRT dataset.
BibTeX
@inproceedings{hu2021part,
  author = {Ping Hu and Hongwei Zheng and Wei-Shi Zheng},
  title = {{Part Relational Mean Model for Group Re-Identification}},
  booktitle = {IEEE Access},
  year = {2021},
  url = {https://openreview.net/forum?id=XlMRfDLJv6},
}
Figure