Group Re-Identification via Unsupervised Transfer of Sparse Features Encoding

Giuseppe Lisanti, Niki Martinel, Alberto Del Bimbo, Gian Luca Foresti

TL;DR

The paper introduces PREF (Pooling Residuals of Encoded Features), an unsupervised transfer learning approach for group re-identification. By learning a sparse dictionary of human appearance from abundant single-person datasets, PREF encodes group images based on their deviation from these learned "atoms," achieving state-of-the-art performance without requiring manual foreground segmentation or target-domain training data.

What Problem Does This Paper Address?

While traditional person re-identification (Re-ID) focuses on tracking a single individual across disjoint camera views, people in the real world frequently move in groups. Group Re-ID leverages this collective visual context to resolve ambiguities like partial occlusions.

However, groups introduce unique challenges: individuals within a group frequently swap positions, occlude one another, or leave the group entirely, all against changing backgrounds.

Figure 1. Major group re-identification issues. Examples of: (a) position swap between person in a group; (b) different background between group images; (c) group images with partial occlusion.

Standard spatial descriptors fail when group layouts change. Existing group Re-ID methods often rely heavily on manual foreground/background segmentation masks or require training data from the exact same target environment, limiting their real-world scalability.

Methods

The authors propose a pipeline that transfers visual knowledge from single-person Re-ID datasets to the group Re-ID task using unsupervised sparse dictionary learning.

Figure 2. Proposed group re-identification pipeline. Top row shows the unsupervised single-person dictionary learning. Bottom row depicts the re-identification process with feature extraction and subsequent sparse residual encoding obtained with the transferred dictionary atoms.
  1. Group Appearance Modeling: Group images are resized, and overlapping $16 \times 16$ patches are extracted. To suppress background noise without manual masks, the authors combine three person detectors (DPM, ACF, and R-CNN) to create a soft filtering mask that weights the pixel contributions of color histograms (HS, RGB, and Lab).
  2. Unsupervised Dictionary Learning: Using single-person datasets, the model learns a dictionary of sparse visual "atoms" (representing common human body parts or textures) via Least Angle Regression (LARS) and block-coordinate descent.
  3. Sparse Residual Encoding: For an unseen group image, the system calculates the residual vector (dissimilarity) between the group's patches and the learned single-person dictionary atoms, weighted by sparse reconstruction coefficients.
Proposed sparse residual encoding
  1. Sparse Residual Pooling: To ensure the final descriptor is invariant to layout changes and group size, the patch-level residuals are aggregated into a compact image signature using average pooling. Finally, PCA is applied, and matching is performed using cosine distance.

Datasets

The approach utilizes separate source datasets for dictionary learning and target datasets for evaluation:

  • Source Datasets (Single-Person): ETHZ, CAVIAR, and VIPeR.
  • Target Datasets (Group Re-ID):
  • i-LIDS groups: The standard benchmark containing 274 images of 64 groups from an airport arrival hall.
  • Museum Groups Dataset (New): 524 manually annotated images of 18 groups captured across four cameras in a museum hall.
  • Outdoor Groups Re-Identification Dataset (OGRE) (New): A large-scale, challenging dataset containing approximately 2,500 images of 39 groups in a parking lot, featuring severe viewpoint changes and lighting variations. Detections were collected weakly using a Kernelized Correlation Filter (KCF) tracker.

Key Insights and Ideas

  • Leveraging Single-Person Data for Groups: Group Re-ID data is notoriously scarce. The core "aha moment" is that a group can be effectively modeled as a flexible combination of basic human appearance parts learned entirely from abundant, unrelated single-person datasets.
  • Counter-Intuitive Spatial Invariance: One might think that preserving the relative spatial layout of people would help identify a group. However, the authors show that explicitly adding $(x,y)$ spatial coordinates to the feature vectors actually degrades performance by roughly 7%. Dropping spatial constraints makes the model robust to position swaps.
  • Average Pooling Over Max Pooling: While max pooling is common in visual encoding, average pooling performs significantly better here because it smooths out noisy patch assignments caused by background clutter and occlusions.

Key Results

  • State-of-the-Art Performance: On the standard i-LIDS groups benchmark, PREF outperforms existing state-of-the-art methods (like BRO and CRRRO) by approximately 8% in Rank-1 accuracy.
  • Superiority Over Standard Encodings: PREF consistently beats classic local feature encoding frameworks like the Improved Fisher Vector (IFV) and VLAD across all three evaluation datasets (i-LIDS, Museum, and OGRE). For instance, on the i-LIDS dataset, PREF achieved a Rank-1 accuracy of 31.1% compared to IFV's 26.3% and VLAD's 23.8%.

Limitations and Future Work

The primary limitation of PREF is its reliance on direct feature matching. Because it does not employ a dedicated cross-view metric learning step, its performance drops under drastic illumination changes between cameras. The authors note that integrating metric learning solutions into the sparse residual framework is a promising direction to better handle severe cross-view feature transformations.

Summary updated

Abstract

Person re-identification is best known as the problem of associating a single person that is observed from one or more disjoint cameras. The existing literature has mainly addressed such an issue, neglecting the fact that people usually move in groups, like in crowded scenarios. We believe that the additional information carried by neighboring individuals provides a relevant visual context that can be exploited to obtain a more robust match of single persons within the group. Despite this, re-identifying groups of people compound the common single person re-identification problems by introducing changes in the relative position of persons within the group and severe self-occlusions. In this paper, we propose a solution for group re-identification that grounds on transferring knowledge from single person re-identification to group re-identification by exploiting sparse dictionary learning. First, a dictionary of sparse atoms is learned using patches extracted from single person images. Then, the learned dictionary is exploited to obtain a sparsity-driven residual group representation, which is finally matched to perform the re-identification. Extensive experiments on the i-LIDS groups and two newly collected datasets show that the proposed solution outperforms state-of-the-art approaches.
BibTeX
@inproceedings{lisanti2017group,
  author = {Giuseppe Lisanti and Niki Martinel and Alberto Del Bimbo and Gian Luca Foresti},
  title = {{Group Re-Identification via Unsupervised Transfer of Sparse Features Encoding}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2017},
  doi = {10.1109/iccv.2017.268},
}
Figure