Grouping Attribute Recognition for Pedestrian with Joint Recurrent Learning

Xin Zhao, Liufang Sang, Guiguang Ding, Yuchen Guo, Xiaoming JinIJCAI 2018 · DOI

Abstract

Pedestrian attribute recognition in surveillance is challenging due to low resolution, poor lighting, and complex attribute correlations. This paper introduces Grouping Recurrent Learning (GRL), an end-to-end framework that organizes traits into semantic and spatial groups. By integrating a body-region proposal network with a Long Short-Term Memory (LSTM) network, GRL effectively captures "if-then" relationships and mutual exclusions (e.g., a person cannot be two different age groups simultaneously), achieving state-of-the-art results on major benchmarks like PETA and RAP.

Introduction

Recognizing pedestrian attributes—such as age, gender, and clothing—from surveillance footage is notoriously difficult. Images are often low-resolution, and datasets are relatively small. The authors identify a major gap in existing deep learning approaches: most models treat attributes as independent labels, failing to account for natural clusters. In reality, attributes exhibit strong correlations:

  1. Spatial Correlation: "Hat" and "Glasses" are both found in the head region.
  2. Semantic Correlation: "Long hair" is highly correlated with "Female."
  3. Mutual Exclusion (Mutex): A person cannot be "Teenager" and "Elderly" at the same time.

Previous methods often ignored these intra-group conflicts and inter-group dependencies, leading to illogical predictions.

Methodology

The GRL model consists of three main components designed to leverage these correlations.

Body Region Proposal

Instead of analyzing the whole image for every attribute, the model uses a Fully Convolutional Network (FCN) to detect body joints. It then generates specific regions of interest (ROIs) for the head, upper body, and lower body to focus the feature extraction process.

The body region generation process, from joint detection to sub-region calculation and ROI proposal.

Feature Extraction and Recurrent Learning

An Inception-v3 backbone extracts global features, while ROI average pooling extracts localized features from the proposed body regions. These features are fed into a Long Short-Term Memory (LSTM) network where attributes are organized into groups (e.g., a "Head" group containing hair style and glasses). The LSTM learns the context between groups (inter-group correlation), while attributes within the same group share a fully connected layer to learn intra-group relationships like mutual exclusion.

Key Insights

  • The Power of Order: The sequence in which the LSTM processes attribute groups matters. Predicting "global" attributes (like gender and age) first provides better context for predicting "local" attributes (like footwear or accessories).
  • Spatial Focus Matters: By using ROI pooling on specific body parts, the model avoids background noise and irrelevant body parts when identifying specific traits like hair color. Ablation studies showed that removing ROI pooling dropped accuracy on the PETA dataset by approximately 1%.
  • Solving the "Mutex" Problem: By grouping mutually exclusive attributes and processing them through shared layers, the model significantly reduces semantic errors, such as predicting two different body shapes for one person.

Experimental Results

The GRL model was evaluated on the two largest publicly available datasets: PETA (19,000 images, 35 binary attributes) and RAP (41,585 images, 51 binary attributes). It outperformed several high-profile competitors, including DeepMar, HydraPlus-Net, and JRL.

  • PETA Dataset: Achieved a mean Accuracy (mA) of 86.70% and an F1 score of 86.51%.
  • RAP Dataset: Achieved an mA of 81.20%, outperforming the multi-model ensemble JRL* by over 3%.

Conclusion

Grouping Recurrent Learning provides a robust solution for pedestrian attribute recognition by explicitly modeling the logical and spatial dependencies between traits. While the model currently relies on manually defined attribute groups and is dependent on the accuracy of initial joint detection, it demonstrates the significant benefits of structured, recurrent learning in computer vision tasks. Future work may explore automated group discovery to further enhance the model's flexibility.

Summary updated

BibTeX
@inproceedings{zhao2018grouping,
  author = {Xin Zhao and Liufang Sang and Guiguang Ding and Yuchen Guo and Xiaoming Jin},
  title = {{Grouping Attribute Recognition for Pedestrian with Joint Recurrent Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.24963/ijcai.2018/441},
}
Figure