A Solution to Co-occurrence Bias: Attributes Disentanglement via Mutual Information Minimization for Pedestrian Attribute Recognition

Yibo Zhou, Hai-Miao Hu, Jinzuo Yu, Zhenbo Xu, Weiqing Lu, Yuran CaoarXiv 2023 · DOI · arXiv

Introduction

Pedestrian Attribute Recognition (PAR) is a fundamental task in computer vision, involving the identification of specific traits—such as age, clothing style, or accessories—from surveillance imagery. While current state-of-the-art models have achieved high accuracy by modeling the relationships between attributes, they often fall victim to "co-occurrence bias." This bias occurs when a model learns to rely on unreliable statistical correlations found in training data (e.g., assuming a person in a skirt is always female) rather than actual visual evidence.

This paper proposes a novel framework to address this issue by "disentangling" attribute features. By minimizing the mutual information between different attribute representations, the authors force the model to recognize each trait independently, leading to significantly better performance and robustness in real-world, "zero-shot" scenarios where training-set correlations no longer apply.

The Problem: The Trap of Context

In many PAR datasets, certain attributes frequently appear together. If a training set was filmed exclusively in summer, a model might learn that "ShortSleeve" and "Trouser" never co-occur. When deployed in a different climate or season, the model fails because it has "hallucinated" an interdependency that does not exist in reality.

While some researchers attempt to use Graph Neural Networks (GNNs) to model these relationships, this paper argues that such relationships are too volatile across different scenes and cultures to be reliable. The goal is to move toward human-like reasoning: a human recognizes a hat by looking at the hat, regardless of the wearer's gender or other clothing.

Proposed Methodology

The authors introduce a framework for Attribute Disentanglement via Mutual Information Minimization. The core objective is to ensure that the feature used to predict one specific attribute contains zero information about any other attribute.

1. Feature Decomposition

The model takes a global feature vector and decomposes it into $C$ attribute-specific components. Each component is responsible for representing a single attribute, such as "glasses" or "backpack."

2. Posterior-Invariant Learning

The researchers derive a training objective where the predicted probability of an attribute remains invariant even when the features of other attributes are varied. This mathematically enforces the independence of each attribute's representation.

3. Norm-Direction Separated Interpolation (NDSI)

To practically implement disentanglement, the model needs to see a wide variety of feature combinations. The authors propose NDSI, which separates a feature's direction (encoding semantics and style) from its norm (encoding uncertainty and quality). By mixing these components independently during training, the model explores a more realistic and diverse range of feature variations than standard linear interpolation allows.

4. Efficient MixUp-based Training

To avoid the computational burden of running multiple separate inferences per image, the authors developed a "plug-and-play" strategy inspired by MixUp. This allows the model to learn disentanglement in a single pass, ensuring the method remains lightweight and easy to deploy.

Experimental Results

The proposed method was evaluated on major PAR benchmarks, including PA100k, PETA, and RAP. Crucially, the authors tested on "zero-shot" (zs) versions of these datasets (PETAzs and RAPzs), where there is no overlap in identities between training and testing sets. These settings are a more accurate reflection of real-world deployment.

Key performance gains include:

  • PETAzs: Mean Accuracy (mA) improved from a baseline of 71.62% to 78.54% using a ConvNeXt backbone.
  • RAPzs: Mean Accuracy (mA) improved from 72.32% to 80.18%.

Beyond accuracy, the method is highly efficient. Unlike complex graph architectures, this approach only adds a single fully-connected (FC) layer during the inference stage.

Conclusion and Future Work

By forcing neural networks to disentangle attributes, the authors provide a robust solution to the problem of co-occurrence bias. The results demonstrate that independent feature learning is superior to context-heavy modeling when dealing with biased datasets.

While the method significantly reduces bias, the authors note that some residual correlations persist. Future work may involve more sophisticated causal modeling to distinguish between universal causal relationships and coincidental statistical correlations. This disentanglement approach holds promise not only for pedestrian recognition but for any multi-label classification task where dataset bias is a concern.

Summary updated

Abstract

Recent studies on pedestrian attribute recognition progress with either explicit or implicit modeling of the co-occurrence among attributes. Considering that this known a prior is highly variable and unforeseeable regarding the specific scenarios, we show that current methods can actually suffer in generalizing such fitted attributes interdependencies onto scenes or identities off the dataset distribution, resulting in the underlined bias of attributes co-occurrence. To render models robust in realistic scenes, we propose the attributes-disentangled feature learning to ensure the recognition of an attribute not inferring on the existence of others, and which is sequentially formulated as a problem of mutual information minimization. Rooting from it, practical strategies are devised to efficiently decouple attributes, which substantially improve the baseline and establish state-of-the-art performance on realistic datasets like PETAzs and RAPzs. Code is released on https://github.com/SDret/A-Solution-to-Co-occurence-Bias-in-Pedestrian-Attribute-Recognition.
BibTeX
@misc{zhou2023solution,
  author = {Yibo Zhou and Hai-Miao Hu and Jinzuo Yu and Zhenbo Xu and Weiqing Lu and Yuran Cao},
  title = {{A Solution to Co-occurrence Bias: Attributes Disentanglement via Mutual Information Minimization for Pedestrian Attribute Recognition}},
  year = {2023},
  eprint = {2307.15252},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2307.15252},
}
Figure