Abstract
Although various methods have been proposed for pedestrian attribute recognition, most studies follow the same feature learning mechanism, i.e., learning a shared pedestrian image feature to classify multiple attributes. However, this mechanism leads to low-confidence predictions and non-robustness of the model in the inference stage. In this paper, we investigate why this is the case. We mathematically discover that the central cause is that the optimal shared feature cannot maintain high similarities with multiple classifiers simultaneously in the context of minimizing classification loss. In addition, this feature learning mechanism ignores the spatial and semantic distinctions between different attributes. To address these limitations, we propose a novel disentangled attribute feature learning (DAFL) framework to learn a disentangled feature for each attribute, which exploits the semantic and spatial characteristics of attributes. The framework mainly consists of learnable semantic queries, a cascaded semantic-spatial cross-attention (SSCA) module, and a group attention merging (GAM) module. Specifically, based on learnable semantic queries, the cascaded SSCA module iteratively enhances the spatial localization of attribute-related regions and aggregates region features into multiple disentangled attribute features, used for classification and updating learnable semantic queries. The GAM module splits attributes into groups based on spatial distribution and utilizes reliable group attention to supervise query attention maps. Experiments on PETA, RAPv1, PA100k, and RAPv2 show that the proposed method performs favorably against state-of-the-art methods.
The Orthogonality Bottleneck in Multi-Attribute Learning
Pedestrian Attribute Recognition (PAR) involves identifying multiple traits—such as gender, clothing type, and accessories—from a single image. Traditionally, researchers use a mechanism termed OFMA (One-shared-Feature-for-Multiple-Attributes). However, this approach suffers from a fundamental geometric flaw.
As the number of attributes increases, the optimal shared feature vector mathematically converges toward a 90-degree angle (orthogonality) relative to individual attribute classifiers. When a feature vector is nearly orthogonal to the decision boundary, the model's predictions become "low-confidence." Because the feature sits right on the boundary, even minor noise—such as a slight change in pose or lighting—can flip the prediction, leading to poor robustness during inference.
Disentangled Attribute Feature Learning (DAFL)
To overcome the limitations of shared features, we propose the Disentangled Attribute Feature Learning (DAFL) framework. This framework shifts from OFMA to an OFOA (One-specific-Feature-for-One-Attribute) mechanism, ensuring that each attribute has a dedicated representation that can maintain high similarity with its respective classifier.
Key Components
- Learnable Semantic Queries: The model maintains a set of queries ($Q$) for each attribute. These queries learn unique semantic characteristics across the dataset, acting as "search terms" to find relevant information in the image.
- Cascaded Semantic-Spatial Cross-Attention (SSCA): This module uses the queries to probe the image feature map. It iteratively refines its focus to locate specific spatial regions (e.g., the head for a "hat") and aggregates those regional features into a disentangled representation.
- Group Attention Merging (GAM): Since many attributes share spatial locations (e.g., "hat" and "glasses"), attributes are grouped by spatial distribution. This "group memory" provides stable supervision for the attention mechanism, particularly helping the model learn rare attributes with limited training data.
- Semantic Triplet Loss: This loss function ensures that features for the same attribute remain similar across different individuals while keeping features for different attributes distinct in the embedding space.
Experimental Results
DAFL was evaluated on four major pedestrian attribute benchmarks: PETA, RAPv1, PA100k, and RAPv2. The method achieved State-of-the-Art (SOTA) performance across all datasets, with a notable improvement in the mean Accuracy (mA) metric. For instance, on the PA100k dataset, DAFL achieved 83.54% mA, a significant jump over previous benchmarks.
Beyond accuracy, the disentangled approach provides superior robustness. Qualitative analysis shows that while baseline models often fail when a pedestrian's pose changes, DAFL maintains high-confidence, correct predictions because its attribute-specific features are geometrically further from the decision boundaries.
Conclusion
By mathematically identifying the orthogonality bottleneck in shared feature learning, this work provides a theoretical foundation for disentangled representations in multi-label tasks. The DAFL framework effectively exploits the consistent spatial structure of humans to learn robust, attribute-specific features, setting a new standard for pedestrian attribute recognition.
Summary updated