End-to-End Probabilistic Label-Specific Feature Learning for Multi-Label Classification

Jun-Yi Hang, Min-Ling Zhang, Yanghe Feng, Xiaocheng Song

Abstract

Traditional multi-label classification (MLC) models often utilize a single set of features for all labels, failing to capture the unique characteristics of individual categories. While label-specific feature learning has emerged as a solution, existing methods often rely on decoupled, multi-stage pipelines that lack joint optimization. We propose PACA, an end-to-end framework that learns label-specific features and prototypes within a unified probabilistic latent space. By leveraging Normalizing Flows to model complex, multimodal label distributions, PACA adaptively captures label characteristics. Experimental results across 14 benchmark datasets demonstrate that PACA significantly outperforms state-of-the-art methods.

Introduction

In multi-label classification, an instance can simultaneously belong to multiple categories. For example, an image might be labeled as both "sunset" and "beach." The visual features relevant to identifying a sunset (e.g., orange hues, low sun) differ significantly from those identifying a beach (e.g., sand, waves). Standard MLC models that use a shared feature representation for all labels often struggle with this diversity.

To address this, researchers developed label-specific features, typically using prototype-based approaches like LIFT. However, these methods usually involve a rigid three-stage process: clustering to find prototypes, feature transformation based on distances, and finally training a classifier. Because these stages are decoupled, the prototypes are not optimized for the final classification task.

The PACA Framework

PACA (End-to-End Probabilistic lAbel-speCific feAture learning) overcomes these limitations by integrating prototype acquisition and feature learning into a single differentiable neural network.

Resolving Contradictory Forces

A key insight of PACA is the necessity of multi-prototype learning. In a shared latent space, a single prototype per label can create "contradictory forces." As shown in Figure 1, two data points might need to be pulled together for one label but pushed apart for another. By allowing multiple prototypes (represented as peaks in a probability distribution), PACA resolves these conflicts, allowing for a more flexible and accurate representation.

Comparison of single-prototype (a) vs. multi-prototype (b) learning.

Probabilistic Latent Metric Space

PACA utilizes a Variational Autoencoder (VAE) framework to map instances into a latent space. Instead of fixed centroids, PACA models each label's positive and negative distributions using Neural Autoregressive Flows (NAF). This allows the model to learn complex, multimodal distributions that can adapt to the underlying data complexity without pre-defining the number of clusters.

The PACA framework architecture.

Label-specific features $\phi_j(x)$ are generated by calculating the KL divergence between an instance’s embedding $q_\phi(z|x)$ and the learned distributions for each label $j$. This distance-based feature representation is then fed into a label-specific classifier $g_j$.

Experimental Evaluation

The authors evaluated PACA on 14 benchmark datasets across diverse domains, including multimedia, text, biology, and video. PACA was compared against six state-of-the-art methods, including traditional approaches like ML-KNN and deep learning models like MPVAE.

Key Results

  • Performance: PACA achieved the best performance in 81% of cases across all evaluation metrics, including Average Precision and Hamming Loss.
  • Statistical Significance: Wilcoxon signed-ranks tests confirmed that PACA significantly outperforms deep baselines, validating the advantage of explicit label-specific feature learning.
  • Ablation Studies: Removing the multi-prototype capability or the label correlation regularizer led to significant performance drops, confirming the importance of these components.

Conclusion

PACA represents a significant step forward in multi-label classification by unifying label-specific feature learning and prototype acquisition into an end-to-end probabilistic framework. By moving away from rigid centroids to flexible Normalizing Flows, PACA effectively captures the multimodal nature of real-world labels. Future work will explore instance-specific prototypes to further refine the model's ability to capture fine-grained semantic nuances.

Summary updated

Abstract

Label-specific features serve as an effective strategy to learn from multi-label data with tailored features accounting for the distinct discriminative properties of each class label. Existing prototype-based label-specific feature transformation approaches work in a three-stage framework, where prototype acquisition, label-specific feature generation and classification model induction are performed independently. Intuitively, this separate framework is suboptimal due to its decoupling nature. In this paper, we make a first attempt towards a unified framework for prototype-based label-specific feature transformation, where the prototypes and the label-specific features are directly optimized for classification. To instantiate it, we propose modelling the prototypes probabilistically by the normalizing flows, which possess adaptive prototypical complexity to fully capture the underlying properties of each class label and allow for scalable stochastic optimization. Then, a label correlation regularized probabilistic latent metric space is constructed via jointly learning the prototypes and the metric-based label-specific features for classification. Comprehensive experiments on 14 benchmark data sets show that our approach outperforms the state-of-the-art counterparts.
BibTeX
@inproceedings{hang2022end,
  author = {Jun-Yi Hang and Min-Ling Zhang and Yanghe Feng and Xiaocheng Song},
  title = {{End-to-End Probabilistic Label-Specific Feature Learning for Multi-Label Classification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i6.20641},
}
Figure