Conditional Restricted Boltzmann Machines for Multi-label Learning with Incomplete Labels

Xin Li, Feipeng Zhao, Yuhong GuoAISTATS 2015 · OpenReview

Abstract

Standard multi-label learning (MLL) methods typically operate under the "Complete Label Assumption," where every relevant tag for a training instance is assumed to be present. However, in real-world scenarios, labels are often incomplete due to annotator fatigue or lack of expertise. This paper proposes a Conditional Restricted Boltzmann Machine (CRBM) framework designed to handle missing labels by modeling high-order label dependencies and integrating auxiliary semantic knowledge from Wikipedia. Experimental results across four diverse datasets demonstrate that the CRBM significantly outperforms existing state-of-the-art methods in both label recovery and predictive performance.

Introduction

Multi-label learning involves assigning multiple relevant labels to a single instance, such as an image containing "sky," "road," and "car." While powerful, most MLL algorithms are sensitive to missing labels in the training set. When a positive label is missing, models often treat it as a true negative, leading to "negative prediction patterns" that degrade accuracy. To address this, we need models that can simultaneously impute missing labels and learn the underlying predictive patterns.

The CRBM Framework

The authors introduce a three-layer Conditional Restricted Boltzmann Machine (CRBM) to capture the complex relationships between features and labels.

The three-layer CRBM architecture consisting of input features (x), labels (y), and a latent layer (h).

Unlike traditional models that only consider simple pairwise label correlations, the CRBM utilizes a hidden layer ($h$) to capture high-order dependencies. This allows the model to understand groups of labels that frequently co-occur. Furthermore, the model incorporates Explicit Semantic Analysis (ESA) to leverage external knowledge from Wikipedia. This provides a "common sense" prior that helps guide the recovery of missing labels even when training data is sparse.

Training and Label Recovery

The training process employs a Viterbi-style Expectation-Maximization (EM) algorithm:

  1. M-step (Model Learning): Updates model parameters using Contrastive Divergence (CD-1) based on the current best guess of the labels.
  2. E-step (Label Recovery): Recovers missing labels by maximizing the joint probability of observed and missing labels, regularized by the Wikipedia-based semantic priors.

Experimental Evaluation

The CRBM was tested on four diverse datasets: Corel5K (images), Mediamill (video), CLEF2010 (images), and Delicious (text). To simulate real-world conditions, label missing rates were varied from 10% to 70%.

Micro-F1 performance of CRBM compared to LCML, MLRGL, and FastTag across different label missing rates on four datasets.

Key Results

As shown in the performance plots, the CRBM consistently maintains higher Micro-F1 scores compared to competitors like LCML and FastTag, especially as the label missing rate increases.

  • Superior Recovery: On the Corel5K dataset with 30% missing labels, the model achieved a 94.1% recovery accuracy.
  • Robustness: Even at a 70% missing rate, the CRBM remains effective, outperforming other methods across Macro-F1, Micro-F1, and AUC metrics.
  • The Wikipedia Advantage: Ablation studies confirmed that integrating external semantic knowledge significantly boosts performance over models using only local dataset statistics.

Conclusion and Future Work

By combining high-order dependency modeling with external semantic knowledge, the CRBM offers a robust solution for multi-label learning with incomplete data. While the model introduces some computational overhead due to the RBM architecture, its ability to "fill in the blanks" makes it highly valuable for real-world applications. Future work may explore deeper architectures, such as Deep Belief Networks, to capture even more complex label structures.

Summary updated

Abstract

Standard multi-label learning methods assume fully labeled training data. This assumption however is impractical in many application domains where labels are difficult to collect and missing labels...
BibTeX
@inproceedings{li2015conditional,
  author = {Xin Li and Feipeng Zhao and Yuhong Guo},
  title = {{Conditional Restricted Boltzmann Machines for Multi-label Learning with Incomplete Labels}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2015},
  url = {https://openreview.net/forum?id=OLC8sJX4pC},
}
Figure