Learning from Weak-Label Data: A Deep Forest Expedition

Qian-Wei Wang, Liang Yang, Yufeng Li

Abstract

Weak-label learning presents a significant challenge in multi-label classification, where training instances often lack a complete set of ground-truth labels. This paper introduces LCForest (Label Complement cascade Forest), a novel tree-ensemble approach that leverages a label complement mechanism and frequency estimation to iteratively recover missing labels. By treating missing labels as latent variables within a deep forest architecture, LCForest effectively addresses label correlation and class imbalance issues without the computational overhead of deep neural networks.

Introduction

In standard multi-label learning, training instances are assumed to be perfectly tagged. However, real-world annotators often miss labels due to fatigue or the sheer volume of categories, leading to weak-label learning. In this scenario, a "0" in the label matrix is ambiguous—it could represent a true negative or an unknown positive. This ambiguity creates three primary challenges: broken label correlations, the "Positive and Unlabeled" (PU) learning problem where models are overwhelmed by false negatives, and exacerbated class imbalance.

The LCForest Architecture

To address these issues, the authors propose LCForest (Label Complement cascade Forest). Unlike Deep Neural Networks (DNNs) that rely on backpropagation, LCForest builds on the "Deep Forest" architecture, utilizing layers of random forests to learn hierarchical representations.

The detailed LCForest architecture featuring the label complement mechanism and complement flags across layers.

The core innovation of LCForest lies in its three-part "secret sauce":

  • Label Complement Structure: Each layer performs internal cross-validation. Predictions from the layer's forests are used to "fill in" missing labels in the training set for subsequent layers.
  • Label Frequency Estimation (TIcE): To prevent the model from "hallucinating" too many labels, the TIcE technique estimates the true frequency of labels in the population.
  • Complement Flag Mechanism: Using the TIcE estimates, the model sets an upper bound on label recovery. Once a class reaches its estimated limit, a "flag" stops further label addition to prevent over-fitting to noise.

Experimental Evaluation

The model was evaluated on four diverse datasets: Yeast (genetics), TMC (aviation safety), Scene (images), and Medical (NLP). To simulate weak-label conditions, the researchers randomly removed 20% to 50% of the relevant labels.

Key Results and Insights

LCForest consistently outperformed state-of-the-art methods, including low-rank matrix methods like WELL and deep belief networks like DBPNN.

Performance comparison of LCForest against baseline methods across various incomplete label ratios.

As shown in the results table, LCForest maintains superior performance even as the "Incomplete Label Ratio" increases. For instance, on the Yeast dataset, while competitors' Macro-F1 scores plummeted as more labels were removed, LCForest remained relatively stable. This robustness is attributed to the model's ability to "clean" its own training data as it progresses through deeper layers, bringing the label matrix closer to the true ground truth.

Conclusion

LCForest demonstrates that "deep" learning does not strictly require differentiability or backpropagation. By treating missing labels as latent variables and utilizing a self-correcting training process, LCForest provides a robust and efficient solution for weak-label learning. Future work may explore multi-grain scanning to enhance performance on spatial or sequential data.

Summary updated

Abstract

Weak-label learning deals with the problem where each training example is associated with multiple ground-truth labels simultaneously but only partially provided. This circumstance is frequently encountered when the number of classes is very large or when there exists a large ambiguity between class labels, and significantly influences the performance of multi-label learning. In this paper, we propose LCForest, which is the first tree ensemble based deep learning method for weak-label learning. Rather than formulating the problem as a regularized framework, we employ the recently proposed cascade forest structure, which processes information layer-by-layer, and endow it with the ability of exploiting from weak-label data by a concise and highly efficient label complement structure. Specifically, in each layer, the label vector of each instance from testing-fold is modified with the predictions of random forests trained with the corresponding training-fold. Since the ground-truth label matrix is inaccessible, we can not estimate the performance via cross-validation directly. In order to control the growth of cascade forest, we adopt label frequency estimation and the complement flag mechanism. Experiments show that the proposed LCForest method compares favorably against the existing state-of-the-art multi-label and weak-label learning methods.
BibTeX
@inproceedings{wang2020learning,
  author = {Qian-Wei Wang and Liang Yang and Yufeng Li},
  title = {{Learning from Weak-Label Data: A Deep Forest Expedition}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i04.6092},
}
Figure