Abstract
Weak-label learning presents a significant challenge in multi-label classification, where training instances often lack a complete set of ground-truth labels. This paper introduces LCForest (Label Complement cascade Forest), a novel tree-ensemble approach that leverages a label complement mechanism and frequency estimation to iteratively recover missing labels. By treating missing labels as latent variables within a deep forest architecture, LCForest effectively addresses label correlation and class imbalance issues without the computational overhead of deep neural networks.
Introduction
In standard multi-label learning, training instances are assumed to be perfectly tagged. However, real-world annotators often miss labels due to fatigue or the sheer volume of categories, leading to weak-label learning. In this scenario, a "0" in the label matrix is ambiguous—it could represent a true negative or an unknown positive. This ambiguity creates three primary challenges: broken label correlations, the "Positive and Unlabeled" (PU) learning problem where models are overwhelmed by false negatives, and exacerbated class imbalance.
The LCForest Architecture
To address these issues, the authors propose LCForest (Label Complement cascade Forest). Unlike Deep Neural Networks (DNNs) that rely on backpropagation, LCForest builds on the "Deep Forest" architecture, utilizing layers of random forests to learn hierarchical representations.
The core innovation of LCForest lies in its three-part "secret sauce":
- Label Complement Structure: Each layer performs internal cross-validation. Predictions from the layer's forests are used to "fill in" missing labels in the training set for subsequent layers.
- Label Frequency Estimation (TIcE): To prevent the model from "hallucinating" too many labels, the TIcE technique estimates the true frequency of labels in the population.
- Complement Flag Mechanism: Using the TIcE estimates, the model sets an upper bound on label recovery. Once a class reaches its estimated limit, a "flag" stops further label addition to prevent over-fitting to noise.
Experimental Evaluation
The model was evaluated on four diverse datasets: Yeast (genetics), TMC (aviation safety), Scene (images), and Medical (NLP). To simulate weak-label conditions, the researchers randomly removed 20% to 50% of the relevant labels.
Key Results and Insights
LCForest consistently outperformed state-of-the-art methods, including low-rank matrix methods like WELL and deep belief networks like DBPNN.
As shown in the results table, LCForest maintains superior performance even as the "Incomplete Label Ratio" increases. For instance, on the Yeast dataset, while competitors' Macro-F1 scores plummeted as more labels were removed, LCForest remained relatively stable. This robustness is attributed to the model's ability to "clean" its own training data as it progresses through deeper layers, bringing the label matrix closer to the true ground truth.
Conclusion
LCForest demonstrates that "deep" learning does not strictly require differentiability or backpropagation. By treating missing labels as latent variables and utilizing a self-correcting training process, LCForest provides a robust and efficient solution for weak-label learning. Future work may explore multi-grain scanning to enhance performance on spatial or sequential data.
Summary updated