Abstract
In this paper, we propose a novel and interpretable grey-box ensemble using a self-labeled approach for semi-supervised classification problems. The prospective grey-box ensembles a more interpretable white-box model with a black-box technique. This scheme could guide the comparatively data expensive white-box component with the results from the more accurate black-box part. We evaluate the proposal in an inductive learning setting showing good performance in partially labeled datasets.
Introduction
In machine learning, a persistent tension exists between model performance and interpretability. High-accuracy "black-box" models, such as Random Forests or Neural Networks, often lack transparency, making it difficult to understand the reasoning behind their predictions. Conversely, "white-box" models like Decision Trees are easily auditable but typically require substantial labeled data to achieve competitive accuracy.
This challenge is particularly acute in semi-supervised learning environments, where human-labeled data is scarce and expensive, while unlabeled data is abundant. Most existing semi-supervised methods prioritize accuracy over interpretability. This paper introduces a "Grey-Box" ensemble approach designed to bridge this gap, providing a model that is both accurate in data-scarce scenarios and interpretable for domain experts.
The Grey-Box Architecture
The proposed Grey-Box ensemble utilizes a "self-labeled" strategy to leverage unlabeled data. The architecture operates in three distinct phases:
- Black-Box Training: A high-performance black-box model (e.g., Random Forest) is trained on the initial limited set of labeled data.
- Synthetic Labeling: This trained black-box model acts as a "teacher," predicting labels for the pool of unlabeled data to create a larger synthetic dataset.
- White-Box Training: A white-box model (e.g., a J48 Decision Tree) is then trained on the combination of the original human-labeled data and the synthetic labels generated by the black-box.
By using the black-box as an intermediate labeler, the white-box model—which is often "data-hungry"—can learn from a much larger dataset. The final output is a white-box model that can be represented as a set of human-readable "if-then" rules, but one that has inherited the pattern-recognition strengths of the black-box.
Experimental Evaluation
The researchers validated the Grey-Box approach using 121 benchmark datasets from the UCI Machine Learning Repository. Additionally, they conducted a case study in bioinformatics within the BRIDGEIRIS project, focusing on the pathogenesis classification of Arrhythmia Syndromes. This real-world application involved over 23,000 variants, where manual clinician classification is prohibitively expensive.
Performance Analysis
The evaluation utilized the Kappa statistic, a robust metric for imbalanced datasets. Results indicated that the Grey-Box model significantly outperforms standard white-box models (like J48) when labeled data is limited.
As shown in the performance curves, the Grey-Box model (GBM_RFJ48) consistently sits between the pure black-box (Random Forest) and the pure white-box (J48). The advantage is most pronounced when labeled data constitutes between 5% and 70% of the total dataset. While the black-box maintains a slight edge in raw accuracy, the Grey-Box provides the critical benefit of interpretability, which is essential in high-stakes fields like medicine.
Conclusion
The Grey-Box ensemble offers a viable solution for semi-supervised classification where both performance and transparency are required. By guiding a white-box model with black-box predictions, the system overcomes the limitations of data scarcity. Future work will focus on deeper integration into clinical workflows and exploring alternative pairings of black-box and white-box components, such as Bayesian Networks or Fuzzy Cognitive Maps, to further optimize the balance between accuracy and interpretability.
Summary updated