Abstract
In many machine learning applications, particularly in clinical proteomics and bioinformatics, obtaining labeled data is a costly and time-intensive process. This paper presents an active learning approach for prototype-based models, specifically Generalized Relevance Learning Vector Quantization (GRLVQ). By introducing a formal definition of a margin for LVQ networks, the authors propose two selection strategies that prioritize data points near the decision boundary. Empirical results on breast cancer and proteomic datasets demonstrate that these active learning strategies can achieve comparable or superior classification performance using as little as 30% of the total available training data.
Introduction
The assumption that "more data is better" often fails in specialized domains where labeling requires expert intervention. In fields like cancer research, every labeled sample represents significant lab work and clinical expertise. Traditional supervised learning typically follows a "passive" paradigm, processing all available data regardless of its informational value. This paper explores an "active" alternative, where the model identifies and requests labels for the most informative samples.
While active learning has been successfully applied to global models like Support Vector Machines (SVMs), its application to local, competition-based architectures like Learning Vector Quantization (LVQ) remained relatively unexplored. LVQ models are valued for their interpretability, using prototypes to represent classes in the feature space.
Methodology
The researchers developed their framework using Generalized Relevance Learning Vector Quantization (GRLVQ). The core innovation is the formalization of a Margin for LVQ networks:
Margin = (Distance to the closest wrong prototype) - (Distance to the closest correct prototype)
A margin near zero indicates that a point lies close to the decision boundary, representing a high degree of uncertainty for the model. Conversely, a large positive margin suggests high confidence.
Two active selection strategies were proposed:
- Threshold Strategy: The model only requests labels for points where the margin falls below a predefined threshold ($\rho$).
- Probabilistic Strategy: The probability of selecting a point for training is inversely proportional to its margin, favoring points near the boundary.
Experimental Setup
The strategies were evaluated on three distinct datasets:
- WDBC (Wisconsin Breast Cancer): A classic UCI dataset with 569 samples and 30 features.
- Proteom1 & Proteom2: High-dimensional clinical proteomic datasets derived from mass spectrometry. Proteom1 contains 199 samples across 3 classes with 250 dimensions, while Proteom2 contains 737 samples and 148 dimensions.
Results and Discussion
The experimental results demonstrate a significant "less is more" effect. By focusing on low-margin points, the active learning strategies directly tighten the generalization bounds of the model, ensuring better performance on unseen data.
As shown in Table 1, the Threshold Strategy (Strategy 1) achieved 92% prediction accuracy on the WDBC dataset using only 30% of the data queries, compared to 90% accuracy for the standard model using 100% of the data. In the Proteom2 dataset, accuracy improved from 70% to 72% while ignoring nearly half of the training samples. Furthermore, the active strategies demonstrated faster convergence, reaching peak performance earlier in the training process by avoiding redundant, easy-to-classify points.
Conclusion
This research highlights the efficiency of margin-based active learning for LVQ networks. By selecting "better" data rather than simply "more" data, the proposed methods reduce the labeling burden in data-scarce environments. While the authors noted some sensitivity to parameter tuning and potential overfitting in very small datasets with the probabilistic strategy, the overall findings provide a robust framework for optimizing the data labeling process in high-stakes fields like clinical proteomics.
Summary updated