Abstract
This paper presents Triplet AL, a novel active learning framework designed to enhance model performance in data-scarce environments. By leveraging a triplet network to map data into a metric embedding space, the authors introduce the "Top-Two-Margin" metric to identify high-confidence unlabeled samples. These samples are then pseudo-labeled and integrated into the training set. Experimental evaluations on STL-10 and CIFAR-10 benchmarks demonstrate that Triplet AL consistently outperforms traditional active learning baselines across multiple architectures, including LeNet-5, ResNet-152, and DenseNet-121.
Introduction
Deep learning models typically require vast amounts of labeled data to achieve high performance. However, the process of manual labeling is often prohibitively expensive and time-consuming. Active Learning (AL) seeks to mitigate this by strategically selecting the most informative unlabeled samples for labeling.
Traditional AL methods often rely on a classification model’s final softmax output to estimate uncertainty. Unfortunately, softmax scores can be poorly calibrated, leading to unreliable sample selection. This paper proposes a more robust alternative using metric learning, specifically a triplet network, to measure sample confidence more accurately for pseudo-labeling in a semi-supervised setting.
Methodology
The proposed Triplet AL framework utilizes a dual-model architecture consisting of a standard classification model ($M_1$) and a Triplet Network ($M_2$). The Triplet Network is trained using triplet loss to ensure that samples from the same class are clustered together in an embedding space while samples from different classes are pushed apart.
The Top-Two-Margin Metric
To select samples for pseudo-labeling, the system first calculates the mean embedding (class center) for each class using the initial labeled dataset. For any unlabeled sample $u_i$, the triplet network generates an embedding, and the algorithm calculates the distance to the nearest class center ($D_{i1}$) and the second nearest center ($D_{i2}$).
The confidence metric, or Margin, is defined as: $$\text{Margin} = D_{i2} - D_{i1}$$
Unlike standard active learning which prioritizes the most uncertain samples (smallest margin) for human labeling, Triplet AL selects samples with the largest margin. These high-confidence samples are assigned a pseudo-label based on their nearest center and added to the training set for iterative retraining.
Experimental Results
The researchers evaluated Triplet AL on the STL-10 and CIFAR-10 datasets. The method was tested against several baselines, including Random Sampling, Least Certainty, BALD, and DeepFool AL.
Key findings include:
- Consistent Superiority: Triplet AL (indicated by the red diamond line in performance plots) consistently outperformed all other methods as more samples were added.
- Significant Gains in Small Models: The most dramatic improvements were seen in LeNet-5, with an accuracy jump of 14.55% on STL-10 and 6.29% on CIFAR-10.
- Scalability to Deep Architectures: The method also provided steady gains for ResNet-152 and DenseNet-121, typically ranging from 1.5% to 2.1%.
Discussion and Future Work
The success of Triplet AL suggests that distances in a learned metric space provide a more reliable indicator of class membership than raw classification probabilities. By allowing the model to act as its own "Oracle" through pseudo-labeling high-confidence samples, the training set can grow effectively with minimal human intervention.
However, the authors note that maintaining two separate models increases computational overhead. Future research will focus on multi-task learning to unify classification and embedding into a single model. Additionally, strategies to mitigate "confirmation bias" from potential pseudo-label noise and improving scalability for datasets with a high number of classes remain important areas for exploration.
Summary updated