Active Learning (AL) is often framed as a "free lunch" for data-starved researchers. By intelligently selecting only the most informative samples for human annotation, frameworks like Active Teacher and ActiveST can reach full supervised performance using as little as 3% to 40% of the total dataset.
However, a recurring theme across recent research is that this reduction in human effort comes at a steep price: a massive increase in computational overhead. The "efficiency" of active learning is a balancing act between human hours and GPU hours.
The Iterative Retraining Bottleneck
The core mechanism of active learning is a loop: the model is trained on current data, it "queries" the unlabeled pool for the most uncertain samples, a human labels them, and the model is updated.
In traditional supervised learning, you train the model once. In active learning, as noted by Peng Mi and colleagues in their work on Active Teacher, the training cost scales linearly with the number of active learning iterations. If you query the oracle 10 times, you might end up training the model 10 times. For large-scale tasks like object detection on MS-COCO, this iterative retraining can become a significant bottleneck, potentially negating the financial benefits of reduced human labor.
This overhead isn't just limited to deep learning. Even in stochastic systems, Martin Tappler and Bernhard K. Aichernig found that L*-based learning for MDPs requires frequent hypothesis re-evaluation, leading to execution times substantially longer than passive learning.
The Scoring and Extraction Burden
Beyond retraining, the system must decide which samples to label. This requires scoring every single item in the unlabeled pool—often millions of samples—using the current model.
Dittaya Wanvarie and Hiroya Takamura highlight this in sequence labeling tasks, where frequent retraining on partially annotated data significantly increases machine processing time. Similarly, in video-based person re-identification, Baisheng Lai and Xiaojin Gong point out that iterative frameworks must re-extract features and propagate labels across the entire dataset multiple times, creating a massive computational footprint compared to a single-pass supervised approach.
Balancing the Total Cost of Ownership
If the goal of active learning is to save money or time, we must look at the "Total Cost of Ownership" (TCO) of a label. Hang Qiu and Ramesh Govindan introduced MCAL to address exactly this. They argue that traditional AL only optimizes for accuracy, ignoring the marginal cost of GPU hours. By jointly optimizing for human labor and machine compute, they show that the most "informative" sample isn't always the best one to pick if the cost to retrain the model to learn from it is too high.
In some cases, AL might even be the least efficient way to improve a model. Lars Schmidt-Thieme and Thorben Werner suggest in recent analysis that while AL provides a final performance "squeeze," the high cost of repeated training cycles makes it difficult to scale compared to one-time costs like unsupervised pretraining or data augmentation.
Toward Computational Efficiency
Researchers are actively seeking ways to mitigate these costs. One promising direction is Online or Incremental Learning.
In O-MedAL, Asim Smailagic and Alex Gaudio propose an incremental training strategy for medical imaging. By only processing new samples during backpropagation, they cut training computations by 67%. However, they warn of a "wall-clock" trap: the cumulative time spent on frequent feature extraction and validation cycles at every iteration can still exceed the time of a standard, single-pass training run.
Another approach is Batching, as discussed by Stephen Mussmann and Percy Liang in their theoretical work. While selecting labels in large batches reduces the number of retraining cycles, it often deviates from the sequential selection assumptions that make active learning theoretically optimal, creating a tension between computational speed and label efficiency.