Active learning operates on an intuitive premise: rather than labeling data at random, a machine learning model should selectively query the most informative examples to maximize performance under a strict annotation budget. In a perfectly balanced world, standard acquisition functions—such as querying samples with the highest predictive uncertainty or maximizing geometric diversity—work remarkably well.
However, real-world data is rarely balanced. When applied to long-tailed distributions or datasets containing rare target classes, standard active learning frameworks frequently collapse. Instead of accelerating data efficiency, they introduce sampling biases that reinforce the majority class, leaving the model blind to critical minority categories.
Understanding why active learning fails under class imbalance requires looking at how skewed data distorts a model's internal representation of uncertainty, and how researchers are redesigning acquisition loops to break these destructive feedback loops.
The Failure Modes of Standard Acquisition
When a dataset is heavily imbalanced, standard active learning heuristics suffer from three distinct structural failure modes: the uncertainty trap, path-dependent boundary refinement, and the budget paradox.
The Uncertainty Trap
Uncertainty-based sampling assumes that a model’s high entropy or low prediction confidence signals a valuable learning opportunity. In an imbalanced setting, this assumption breaks down. Because the majority class dominates the candidate pool, the model naturally develops a strong prior bias toward it.
As Bonaventure F. P. Dossou demonstrates in A Study of Acquisition Functions for Medical Imaging Deep Active Learning, this bias causes the model to become "certainly wrong" about rare classes. The acquisition function misinterprets the model's systemic neglect of the minority class as certainty, leading it to prioritize majority-class samples to incrementally stabilize overall accuracy.
Furthermore, in tasks like object or animal detection, Benjamin Kellenberger et al. show in Half a Percent of Labels is Enough: Efficient Animal Detection in UAV Imagery Using Deep CNNs and Active Learning that uncertainty sampling frequently wastes the labeling budget on ambiguous background noise rather than locating scarce positive instances.
Path Dependence and Boundary Traps
Standard active learning algorithms are highly path-dependent. When a decision boundary is initially drawn, it is heavily skewed toward the majority class due to the scarcity of minority examples. Shyam Nuggehalli notes in Improved Algorithm for Deep Active Learning under Imbalance via Optimal Separation that this skew causes the active learning loop to repeatedly sample near a flawed, majority-biased boundary.
As a result, the model enters a failure mode described by Pietro Lesci et al. in AnchorAL: Computationally Efficient Active Learning for Large and Imbalanced Datasets: the algorithm over-refines known, localized decision spaces while failing to discover entirely new, isolated clusters of minority-class data. If the initial random pool fails to capture a rare class, the active learning loop may never search the space required to find it.
The Budget Paradox
A natural intuition might suggest that increasing the total labeling budget would eventually force the model to look at rare classes. However, Marc Schachtsiek et al. uncover a paradox in Class Balanced Dynamic Acquisition for Domain Adaptive Semantic Segmentation using Active Learning: increasing the budget can actually exacerbate class imbalance. Because dominant classes generate a higher absolute volume of uncertainty across a large dataset, traditional acquisition functions continue to over-sample from frequent categories, yielding a labeled training set that is even more skewed than the underlying ground truth distribution.
[Imbalanced Data Pool] ──> [Model Develops Majority Bias] ──> [Uncertainty Hijacked by Majority Class]
▲ │
└─────────────────── [Query More Majority Samples] ───────────────────┘
Designing Imbalance-Robust Acquisition
To overcome these limitations, recent research focuses on explicitly decoupling sample selection from naive model uncertainty, shifting instead toward rareness-aware metrics, submodular optimization, and dynamic batch updates.
Targeting Rarity Directly
Rather than relying on general uncertainty as a proxy for information, several frameworks explicitly estimate the probability that an unlabeled sample belongs to a rare class. For instance, Vasudha Varadarajan et al. introduce a Probability-of-Rare-Class (PRC) strategy in Transfer and Active Learning for Dissonance Detection: Addressing the Rare-Class Challenge, which prioritizes the discovery of scarce positive examples over general data diversity. Similarly, Lile Cai et al. implement a rareness-aware sampling mechanism in Exploring Active Learning for Semiconductor Defect Segmentation to prevent the selection process from favoring common background structures over critical anomalies.
Submodular Information Measures
A more mathematically unified approach involves using submodular functions to balance the tension between global data diversity and targeted relevance. Suraj Kothawade et al. demonstrate in SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarios that submodular information measures can be formulated to specifically target rare classes while penalizing redundancy. By optimizing these functions, the acquisition loop naturally resists the gravity of the majority class, ensuring that the selected batch contains a representative mixture of the entire target label space.
Rarity as an Anomaly
Another powerful intuition is to treat the identification of tail-class examples as an unsupervised anomaly detection problem. Because a deep neural network trained on imbalanced data develops highly optimized activation patterns for head classes, tail-class examples produce distinct, out-of-distribution feature representations.
Gursimran Singh et al. exploit this behavior in Mining Minority-class Examples With Uncertainty Estimates by using a one-class autoencoder to flag these anomalous activation patterns. This allows the system to actively mine minority-class examples from massive unlabeled pools, bypassing the model's inherent classification bias.
The Necessity of Inter-Batch Dynamics
Fixing the acquisition function is only half the battle; the dynamics of batch selection also present a hidden trap. If an algorithm selects a large batch of data points simultaneously based on a static model state, a class-balanced acquisition function will often select highly homogeneous, redundant samples from the minority class.
To solve this, Zhixin Huang et al. propose AL-FaMoUS in Active Learning with Fast Model Updates and Class-Balanced Selection for Imbalanced Datasets. The framework integrates fast, on-the-fly updates to the model's final layer during the batch selection process. By updating the model's state as each sample is chosen, the acquisition function immediately recognizes when it has acquired sufficient information about a specific minority sub-cluster, forcing subsequent queries to maintain intra-batch diversity.