Active learning is the practice of asking a human to label only the examples a model is most unsure about. It breaks down in a familiar way. The examples that look most uncertain are often not informative but simply ambiguous, such as a blurred pixel, an occluded object, or an image that honestly sits between two classes. Labeling them teaches the model nothing. Current fixes for this either run a large teacher model over the whole pool to screen out noise (Active Data Curation Effectively Distills Large-Scale Multimodal Models) or apply ad hoc rules such as picking median-uncertainty clients (Privacy-preserving and Uncertainty-aware Federated Trajectory Prediction for Connected Autonomous Vehicles). We propose a cheaper signal built from two other ingredients. The first is training that deliberately jumps: large, abrupt learning-rate steps that knock the model out of one solution basin and into another. The second is a hyperbolic embedding space, a curved space whose volume grows exponentially away from the origin, like a tree. In this space, distance from the origin naturally encodes how specific, or deep in the class hierarchy, a prediction is.
The idea fits in two sentences. Train once with a schedule that hops between several distinct basins, and record where each unlabeled example lands in the hyperbolic space after each hop. Then query the examples whose landing points are far apart in hyperbolic distance. Ambiguous examples stay near the origin in every basin and are therefore close to themselves, while examples the model could learn but has not settled get committed to different branches by different basins and are therefore far apart.
Core Claim
The geometry does the filtering. In the Poincaré ball, a standard model of hyperbolic space, take two points at hyperbolic radii $r_u, r_v$ from the origin, separated by an angle $\theta$. For moderate-to-large radii their geodesic distance is approximately
$$d_H(u,v) \approx r_u + r_v + 2\log\sin(\theta/2),$$
which follows from the hyperbolic law of cosines. Disagreement in direction, meaning disagreement about which branch of the hierarchy an example belongs to, therefore costs a lot only when both placements are confident (large $r$). Disagreement between two near-origin placements costs almost nothing. We define the acquisition score of a pool example $x$ as its mean pairwise geodesic distance across $K$ basin snapshots,
$$E(x) = \frac{2}{K(K-1)} \sum_{k<l} d_H\big(z_k(x), z_l(x)\big),$$
where $z_k(x)$ is the example's embedding after the $k$-th hop. We claim three things. First, $E$ separates learnable-but-unsettled examples from ambiguous ones without a teacher model. Second, the separation requires the snapshots to come from genuinely different basins, which is what discontinuous training provides. Third, the hyperbolic metric does this better than the same disagreement measured in Euclidean space or as hierarchy-weighted softmax disagreement.
The proposal also takes a position on a contradiction in the literature. Hyperbolic Active Learning for Semantic Segmentation under Domain Shift (HALO) reports that hyperbolic radius tracks data scarcity, with rare classes pushed outward and a correlation of −0.899. By contrast, Asymmetric Multi-View Clustering with Hyperbolic Uncertainty Modeling and Learning the Predictability of the Future read large radius as confidence. Our hypothesis is that radius in a single model is ambiguous between these readings, and that consistency of placement across basins is what disambiguates them. A scarce class pushed outward along consistent directions is learned. One pushed outward along inconsistent directions is epistemic, meaning the uncertainty comes from missing data rather than from the example itself.
Research Program
Where the jumps come from. Special Properties of Gradient Descent with Large Learning Rates shows that large discrete steps, not gradient noise, are what carry gradient descent across barriers. Its "SGD with repeats" control kept the noise fixed while shrinking the steps, and those runs stayed trapped. Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent makes the matching point that continuous-flow models underestimate this hopping. Hop, skip, jump to Convergence shows that abrupt transitions between constant learning rates move large models between distinct convergence paths. These papers supply the mechanism. We reuse it as a cheap way to visit several basins within one training run, and none of them uses the visited basins as an uncertainty signal.
The hop schedule. Within each active-learning round we alternate a short high-constant-learning-rate phase, sized to leave the current basin, with a short low-rate settling phase. We snapshot the model at the end of each settling phase, for $K=5$ snapshots per round. Each new round also opens with a hop. This tests a secondary question raised by the gap analysis: whether a jump frees the model from a pre-acquisition minimum better than small-step fine-tuning does.
The embedding model. The embedding head follows HALO's Poincaré projection for segmentation and a hyperbolic classification layer from Hyperbolic Neural Networks++ for classification. We keep curvature fixed. For stability we borrow two pieces from Robust Hyperbolic Learning with Curvature-Aware Optimization: its Riemannian AdamW optimizer and its maximum-distance rescaling, which caps how close embeddings get to the ball's boundary. We deliberately do not try to "tunnel" near the boundary. A rescaling cap is a simpler, already-working fix for that problem.
The analysis. Beyond accuracy, we measure what radius encodes under each training loss, so the HALO/HAMC disagreement can be resolved empirically rather than argued.
Cost. The scoring cost is $K$ forward passes over the pool. That is comparable to Monte Carlo dropout, which estimates uncertainty by repeating forward passes with random units switched off (Wu et al., Tsymbalov et al.). It is far below the cost of a reference teacher, and the extra training cost is zero.
Baselines
The naive reference is uniform random sampling at the same budget. Everything else must beat it to justify the extra pool passes.
The strongest existing method is HALO, which scores each pixel by the product of hyperbolic radius and prediction entropy. It reports 74.5% mIoU on GTAV→Cityscapes at a 5% annotation budget, above the 71.9% fully supervised baseline and +3.3 mIoU over the prior best method, RIPU (Hyperbolic Active Learning for Semantic Segmentation under Domain Shift). It is the closest competitor because it already uses hyperbolic radius for acquisition, and it is the method most exposed to the failure we target: high entropy at large radius can mean a corrupted pixel. We will re-run HALO with its released configuration on the same backbone (DeepLab-v3+), the same budget schedule, and the same total training compute. Our method spends its compute inside the hop schedule rather than in extra training. Beating it means exceeding our re-run of HALO on mIoU at 5% budget across three seeds, and also matching or exceeding the published 74.5%. If our re-run falls short of 74.5, we report both numbers.
On classification there is no reported result found on Lacuna for active learning on CIFAR-100 under these protocols. We therefore compare against our own runs of three standard acquisition rules: predictive entropy; margin sampling as used by PixelPick, which queries the pixels where the top two class scores are closest; and Monte Carlo dropout variance. All three run with the same backbone, budget, and tuning grid of four learning rates and the same number of rounds.
Three ablations isolate the key ingredients. The first keeps the score but takes $K$ snapshots from an ordinary continuous cosine schedule, which isolates the discontinuity. The second computes the same pairwise disagreement with Euclidean embeddings and with softmax outputs weighted by tree distance between classes, which isolates the hyperbolic metric. The third uses radius alone, as HAMC does, and HALO's radius-times-entropy product inside our own training run, which isolates cross-basin consistency from single-model radius. An independent-seed ensemble of $K$ models with the same score serves as an upper reference that costs $K$ times the training compute.
Why not just…?
Why not just use an ensemble with standard epistemic disagreement, for example the mutual information between predictions and model identity? This is the most serious threat. Ensemble disagreement already separates epistemic from aleatoric uncertainty in principle. Our method differs on two axes: it gets its ensemble from one run via jumps, and it measures disagreement in a metric that discounts low-commitment placements. The independent-seed ensemble reference and the tree-weighted softmax ablation test both axes directly. If tree-weighted softmax disagreement over hop snapshots matches the hyperbolic score, then the geometry adds nothing, and we will report that the contribution reduces to cheap basin-hopping ensembles.
Why not just take checkpoints along a normal schedule? Checkpoints taken close together on a smooth path sit in one basin, so they should disagree little and mostly on genuinely noisy items, where predictions fluctuate. That is the opposite of what we want. The evidence that small steps fail to leave basins comes from Special Properties of Gradient Descent with Large Learning Rates, but that evidence concerns optimization, not acquisition. The continuous-checkpoint ablation settles the question. If the rank correlation between its scores and ours exceeds 0.9 and accuracy is unchanged, the discontinuity claim is false.
Why not just filter noise with a teacher, or with median-uncertainty selection? A teacher works (Active Data Curation Effectively Distills Large-Scale Multimodal Models), but it requires a stronger model of the target domain, which is exactly what is missing under synthetic-to-real shift. Median selection (Privacy-preserving and Uncertainty-aware Federated Trajectory Prediction for Connected Autonomous Vehicles) discards the high-uncertainty informative examples along with the noise. We include median-entropy selection as a baseline in the noise experiment to check whether this crude rule is already enough.
Proposed Experiments
The first experiment is a controlled noise test on CIFAR-100, chosen because it carries a two-level hierarchy of 20 superclasses over 100 classes. We seed the pool with 10% corrupted items: blends of two images from different superclasses, for which the oracle returns either label at random. This gives ground-truth aleatoric items. Over ten rounds of 1% budget each, we measure test accuracy and the fraction of queries spent on corrupted items. The decisive check is a strictly lower corrupted-query fraction than entropy, Monte Carlo dropout, and HALO-style scoring, with accuracy at matched budget not lower than the best baseline.
The second experiment is domain-shift segmentation on GTAV→Cityscapes, SYNTHIA→Cityscapes, and Cityscapes→ACDC. Cityscapes' category grouping (flat, construction, object, nature, sky, human, vehicle) supplies the hierarchy, and the comparison against HALO follows the protocol above.
The third experiment is the radius analysis. We report Spearman correlations of mean radius with class frequency, with injected noise, and with cross-basin consistency, under both HALO's loss and a plain hyperbolic classification loss.
The proposal fails cleanly in two cases. If continuous checkpoints match hop snapshots, the jumps do not matter. If Euclidean or tree-weighted softmax disagreement matches the hyperbolic score, the geometry does not matter.
References
- Active Data Curation Effectively Distills Large-Scale Multimodal Models
- Privacy-Preserving and Uncertainty-Aware Federated Trajectory Prediction for Connected Autonomous Vehicles
- Hyperbolic Active Learning for Semantic Segmentation under Domain Shift
- Asymmetric Multi-View Clustering with Hyperbolic Uncertainty Modeling
- Learning the Predictability of the Future
- Special Properties of Gradient Descent with Large Learning Rates
- Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent
- Hop, skip, jump to Convergence: Dynamics of Learning Rate Transitions for Improved Training of Large Language Models
- Hyperbolic Neural Networks++
- Robust Hyperbolic Learning with Curvature-Aware Optimization
- Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training
- Dropout-Based Active Learning for Regression
- All you need are a few pixels: semantic segmentation with PixelPick