Integral Control of Label Allocation for Worst-Group Error in Active Learning

Active learning is the practice of letting a model choose which unlabeled examples a human should label next. It usually picks whatever the model is most uncertain about right now. On data with rare subgroups, such as waterbirds photographed on land or blond men in CelebA, this instantaneous rule tends to stall. The model's error on the rare group stops falling well before the labeling budget runs out, because each round's picks are drawn back toward the abundant majority. Control engineers know this pattern. A controller that reacts only to the current error, called a proportional controller, settles with a persistent leftover error whenever something steadily pushes against it. The standard fix is an integral term, a running sum of past errors that keeps increasing its push until the error is gone.

This note proposes a simple outer loop for any active learner. At each round it measures per-group error on a fixed, randomly drawn validation set, keeps a leaky running sum of each group's excess error, and uses that sum to decide what share of the next labeling batch each group receives. The underlying acquisition function then picks examples within each share. What we intend to add beyond the familiar "integral term" is a stopping rule. The controller stops accumulating for a group when extra labels stop buying error reduction. That prevents the controller from spending the whole budget on a group whose errors are label noise rather than lack of data.

Core Claim

The argument that a proportional rule leaves a leftover error in active learning needs care, because labels already accumulate and one might expect that to act as an integrator. It does not. What drives a group's error under ordinary retraining is the group's share of the labeled pool, and that share behaves like a moving average of past allocations rather than a sum of them. Write $a_g(t)$ for the fraction of round $t$'s batch given to group $g$, $p_g(t)$ for group $g$'s fraction of the labeled pool, and $\rho_t = B/N_{t+1}$ for the batch size divided by the pool size after the round. Then

$$p_g(t+1) = (1-\rho_t)\,p_g(t) + \rho_t\, a_g(t).$$

Suppose the allocation is set by a proportional rule, $a_g = k\, e_g$, where $e_g$ is the group's excess error. At equilibrium the pool share needed to fix group $g$, call it $p_g^\*$, must equal $k\,e_g^\*$. The group therefore has to keep a nonzero error $e_g^\* = p_g^\*/k$ simply to justify its share. This is the same trap the file describes for P-controlled steering. An integral allocation $a_g(t) \propto \sum_{\tau \le t} \lambda^{t-\tau} e_g(\tau)$, with $\lambda$ a leak factor, can hold the needed allocation while the error is zero. The same equation shows a second effect: $\rho_t$ shrinks as the pool grows, so the system responds more and more slowly to allocation changes. Gains therefore have to be scaled with $1/\rho_t$, a technique control engineers call gain scheduling, or the controller becomes sluggish late in the budget. Our claim is that an integral allocation with gain scheduling and progress-gated anti-windup reaches a target worst-group accuracy with fewer labels than instantaneous-disparity or worst-group-greedy allocation. We further claim it does this without collapsing when one group has irreducible noise.

Relation to Prior Work

Three control-meets-learning papers supply the method but do not touch data selection. Activation Steering with a Feedback Controller shows that common LLM steering methods are proportional controllers across layers and adds integral and derivative terms. It operates only on inference-time activations. ControlVAE runs a PI loop on the KL weight during training on a fixed labeled dataset. It is the closest precedent for "a feedback loop around a training quantity," and its finding that closed-loop control beats open-loop annealing (8.81 versus 17.81 perplexity on Switchboard) motivates the setup here. We apply this idea to which data gets labeled, not to a loss weight. PIDformer places PID terms inside attention and is unrelated to acquisition.

The windup concern comes from control practice. Multi-level integral separation in DC motor speed control reduced step overshoot from 10.74% to about 0%. The leaky integrator in Direct Centroidal Control bounds how much past error can accumulate. Both switch integration off by error magnitude. We switch it off by learning progress, meaning the drop in validation error per label a group recently received. That condition matters in active learning, where large error can mean either "needs data" or "unlearnable."

Several active learning papers motivate or constrain the design. Dropout Disagreement selects hard minority examples in one shot and reports that on CelebA the selected set remains majority-dominated. It has no memory across rounds, and that missing memory is exactly what we add. Fair Active Learning mixes accuracy and fairness with a fixed weight or by duplicating points, which is a static or proportional policy. Agnostic Multi-Group Active Learning gives label-complexity bounds for multi-group disagreement methods. Our analysis will be compared to these bounds, not claimed to improve them. PriorAL and the auxiliary discriminator for infrastructure defects change the within-round scoring rule. Our loop sits above such rules and could wrap either one.

Critical Gap Between Generalization Error and Empirical Error in Active Learning constrains the design. It shows that errors measured on actively acquired data are off by 10–15%, while a fixed random validation set removes the bias. For this reason our feedback signal must come from a fixed random validation set with group labels, never from the acquired pool. Otherwise the controller would be regulating a biased sensor.

Research Program

The controller is built as follows. Each round, the model is retrained on the labeled pool and evaluated per group on a fixed random validation set of a few hundred points. We compute each group's excess error $e_g$ relative to a tolerance above the best group. The leaky integral state is updated, but only while that group's recent error drop per allocated label exceeds a threshold. Otherwise the state is frozen, and it decays if progress stays flat. The integral states are passed through a softmax to give batch shares, gains are scaled by $1/\rho_t$, and a base acquisition function (entropy or BALD) fills each share from the pool.

Group membership in the pool is unknown. It is predicted by a small classifier trained on the validation set's group labels, in the spirit of the one-vs-all discriminator from the infrastructure paper. Errors in this group predictor act as actuator error, and we will measure how much that costs. The validation labels count toward the budget in every comparison.

The analysis has two parts. For the moving-average plant above, with error assumed monotone in pool share, we will derive the offset left by proportional allocation and the conditions under which leaky integral allocation converges without oscillation. We will also study how the leak factor trades residual offset against the risk of windup.

Baselines

The naive reference points are random sampling and plain entropy or BALD acquisition, run with the same backbone, retraining schedule, batch size, total budget (validation labels included), and five seeds. On imbalanced CIFAR-10, the infrastructure paper reports that BALD and entropy improved minority recall by only 5% and 0.3% over random. We expect these baselines to plateau on worst-group accuracy.

The strongest existing method on the headline benchmarks is DD-DFR from Dropout Disagreement. It reports 91.6% worst-group accuracy on Waterbirds with 48 class annotations and 83.0% on CelebA with 472. The fully supervised DFR† reference reaches 91.8% with 599 annotations and 85.1% with 9933. We will reproduce DD-DFR with its published protocol and also give it the same group-labeled validation set our method uses, so it is not disadvantaged. Our method beats it if it reaches DD-DFR's worst-group accuracy with fewer total labels. On CelebA, the more demanding target is to close the gap to DFR†'s 85.1% within a few thousand labels.

For class imbalance we compare against PriorAL, which reports reaching target accuracy on merge-imbalanced CIFAR-100 with 10% of labels where the best baselines needed 20%. We will also wrap PriorAL's own scoring inside our allocator to test whether the two gains add up. For fairness we compare to FAL Nested-Append on Adult and COMPAS; no reported result was found on Lacuna for FAL in numeric form.

The key ablations share everything with our method except one ingredient each. The first is a proportional-only allocator, where shares follow current excess error; it isolates the integral term. The second is integral allocation without progress gating, which isolates anti-windup. The third is integral allocation without gain scheduling. The fourth is multiplicative-weights allocation, an exponentiated sum of past group errors like the online group weights used in robust training. That last method is itself an unleaky integral controller, so it separates "any memory" from our specific leak and gating choices. No reported result was found on Lacuna for multiplicative-weights allocation in active learning.

Why not just…?

Why not just give each group an equal or inverse-frequency share of every batch? Static balancing is an open-loop schedule. It fixes the allocation without knowing how hard each group is, so it over-labels easy minority groups and under-labels hard ones. ControlVAE's comparison against open-loop annealing is the closest evidence in the file that feedback beats fixed schedules. This baseline is still a real threat on two-group problems like Waterbirds, where the right share may happen to be near uniform. We include it, and we expect our advantage to appear mainly on CelebA and on settings with many groups of unequal difficulty.

Why not just always query the currently worst group? Worst-group-greedy is a bang-bang proportional rule. It switches its target whenever the ranking of groups flips, so it chatters between groups. It also has no way to stop when the worst group is worst because of label noise. The noisy-group stress test described below is designed to show this failure. If greedy matches our method on clean data, we will say so, and the claim then narrows to robustness under noise.

Why not just use multiplicative weights over groups, as in robust training? This is the most serious alternative, because it already has memory. We treat it as the central ablation rather than as a strawman. The only things our method adds on top of it are the leak, progress gating, and gain scheduling. If those do not improve labels-to-target or budget wasted on noisy groups, the honest conclusion is that multiplicative weights is sufficient for active learning allocation, and we would report that.

Why not just reweight the training loss instead of changing what gets labeled? Loss reweighting reuses the labels already collected. It cannot create coverage of a rare group's variations that were never sampled. DFR† needed far more labels than DD-DFR for similar accuracy, which suggests that choosing what to label is the scarce resource. We will run integral control on loss weights as a comparison arm to measure this directly.

Proposed Experiments

The main benchmarks are Waterbirds and CelebA for worst-group accuracy, and long-tailed CIFAR-10 and CIFAR-100 with classes treated as groups. Adult and COMPAS serve as a fairness check, with demographic parity gap as the controlled quantity. The metrics are labels needed to reach a target worst-group accuracy, area under the worst-group-accuracy-versus-labels curve, final residual gap to the best group, and the fraction of budget spent on a group after its progress has stalled.

The first decisive check is the predicted offset. The proportional allocator should settle with a nonzero worst-group gap that shrinks as its gain grows, roughly like $1/k$. The integral allocator's gap should approach the tolerance. If the proportional allocator shows no offset, the core premise fails.

The second decisive check is windup. We inject 30% label noise into one minority group of CelebA. Without gating, the integrator should pour labels into that group. With gating, its share should freeze while the other groups keep improving.

A third check uses predicted rather than true pool group labels, which measures how much the controller's advantage survives imperfect group assignment. A final check swaps the validation sensor for errors measured on the acquired pool, which should reproduce the degradation predicted by the Critical Gap paper.