Labeling every object category in every image is expensive, so a line of work trains multi-label classifiers from just one confirmed positive label per image. This setting is called single-positive multi-label learning. It works well in benchmarks where that one label is drawn uniformly at random from the objects present. Real annotators do not choose that way. They name the big, central, salient thing. When the single label is chosen by object size instead of uniformly, every leading method loses an average of 7.3 mean average precision (MAP) points on MS-COCO. The obvious fix today is to ask a large vision-language model, a generator that answers questions about images, to fill in the missing labels. Our concern is that this fix inherits the same bias. A teacher looking at the whole image also overlooks small, peripheral objects, so its errors line up with the annotator's omissions. Both sources then go silent on the same objects.
We propose a cheap, targeted alternative. A small calibration subset of images gets expensive teacher attention. On those images, the teacher must state a hypothesis about a missing category tied to a specific region of the image, such as "possible cup, lower-left near the hand." Each hypothesis is then checked by cropping that region and re-querying it. The verified extra positives serve two purposes. They become pseudo-labels, and they let us estimate how likely annotators were to report each kind of object. That estimate is a propensity model, and it corrects the training loss on the whole unverified dataset. The teacher's explanation is used as a checkable proposal of where to look, never as ground truth. Teacher compute is spent only where the propensity model says human reporting is weakest.
Core Claim
Under realistic reporting bias, the labels missing from single-positive data are the ones a global vision-language teacher is also most likely to miss. Pseudo-labeling with such a teacher therefore compounds the bias rather than removing it. Grounded verification on a small, bias-targeted subset should recover most of the lost MAP, concentrated on small and off-center objects. It should also cost far fewer teacher queries than the obvious alternatives, which are labeling every image or tiling every image into crops.
The claim breaks into three testable parts. First, teacher recall falls with object area in the same way annotator reporting does. Second, region-grounded verification flattens that curve. Third, a propensity correction fit on a few thousand verified images transfers to the full training set.
Prior Work and Where This Sits
The motivating evidence comes from Understanding Label Bias in Single Positive Multi-Label Learning. It simulates size, location, and semantic biases in which category gets reported on MS-COCO and shows large drops for the loss functions known as AN, ROLE, and LEM. It is purely diagnostic and proposes no fix; we supply one.
The theory explains why the failure is structural. Revisiting Pseudo-Label for Single-Positive Multi-Label Learning, which introduces the MIME method, proves that learnability requires every true positive class to have a non-zero chance of being the observed label. Severe size bias pushes that chance toward zero for tiny objects. Pseudo-label refinement based on the model's own confidence cannot recover them, because the model never sees evidence for them.
Can Class-Priors Help Single-Positive Multi-Label Learning?, which introduces CRISP, corrects for how often each class occurs, using positive-unlabeled learning, where the data contain confirmed positives plus unlabeled examples. Its correction is one number per class. Our propensity model depends on properties of each object instance, such as its size and position. That is exactly the dimension along which the bias acts.
Multi-Label Learning From Single Positive Labels supplies the ROLE estimator and the strong baseline of treating unobserved labels as negative with label smoothing (AN-LS). We reuse both as base learners.
The bias-inheritance concern is grounded in Multi-Label Knowledge Distillation. That paper finds that distillation from the whole image attends to dominant foreground categories and overlooks minor co-occurring ones. It treats this as a student-side flaw under full labels. We treat it as a teacher-side blind spot that lines up with annotator bias. The Curious Case of Hallucinations in Neural Machine Translation shows that distillation can amplify a teacher's errors rather than merely copy them. This is why we never train on unverified teacher claims.
On rationales, Okay, Let's Do This! Modeling Event Coreference with Generated Rationales and Knowledge Distillation distills free-text rationales into a student for event coreference. It documents that the student absorbs the teacher's confabulations and calls for a critic, without building one. QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models builds such a learned critic. It relies on repeated teacher sampling and roughly triples training time, and it is tested only on single-label tasks.
Our verifier is neither learned nor based on voting. It is a perceptual re-check of a specific region, possible only because image rationales can point at pixels. Unlike all three rationale papers, we do not distill the rationale text into the student at all. Its only job is to say where to look.
On cost, DISCO: Distilling Counterfactuals with Large Language Models shows that 75,000 filtered samples beat a million noisy ones, but it over-generates and then discards most output. CHIMERA: Compact Synthetic Data for Generalizable LLM Reasoning shows that compact, heavily verified data can suffice, at very high cost per sample. We take both lessons and push them further: we choose in advance which images deserve generation, instead of filtering afterward.
Research Program
The pipeline has three stages, plus a stopping rule.
Stage 1: base learner. We train a standard single-positive learner (AN-LS or ROLE) on the biased labels. This gives a cheap uncertainty signal for each image and class.
Stage 2: teacher queries on a calibration subset. We select a subset of roughly 2 to 10 percent of training images. For each image, an open-weight vision-language teacher receives the image and its one observed label. It is asked to propose additional categories that may be present, each with a bounding region and a one-sentence piece of evidence. Every proposed (class, region) pair is verified by re-querying the teacher, or a cheaper zero-shot image-text model, on the enlarged crop with a yes/no question. Only pairs that pass become verified positives. Inferences from context alone ("leash, therefore dog") are explicitly rejected unless the dog itself is confirmed in a crop. This blocks the spurious co-occurrence shortcut that context-based reasoning would otherwise introduce.
Stage 3: propensity correction. Let $y_c(x)\in\{0,1\}$ be whether class $c$ is truly present in image $x$, and let $s_c(x)\in\{0,1\}$ be whether it was the annotator's reported label. The quantity we need is the reporting propensity $e(c,x)=P(s_c=1\mid y_c=1,x)$. We model it with a small regressor on features of the verified region: its relative area, distance from the image center, and the class identity. The verified positives on the calibration subset provide the needed sample of present-but-unreported objects. Assuming reporting depends on $x$ only through these region features, the corrected positive term of the risk is
$$\hat R_+ = \frac{1}{N}\sum_{i=1}^{N}\sum_{c=1}^{L} \frac{s_c(x_i)}{\hat e(c,x_i)}\,\ell\big(f_c(x_i),1\big),$$
where $f_c$ is the classifier's score for class $c$, $\ell$ is binary cross-entropy, $N$ is the number of images, and $L$ is the number of classes. This is standard inverse-propensity weighting from positive-unlabeled learning. Images reported with a rarely chosen small object count more, standing in for the many unreported ones. Region features for images outside the calibration subset come from a class-agnostic box proposer.
Stopping rule. We grow the calibration subset in rounds. Each round prioritizes images where the base learner is uncertain and the current propensity estimate is low. Growth stops when the estimate of $\hat e$ stabilizes.
Baselines
The naive reference is AN-LS trained on size-biased single positives, run with the same backbone and schedule as everything else. On the LEM loss, Understanding Label Bias in Single Positive Multi-Label Learning reports 70.7 MAP under uniform selection and 61.2 MAP under size bias on MS-COCO. That gap is the target we aim to close. The upper bound is full-label binary cross-entropy. For context, ROLE reports 86.5% MAP against 86.7% for full labels on PASCAL VOC 2012, though only under uniform selection (Multi-Label Learning From Single Positive Labels).
The strongest existing method is CRISP, which reports 74.64% mAP on MS-COCO (Can Class-Priors Help Single-Positive Multi-Label Learning?). We found no reported result on Lacuna for CRISP under size-biased selection. We will therefore rerun CRISP on our bias splits with its released settings and tune its weight $\lambda$ on the same validation split we use. Beating CRISP means a higher full-label mAP under size bias and a smaller gap between uniform and size-biased selection, averaged over three seeds.
The most dangerous competitor is teacher pseudo-labeling of the whole image. The same teacher answers yes or no for all 80 categories on every training image, and the student trains on the result. We found no reported result on Lacuna for vision-language pseudo-labeling in this setting. We will match it to our method on total teacher calls and total GPU-hours. Beating it means higher mAP on small objects at equal or lower cost.
Two ablations isolate the key ingredients. The first keeps the pipeline but removes crop verification, so unverified proposals are accepted directly. This tests whether verification is what blocks inherited errors. The second keeps verification but replaces bias-targeted subset selection with random selection. This tests whether targeted allocation is what saves compute.
Why not just…?
Why not just pseudo-label every image with the vision-language teacher? This is the real threat, because modern teachers are strong at recognizing objects. Our hypothesis is that they share the annotators' bias toward large objects. Multi-Label Knowledge Distillation observed that global image models overlook minor co-occurring categories, though in a different role. We test this directly by plotting teacher recall against COCO object area, measured against full ground truth. If teacher recall is flat across sizes, the inheritance hypothesis fails and dense pseudo-labeling is the better answer. We would then report it as such and fall back to the cost argument alone.
Why not just tile each image into crops and ask about every class in every crop? Tiling also removes the size blind spot, and it is our strongest ablation. Its cost grows with tiles × classes × images, while ours grows only with the hypotheses proposed on a small subset. We compare at matched query budgets and at matched accuracy. If tiling the calibration subset alone matches our method, then the explanation step adds nothing beyond the propensity correction, and we will say so.
Why not just reweight by class frequency, as CRISP does? Size bias acts within a class: a small cup and a large cup are reported at different rates. A correction with one number per class cannot capture this. The diagnostic signature is our size-stratified mAP. If CRISP's gains on small objects match ours, the propensity model is unnecessary.
Proposed Experiments
All experiments use MS-COCO 2014 with the uniform, size, location, and semantic selection protocols of Understanding Label Bias in Single Positive Multi-Label Learning. We add PASCAL VOC 2012 and NUS-WIDE as transfer settings. NUS-WIDE has no boxes, so there we only measure whether a propensity model fit on COCO helps.
The metrics are full-label mAP; mAP split by small, medium, and large objects using COCO areas; pseudo-label precision and recall per size bin; overlap between proposed regions and true COCO boxes; and teacher calls and GPU-hours per point of recovered MAP.
Three checks decide the outcome. The first is whether teacher recall declines with object area. This determines whether the inheritance problem exists. The second is whether our method recovers at least half of the 7.3-point gap caused by size bias while using teacher calls on no more than 10 percent of images. The third is whether it beats both dense pseudo-labeling and tiling on small-object mAP at equal compute.
We also run one failure probe: we disable the rejection of context-only inferences. This measures how many spurious co-occurrence positives a rationale-trusting pipeline would admit. It is a direct measurement of the inheritance that Okay, Let's Do This! Modeling Event Coreference with Generated Rationales and Knowledge Distillation reported only qualitatively.
References
- Understanding Label Bias in Single Positive Multi-Label Learning
- Revisiting Pseudo-Label for Single-Positive Multi-Label Learning
- Can Class-Priors Help Single-Positive Multi-Label Learning?
- Multi-Label Learning From Single Positive Labels
- Multi-Label Knowledge Distillation
- The Curious Case of Hallucinations in Neural Machine Translation
- Okay, Let's Do This! Modeling Event Coreference with Generated Rationales and Knowledge Distillation
- QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models
- DISCO: Distilling Counterfactuals with Large Language Models
- CHIMERA: Compact Synthetic Data for Generalizable LLM Reasoning