Chest X-ray classifiers that look strong on rare diseases may be finding them partly by their companions. In multi-label radiology, a rare finding usually appears alongside common ones: a scan with several findings is more likely to carry one more. A model trained on such data can raise its score for a rare label because the usual co-findings are present, not because it sees the lesion. Recommender systems research has a name and a toolkit for this failure. There, a click is split into genuine interest and "conformity", meaning a user follows what is popular. The fix is to model the conformity path separately and remove it at prediction time. We propose the radiology analogue. For each finding, the classifier's score is split into a lesion-evidence part, computed from the image, and a co-occurrence-conformity part, computed from the model's beliefs about the other findings. The second part is then removed or re-calibrated at test time.
The point is not another re-weighting loss. Existing long-tail methods in medical imaging treat prevalence as one number per class. Classifier re-training, focal or asymmetric losses, and prior subtraction all work this way. Co-occurrence is a context-dependent prior, and it can hide a specific clinical failure: rare findings that appear without their usual companions get missed. Our core claim is that this "atypical-presentation gap" is large and measurable on the multi-label CXR-LT benchmark. We further claim that a classifier trained to separate evidence from co-occurrence context shrinks it, and that subtracting a co-occurrence prior after training, with no learned separation, cannot match this.
Core Claim and Positioning
Two recommender methods supply the mechanism. MACR models item popularity as a direct path to the ranking score. It trains an auxiliary branch for that path and subtracts its natural direct effect at inference, reporting large Hit Ratio gains (153% on matrix factorization, 241% on LightGCN) on popularity-neutral test sets. DICE keeps separate interest and conformity embeddings. Lacking labels for either cause, it mines "colliding" cases: a user who picks an unpopular item over a popular one must be driven by interest. We reuse both ideas, but swap the confounder. In MACR, the confounder is a global, per-item popularity. In our setting, it is a per-image, per-finding context prior conditioned on the other findings. The file notes that coarse, global treatment of popularity is an open limitation in the recommender literature itself.
The medical long-tail literature motivates the problem and constrains the design. Holste et al. show that imbalance damage on chest X-rays sits mostly in the classifier head. Decoupled head re-training (cRT) lifts NIH-CXR-LT tail accuracy from 1.7% to 30.0%. Their benchmark is single-label, so co-occurrence never arises. They also show that pixel-space MixUp destroys subtle signals such as pneumothorax lines. We therefore avoid any image blending, even though Balanced-MixUp works on fundus images. The CXR-LT challenge overview supplies the multi-label regime and its unsolved tail. The winning team reached 0.372 macro mAP using weighted asymmetric loss and high resolution, yet per-class AP still ran from 0.063 to 0.918. Top teams also lost on average 12.3% AP when scored against radiologist-verified labels.
The closest discriminative work is LADE. It frames long-tail recognition as entanglement with the marginal label distribution $p_s(y)$ and removes it in training and by post-compensated softmax. It handles only single-label natural images and only the marginal prior. Our target is the conditional prior $p(y_k \mid y_{-k})$, where $y_{-k}$ denotes the other findings on the same image. Two medical works are counterfactual in a different sense. Nie et al. disentangle anatomy from pathology, but for generating images, at minutes per sample. ACAT uses counterfactual saliency to fix spatial imbalance. Neither touches label-context priors in a classifier.
Research Program
The model is a standard chest X-ray backbone feeding two heads. The evidence head produces a logit $e_k(x)$ for each finding $k$ from image features. The context head produces $c_k$ from the model's own detached predicted probabilities for all other findings, $\hat{y}_{-k}$, plus a class bias. The context head is a small network that sees no pixels. During training the two combine additively, as in DICE's click model:
$$z_k(x) = e_k(x) + c_k(\hat{y}_{-k}),$$
and $z_k$ is trained with binary cross-entropy against the observed label.
On its own, this split is not identified, because the evidence head could still absorb context. We add two pressures. The first is a DICE-style collider step. Within each batch, we flag context-defying examples for each rare finding: positives whose context head predicts a low probability, and negatives whose context head predicts a high one. We train $e_k$ alone on these with an extra cross-entropy term. On such cases, only image evidence can explain the label. The second is a discrepancy penalty that discourages $e_k$ from being predictable from $\hat{y}_{-k}$ on ordinary examples. This is DICE's independence loss, re-implemented with a linear probe whose fit we penalize.
At inference we mirror MACR's direct-effect removal and LADE's prior swap:
$$z_k^{\text{test}}(x) = e_k(x) + \lambda \, c_k(\hat{y}_{-k}).$$
Setting $\lambda = 0$ asks what the score would be if co-findings had no direct influence. Setting $\lambda = 1$ recovers the trained model. $\lambda$ is chosen per class group on validation data.
The additive form is an assumption, flagged as a limitation in both DICE and MACR. A gated, non-linear combination is a secondary variant, tested only if the additive version works.
The analysis comes before the method. We will first measure the atypical-presentation gap for ordinary models. For each finding, CXR-LT test positives are split into "typical" (top tertile of training-set conditional prevalence given the other labels) and "atypical" (bottom tertile). We then compare recall at fixed specificity and per-stratum AP. If ordinary models show no gap, the premise fails. We would report that as a negative result and stop.
Baselines
The naive reference is a backbone trained with plain binary cross-entropy at the same resolution and schedule. It sets the floor and the size of the atypical-presentation gap.
The strongest existing method on this exact benchmark is the CXR-LT recipe of weighted asymmetric loss. The challenge overview reports 0.372 macro mAP for the winning team on the text-mined test set. That number came with a full challenge pipeline, including resolutions up to 1024 and domain pretraining. We will re-implement weighted asymmetric loss on our backbone, resolution and compute budget, and report both our re-implementation and the published 0.372 as context. Beating it means higher macro mAP and tail-group mAP than our re-implementation across three seeds, with the gold-standard gain at least as large as the text-mined gain. We will also run our heads on top of asymmetric loss to check whether the effects stack.
Decoupled head re-training (cRT) from Holste et al. is the strongest simple fix for head bias, at 30.0% tail accuracy on NIH-CXR-LT and 36.7% on MIMIC-CXR-LT. We adapt it to multi-label by re-training the head with a sampler that balances per-label positive counts. That sampler distorts co-occurrence, which is itself informative to report.
Marginal prior subtraction is post-compensated softmax from LADE, adapted to per-label log-odds. It removes the class-frequency path, which is exactly what MACR would remove. It is therefore the honest version of the obvious idea of applying MACR to radiology. We found no reported result on Lacuna for post-hoc prior adjustment on CXR-LT.
The decisive ablation is conditional prior subtraction without learned disentanglement. We fit $p(y_k \mid y_{-k})$ from the training label table and subtract its log-odds, evaluated at the model's predicted $\hat{y}_{-k}$, from an ERM model's logits. It tunes $\lambda$ the same way.
Further ablations remove the collider term, remove the discrepancy penalty, and feed the context head pooled image features instead of predicted labels. The last tests whether the label bottleneck is what gives the split its meaning. All methods share data splits, backbone, input resolution, epochs and a fixed budget of hyperparameter trials on the validation set.
Why not just…?
Why not just subtract a conditional co-occurrence prior after training? This is the real threat, and the conditional-prior ablation above is built to settle it. A model trained without separation has already mixed co-occurrence into its image features. Subtracting a prior from its logits would then double-correct typical cases while leaving the image-side shortcut in place. If the post-hoc version closes the atypical-presentation gap as well as the trained heads do, the training machinery is unnecessary. We would then report the cheap method as the contribution.
Why not just retrain the classifier head with balanced sampling? cRT is strong in single-label chest X-rays (Holste et al.), but it treats each class's prevalence as one number. In multi-label data, balancing one label's positives changes the frequencies of its co-findings. We expect cRT to raise tail recall mostly on typical cases. The stratified evaluation will show whether that holds.
Why not just use asymmetric or focal re-weighting, which won CXR-LT? Re-weighting changes how much each label's errors count, not what evidence the score rests on. The challenge's own spread of per-class AP (0.063 to 0.918) and its 12.3% gold-standard drop show that the winning recipe leaves the tail and label-shift problems open. Our method can run on top of it, so the question is whether the gains add.
Why not just generate synthetic rare-disease images? Nie et al. reach 7.24 FID with a disentangled generator, but need minutes of optimization per sample. Synthetic images also inherit whatever co-occurrence pattern the generator was conditioned on, so they need not remove the shortcut. We treat generation as complementary and out of scope.
Proposed Experiments
The primary benchmark is CXR-LT, with 26 findings, multi-label, and macro mAP as the headline metric. We report three readouts on it. The first is the text-mined test set. The second is the 406-image radiologist-verified gold set; it is small, so we give bootstrap confidence intervals. The third is a co-occurrence-neutral test set built by resampling test images so that pairwise label dependence is minimized, the analogue of the uniform test sets used by DICE and MACR. Our main diagnostic is the typical-versus-atypical recall gap for medium and tail findings.
The decisive checks are three. First, the gap must exist for ERM and asymmetric loss. Second, our model at $\lambda = 0$ should shrink it relative to both the marginal and conditional post-hoc baselines without lowering overall mAP by more than seed noise. Third, the improvement should persist, not reverse, on the gold set.
A negative control uses single-label NIH-CXR-LT and MIMIC-CXR-LT. There, the context head can only learn a class bias, so our method should roughly match marginal prior subtraction. A clear win there would mean the gain does not come from modelling co-occurrence. That would undercut the core claim, and we would report it as such.
Finally, for clinical face validity, we will ask whether evidence-head saliency on atypical tail positives falls on the lesion more often than ERM saliency does. This will be checked on a radiologist-reviewed subset of the gold set.
References
- Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System
- Disentangling User Interest and Conformity for Recommendation with Causal Embedding
- Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study
- Balanced-MixUp for Highly Imbalanced Medical Image Classification
- Towards long-tailed, multi-label disease classification from chest X-ray: Overview of the CXR-LT challenge
- Disentangling Label Distribution for Long-Tailed Visual Recognition
- Causal Disentanglement for Robust Long-tail Medical Image Generation
- ACAT: Adversarial Counterfactual Attention for Classification and Detection in Medical Imaging