Medical image classifiers often learn shortcuts. A shortcut is a non-medical cue that happens to predict the label in the training data. A pneumothorax detector, for example, may learn to spot the chest tube used to treat the condition rather than the collapsed lung itself. Separately, there is strong pressure to run these classifiers on cheap bedside or handheld hardware. The most aggressive way to do that is binarization: every weight and activation becomes a single bit, so multiplications turn into bitwise operations. Nobody has checked what binarization does to shortcut reliance in medical imaging. The two failure modes plausibly compound. Binarization destroys fine-grained, low-contrast information first, and subtle pathology is exactly that kind of signal. Radiopaque tubes and skin rulers are the opposite: large and high-contrast. A one-bit network may therefore keep the shortcut and lose the disease.
We propose to measure this directly, at the one-bit boundary, on medical benchmarks with known shortcuts. We then fix it with a training objective that treats performance on shortcut-absent patients as a hard constraint rather than one more weighted loss term. In two sentences: compressing a medical classifier to one bit probably makes it lean harder on artifacts like chest tubes, in a way that a debiased teacher model cannot undo by itself. We test this, and we train binary networks under an explicit constraint on the shortcut-absent subgroup, whose single knob is a clinically interpretable target instead of a fragile loss weight.
Core Claim
The closest evidence comes from Less Precise Can Be More Reliable. That study evaluated vision-language models at 8, 6 and 4 bits on natural images. It found that quantization acts like a spectral low-pass filter, which improves calibration and corruption robustness, but it also makes models more reliant on spurious backgrounds on the Counter-Animal benchmark. It did not study one-bit networks, medical acquisition artifacts, or any training-time remedy. A second line of work cuts the other way. Sagawa et al. showed that wider networks worsen worst-group error because they memorize minority exceptions. That would suggest capacity-starved binary networks might rely less on shortcuts. Our first claim is empirical and could fail. On shortcut-confounded medical benchmarks, binarization widens the gap between shortcut-present and shortcut-absent patients well beyond what full precision or 8-bit integer (INT8) models show. We decompose that gap into two parts. The first is "inherited" reliance, which arrives through distillation from the full-precision teacher. The second is "induced" reliance, which appears even when the teacher is debiased, because the one-bit student cannot reproduce the teacher's subtle features.
Our second claim is methodological. Induced reliance is best handled by constrained optimization, not by reweighting. We train the binary student to solve
$$\min_\theta \; \mathcal{L}_{\text{task}}(\theta) + \mathcal{L}_{\text{distill}}(\theta) \quad \text{s.t.} \quad \mathcal{L}_{\text{absent}}(\theta) \le \epsilon .$$
Here $\theta$ are the binary network's latent weights, and $\mathcal{L}_{\text{task}}$ is cross-entropy on all training images. $\mathcal{L}_{\text{distill}}$ is a feature-alignment loss to a debiased full-precision teacher. $\mathcal{L}_{\text{absent}}$ is cross-entropy restricted to diseased and healthy images where the shortcut cue is absent, for example pneumothorax without a chest tube. $\epsilon$ is the target loss on that subgroup, set relative to the teacher's own value on it. A Lagrange multiplier on the constraint is updated by dual ascent. As a result, the robust subgroup receives gradient weight only when the student is actually failing it, and the user specifies a target, not a weight. This directly targets two open problems from the loss-balancing literature. MGDA-UB picks an arbitrary point on the trade-off frontier between objectives with no way to prioritize one objective. GradNorm moves the sensitivity into its asymmetry parameter rather than removing it.
Research Program
The work has three parts.
The first is an audit. We take a ResNet-18 backbone and produce four versions of the same classifier: full precision, INT8 quantization-aware trained, one-bit with standard distillation, and one-bit with instance-aware thresholds. We evaluate all four on subgroups defined by the presence or absence of the shortcut cue. For one-bit training we reuse DIR-Net. DIR-Net retains information in binary networks through entropy-maximizing binarization, a two-stage gradient estimator, and distillation from a full-precision teacher, and it reports 66.5% ImageNet top-1 for ResNet-18. We also reuse INSTA-BNN. INSTA-BNN shifts each image's binarization threshold using the expected cube of the input activation, and it reports 68.5% for ResNet-18. Neither method has been tested on medical images.
Subgroups are defined from cue labels, not from saliency maps. Saporta et al. showed that Grad-CAM fails expert localization on chest X-rays, so we do not treat attention maps as evidence. The audit also includes the latent-clustering check from Oakden-Rayner et al., who introduced hidden stratification and showed that pneumothorax AUROC fell from 0.94 with a chest drain to 0.77 without one. The clustering check asks whether one-bit features cluster by device more strongly than full-precision features do.
The second part is a mechanism probe specific to instance-aware binarization. INSTA-BNN computes its threshold statistic over the whole spatial map. A bright, large chest tube could therefore dominate that statistic and push faint lung-edge responses below the threshold. Using the bounding boxes and masks available in SIIM-ACR, as used by Saab et al., we measure what fraction of the cubic statistic comes from device pixels versus pathology pixels in each layer. We also test whether excluding device pixels from the statistic at analysis time restores the binarized pathology response. If it does, we have a concrete, hardware-relevant explanation for induced shortcut reliance. That explanation would motivate a robust threshold statistic, such as a clipped or median-based proxy, as a secondary method.
The third part is the constrained trainer described above. It requires shortcut-absent labels only during training. We obtain them cheaply with the "Pick-and-Predict" step of SpurCon: an expert labels ten examples per cue group, and a frozen BiomedCLIP model propagates the labels by prototype similarity. The large foundation model is used only offline, so the deployed artifact remains a pure binary network. This addresses SpurCon's own limitation that it cannot run on edge hardware, and it drops SpurCon's need for patient-ID pairing. The teacher is debiased first, using SpurCon's weighted contrastive loss where patient IDs exist and Saab et al.'s box or mask supervision otherwise. Inherited and induced reliance are then separable by construction.
Baselines
The naive reference point is a full-precision ResNet-18 trained by plain cross-entropy, together with its one-bit counterpart trained with DIR-Net and distilled from that same undebiased teacher. Together they show the default outcome a practitioner would get. Saab et al. report a Robust AUROC of 0.46 on SIIM-ACR for binary-label full-precision training, and SpurCon reports a baseline worst-group accuracy of 35.6% on CheXpert and 0.0% on ISIC 2020.
The strongest full-precision debiasing references are SpurCon, which reports worst-group accuracy rising to 73.0% on CheXpert and 65.0% on ISIC 2020, and Saab et al.'s segmentation-supervised model, which reports a Robust AUROC of 0.82 on SIIM-ACR. These numbers define the ceiling a binary model should approach. They are not targets we expect to exceed. No reported result was found on Lacuna for any binary neural network on SIIM-ACR, CheXpert, ISIC 2020 or ChestX-ray14, so all binary numbers will be produced here.
The strongest binary competitor is "debias the teacher, then binarize with standard DIR-Net distillation and a fixed loss weight." This competitor uses the same debiased teacher as our method, the same data, the same number of training epochs, and the same cue labels. Its distillation weight and an optional subgroup-reweighting coefficient are tuned by a search with the same number of trials we give to $\epsilon$.
We also run balancers that do not need a target value. GradNorm, with its asymmetry parameter tuned under the same budget, treats task, distillation and absent-subgroup losses as three tasks. MGDA-UB treats them as three objectives on the penultimate representation. Group distributionally robust optimization, which upweights whichever subgroup currently has the highest loss, is a further baseline, as is Sagawa et al.'s majority subsampling. The INT8 model serves as the precision reference.
The key ablation removes only the constraint, keeping our debiased teacher and labels but turning the dual variable into a fixed weight. A second ablation uses an undebiased teacher with the constraint, which isolates the induced component. Beating the strongest binary competitor means higher worst-group accuracy or Robust AUROC on held-out data, with confidence intervals from five seeds that do not overlap. Aggregate AUROC must be no more than one point lower, and the variance of the robust metric across the tuning sweep must be visibly smaller.
Why not just…?
Why not just debias the teacher and binarize normally? This is the real threat, and the proposal is built so that it is the main comparison. DIR-Net's distillation will carry over a debiased teacher's features where the student can represent them. The Bouguerra et al. finding suggests that low precision specifically erases fine discriminative foreground detail, which would leave the student to rediscover the high-contrast cue through cross-entropy. If the debiased-teacher baseline already closes the subgroup gap, the induced component is near zero. We would then report "debias before binarize" as the finding and drop the constrained trainer as unnecessary. The decomposition experiment decides which case holds.
Why not just reweight or subsample groups during binary training? Sagawa et al. showed that loss reweighting fails in models that drive training loss to zero, while subsampling works but discards over 96% of the data. Binary networks often do not interpolate, so reweighting might actually work here. That makes it a fair baseline, not a straw man. The constrained method differs because the subgroup weight responds to whether the target is met, rather than being fixed in advance. The sensitivity sweep shows whether this matters.
Why not just use INT8, which runs on standard runtimes? Q-YOLOP-style 8-bit training gives roughly 4x storage savings and is broadly supported, and the Bouguerra et al. study suggests 8-bit models are mostly well behaved. If an 8-bit model fits the target device's memory and latency budget, binarization is not justified. We therefore report latency and memory for both on a Raspberry Pi 3B, the device DIR-Net used, and we state the operating points where only the one-bit model fits. The audit's INT8 arm also tells us whether the shortcut problem is specific to one bit.
Why not just use GradNorm or MGDA-UB? Both balance gradients without knowing which objective is clinically primary. MGDA-UB is defined on a single shared representation, which binary networks with real-valued skip paths complicate. GradNorm still has a parameter to tune. We run both under equal tuning budgets. If either matches the constrained trainer's worst-group performance with comparable sensitivity, our method offers no advantage, and we will say so.
Proposed Experiments
We use SIIM-ACR pneumothorax with chest-tube confounding and report Saab et al.'s Robust AUROC. We use CheXpert pneumothorax confounded by support devices and ISIC 2020 melanoma confounded by rulers, and report worst-group accuracy using SpurCon's group definitions. We use ChestX-ray14, split by chest drain, for drain-present versus drain-absent AUROC.
The first decisive check is the audit. For each dataset we compare the shortcut-absent minus shortcut-present gap across full precision, INT8 and the two one-bit models. Sagawa's capacity argument predicts that binarization shrinks the gap; our hypothesis predicts that it widens it. A widening of at least five AUROC points or worst-group accuracy points, consistent across seeds, supports our claim. No widening would refute it, and we would report that result as a substantive finding in its own right.
The second check is the decomposition: undebiased versus debiased teacher, with and without the constraint, at one bit. This shows how much of the gap is inherited and how much is induced.
The third check is the mechanism probe, which measures the device-pixel share of INSTA-BNN's threshold statistic on SIIM-ACR.
The fourth check is a sensitivity study. We sweep $\epsilon$ across a range and sweep fixed subgroup weights across a matched range, then plot worst-group performance against aggregate AUROC. The constrained trainer is useful only if its curve is both higher and flatter.
All binary models are deployed with a bitwise inference kernel on a Raspberry Pi 3B, alongside a TFLite INT8 build. We report measured latency, peak activation memory and model size rather than FLOP counts, because theoretical savings often fail to translate into wall-clock gains on edge processors.
References
- Less Precise Can Be More Reliable: A Systematic Evaluation of Quantization’s Impact on VLMs Beyond Accuracy
- An Investigation of Why Overparameterization Exacerbates Spurious Correlations
- Multi-Task Learning as Multi-Objective Optimization
- GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
- Distribution-Sensitive Information Retention for Accurate Binary Neural Network
- INSTA-BNN: Binary Neural Network with INSTAnce-aware Threshold
- Benchmarking saliency methods for chest X-ray interpretation
- Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging
- Reducing Reliance on Spurious Features in Medical Image Classification with Spatial Specificity
- SpurCon: Weighted Supervised Contrastive Learning for Mitigating Spurious Cues in Medical Imaging