Screening programs read huge numbers of normal exams to find a handful of cancers. One proposed fix is an AI "rule-out" filter that confidently clears clearly normal exams so radiologists read fewer of them. The hard part is not ranking exams. It is choosing the cutoff below which an exam is cleared, because that cutoff fixes how many cancers slip through. Cutoffs tuned on one scanner or imaging type drift when moved to another. Re-tuning them requires many confirmed cancers in the new setting, and cancers are rare: under 1% of screens in the mammography cohort studied by Bernstein et al.. This proposal tests a simple idea borrowed from how radiologists read. Score each patient by how different her left breast looks from her right, and express that difference relative to how different healthy pairs look on the same kind of scanner. Most of what a scanner does to an image affects both sides alike, so it should cancel in the difference. If it does, a cutoff set once on 2D mammography should keep the same miss rate on breast MRI or chest CT, with no new cancer labels needed.
The second ingredient addresses the obvious blind spot. A left-versus-right comparison cannot see disease that is present on both sides. That failure is exactly the one a rule-out filter must never make. We therefore add a small gate built from clinical record variables and absolute (single-side) image scores. The gate decides which patients are even eligible for asymmetry-based clearance. Clinical variables like age and cancer history are also the same no matter which scanner produced the image, so the gate should transfer as well. The two ingredients play different roles. Bilateral comparison is the scoring mechanism, the negative predictive value (the probability that a cleared exam is truly cancer-free) is the objective, clinical context is a safety constraint, and transfer across imaging types is the test.
Core Claim
Let $z_L$ and $z_R$ be a model's feature vectors for the left and right breast of one patient in imaging type $m$. Let $\mu_m$ and $\sigma_m$ be the mean and spread of the asymmetry distance $d(z_L, z_R)$ among presumed-healthy pairs in that imaging type. The rule-out statistic is the standardized asymmetry
$$s_m = \frac{d(z_L, z_R) - \mu_m}{\sigma_m},$$
and an exam is cleared when $s_m < \tau$ and the clinical gate permits it. Because screening prevalence is below 1%, $\mu_m$ and $\sigma_m$ can be estimated from unlabeled target-modality screens: contamination by the rare cancers barely moves them, and a robust estimator like median and interquartile range bounds the effect.
The claim has three parts, all testable. First, the false omission rate, $\text{FOR} = \text{FN}/(\text{FN}+\text{TN}) = 1-\text{NPV}$, at a fixed $\tau$ chosen on mammography transfers to new imaging types more faithfully for $s_m$ than for an equally standardized single-side score. Second, this holds because shared patient and scanner factors cancel in the difference. Third, the clinical gate removes most of the residual misses, which we expect to concentrate in bilateral and diffuse disease. Put simply: within-patient comparison turns the rule-out threshold into something closer to a modality-independent quantity.
The closest existing work does pieces of this. Yala et al. and Bernstein et al. set rule-out thresholds on mammography alone and name threshold brittleness across sites as open. Dual-Res Tandem Mamba-3D compares flipped left and right breast volumes on non-contrast chest CT, but only for classification, with no triage or clinical data. GLAM aligns the two views of the same breast and explicitly leaves left-right alignment to future work. None asks whether a bilateral score makes the operating point portable.
Research Program
What motivates the idea. Two lines of work supply the motivation. The classical argument of Liu et al. (2001) quantified left-right deviations in brain scans as a lesion detector that needs no population atlas. Chen and Liu (2023) made the parallel point for paired hips: the paired structure is a natural baseline. Both rest on the same observation. Healthy left-right differences within a person are small compared with differences between people. Our hypothesis extends this from "between people" to "between scanners."
What constrains the design. Chen and Liu (2023) warn that symmetric disease removes the healthy baseline. Fineide et al. (2022) show that some physiological asymmetry is itself informative. These constraints are why the gate exists, and why we report the miss rate separately for bilateral cancers.
Scoring model. In each imaging type we isolate the two breast regions. For MRI we use nnU-Net side masks of the kind released with Divide and Conquer. For chest CT we use flipped sub-volumes as in Dual-Res Tandem Mamba-3D. For mammography we use mirrored side images. Each side is encoded by a shared-weight Siamese encoder, a network that processes both sides with the same weights, following Tiulpin et al. (2017). To tolerate imperfect left-right correspondence, the comparison uses cross-side attention in the style of SEAN rather than pixel subtraction. We also test a counterfactual variant: following the Bilateral Asymmetry Guided Counterfactual Generating Network, the contralateral breast conditions a prediction of what the target breast's features would look like if healthy.
Per-modality encoders are trained on whatever diagnostic labels each imaging type has. Labels are used to learn a good ranking. They are never used in the target modality to set the cutoff.
Threshold and training objective. On the source modality (2D mammography) we pick $\tau$ to meet a target FOR. We also try co-optimizing encoder and threshold with the smooth triage surrogate of Okati et al., who prove that a model trained for full automation wastes capacity on cases a human will handle anyway. Their method is reused as-is; our contribution is what happens to the resulting operating point under transfer.
Clinical gate. The gate is a small model, logistic or a shallow tree, over modality-independent variables. These are age, personal and family cancer history, prior contralateral cancer, and known high-risk status where recorded. It also takes the maximum of the two absolute single-side scores. It is trained on the source modality to flag patients at elevated risk of bilateral or diffuse disease, who are then never auto-cleared. We keep the gate deliberately small. Large fused models tend to lean on strong clinical shortcuts, a known failure of late fusion, and missing record fields across centers would make a large model brittle.
Analysis of why it works or fails. We decompose features into components shared between sides and components that differ between sides. We then measure how much of the between-modality shift in the single-side score lives in the shared component. Two failure modes would break the cancellation argument. One is lateralized acquisition effects, such as MRI coil intensity bias that differs by side. The other is positioning differences between left and right mammographic views. We measure both directly on healthy pairs.
Baselines
Naive reference: no triage. The do-nothing reference is reading every exam, with zero caseload reduction and the historical radiologist miss rate. Alongside it we report Yala et al.'s style of policy: a single-image classifier with a high-sensitivity cutoff picked on a validation set. They reported ruling out 19.3% of mammograms, with sensitivity 90.6% versus 90.1% (non-inferior) and specificity improving from 93.5% to 94.2%.
Strongest existing triage policy. This is the Bernstein et al. framework applied to a strong whole-exam risk model of the Mirai type, which sees both breasts but has no explicit asymmetry mechanism. On 114,229 mammograms they report that a cutoff of 0.05 chosen by adjusted net false omission rate gives a 36% caseload reduction with zero net missed cancers. The Youden cutoff gives 75% reduction at the cost of 121 net missed cancers.
We run this baseline on the same data splits, with encoders of matched size, the same training compute, and the same hyperparameter search budget as our model. Its threshold transfers to the target modality the same way ours does: in standardized units, with target statistics estimated from unlabeled screens. Beating it on mammography is not required. On the target modalities, we beat it if, at the same source-chosen cutoff, our realized FOR stays within the pre-registered tolerance of the source FOR while the baseline's does not, with caseload reduction comparable or higher.
Single-side standardized score. This is the fairest version of "absolute appearance." It uses the same encoder, but each side is scored alone and the exam score is the maximum over sides. It is standardized against the target modality's unlabeled distribution by quantile matching. It shares everything with our method except the left-right comparison, so it isolates the core claim.
Swapped-contralateral ablation. Here each breast is paired with a random other patient's breast from the same scanner type. If cancellation of patient-specific factors is what matters, this ablation should lose most of the transfer benefit while keeping scanner-level cancellation. That separates the "same person" effect from the "same scanner" effect.
Gate ablations. We compare three versions: no gate; a gate using only absolute image scores; and clinical variables late-fused into the score instead of used as a gate, in the spirit of RadFusion.
Reference points outside triage. For context we cite PECon, which reports AUROC 0.943 for CT plus clinical-record pulmonary embolism detection, but it is not a triage comparator. For bilateral chest-CT breast screening, no reported result was found on Lacuna for Dual-Res Tandem Mamba-3D's classification metrics, so we will rerun its architecture as the target-modality encoder baseline.
Why not just…?
Why not just recalibrate the cutoff with a few labeled exams from the new scanner? This is the most serious alternative. If a few hundred labeled target exams sufficed, our method would only be a convenience. The problem is prevalence. At the 0.76% rate in Bernstein et al., certifying a small FOR with any confidence needs many observed cancers, and therefore tens of thousands of labeled screens. We test this directly. We plot realized FOR error against the number of target-modality labeled exams used for recalibration, for the single-side score and for our score with zero target labels. If label-free bilateral standardization matches recalibration only at budgets that hospitals can easily afford, the alternative wins. In that case we would reframe the contribution as reducing the recalibration budget rather than eliminating it.
Why not just quantile-match any good model's scores to the new scanner? Matching the score distribution of unlabeled screens aligns the healthy majority, but it says nothing about where cancers fall. Where cancers fall is what FOR depends on. This alternative is exactly the single-side standardized baseline. If it transfers as well as the asymmetry score, the bilateral hypothesis is false, and the experiment is designed to show that cleanly.
Why not just feed clinical variables into the image model? Fusion improves accuracy in RadFusion. For rule-out, however, the specific danger is symmetric disease that the asymmetry score cannot see, and a fused score can average that risk away. We compare gate and fusion on misses within the cleared set, split by bilateral versus unilateral cancers. Fusion would win if it matches the gate's bilateral-cancer miss rate at equal caseload reduction.
Why not just train a bigger multimodal foundation model across scanner types? Approaches like Med-MLLM improve ranking under scarce labels. But Bernstein et al. show that the operating point, not ranking quality, decides missed cancers. A better ranker with a drifting cutoff is still unsafe. Our standardization can sit on top of such an encoder, and we test it with a pretrained encoder as one variant.
Proposed Experiments
Data. The source modality is 2D screening mammography from EMBED, with VinDr-Mammo as a second mammography site; both are used by GLAM. INBreast serves as a curated lesion-level check. The first target is breast MRI, with side masks from the Divide and Conquer volumes. That release does not supply diagnostic outcomes, so a biopsy-linked MRI cohort must be obtained through a clinical partner; the file identifies none. The second target is non-contrast chest CT with breast findings, in the setting of Dual-Res Tandem Mamba-3D, subject to cohort access. As an out-of-organ check we use paired left-right fundus images, the setting of DMS-Net, where bilateral disease such as diabetic retinopathy is common. That makes it a stress test of the gate, and we expect asymmetry alone to fail there.
Metrics. The headline metric is the transfer gap: the absolute difference between target and source FOR at the source-chosen standardized cutoff, with bootstrap intervals. We also report caseload reduction at matched target FOR, net false omission rate as defined by Bernstein et al. where radiologist reads are available, and misses among cleared exams split into bilateral and unilateral cancers. The tolerance for "transfer holds" is fixed before seeing target results.
Decisive checks. Four results decide the outcome. First, our score's transfer gap must be smaller than the single-side standardized score's on at least two of the three target settings, not only on the mammography-to-mammography site shift. Second, the swapped-contralateral ablation must lose most of that advantage. If it does not, the benefit comes from scanner-level normalization rather than within-patient comparison, and the bilateral story is wrong. Third, the recalibration-budget curve shows how many target labels the single-side model needs to match our zero-label transfer. Fourth, misses among auto-cleared exams with the gate must be concentrated in neither bilateral nor diffuse disease. A null result on the first check would be informative: within-patient comparison does not make cutoffs portable, and per-site labeled recalibration remains necessary.
References
- Triage with AI: A Rule-out Framework Quantifying the Risks and Benefits of Screening Mammogram Automation
- A Deep Learning Model to Triage Screening Mammograms: A Simulation Study
- Dual-Res Tandem Mamba-3D: Bilateral Breast Lesion Detection and Classification on Non-contrast Chest CT
- GLAM: Geometry-Guided Local Alignment for Multi-View VLP in Mammography
- Classification-Driven Pathological Neuroimage Retrieval Using Statistical Asymmetry Measures
- AsT: An Asymmetric-Sensitive Transformer for Osteonecrosis of the Femoral Head Detection (Student Abstract)
- Predicting an unstable tear film through artificial intelligence
- Divide and Conquer: A Large-Scale Dataset and Model for Left-Right Breast MRI Segmentation
- Automatic knee osteoarthritis diagnosis from plain radiographs:a deep learning-based approach
- Symmetry-Enhanced Attention Network for Acute Ischemic Infarct Segmentation with Non-Contrast CT Images
- Bilateral Asymmetry Guided Counterfactual Generating Network for Mammogram Classification
- Differentiable Learning Under Triage
- RadFusion: Benchmarking Performance and Fairness for Multimodal Pulmonary Embolism Detection from CT and EHR
- PECon: Contrastive Pretraining to Enhance Feature Alignment Between CT and EHR Data for Improved Pulmonary Embolism Diagnosis
- A medical multimodal large language model for future pandemics
- DMS-Net:Dual-Modal Multi-Scale Siamese Network for Binocular Fundus Image Classification