Imagine a diagnostic model designed to detect a rare, aggressive form of tumor that appears in only 1% of patient scans. If an engineer trains a standard deep neural network on this dataset, the model can achieve a stellar 99% accuracy rate by simply guessing "healthy" every single time.
In a clinical setting, however, this model is worse than useless—it is dangerous.
This tension lies at the heart of majority class bias (or class imbalance bias) in medical machine learning. In real-world clinical datasets, normal or healthy findings naturally and overwhelmingly outnumber pathological cases. When standard machine learning algorithms are applied to this data, they optimize for global accuracy, inadvertently learning to ignore the rare, high-risk anomalies that clinicians care about most.
The Mechanics of the Sensitivity Gap
Why do standard models fail so predictably when data is skewed? The issue stems from how traditional loss functions, such as cross-entropy, drive optimization.
During training, a model updates its weights based on the average error across a batch of samples. If 95% of the batch consists of healthy controls, the gradient signals generated by the majority class will completely drown out the signals from the 5% of diseased cases. The network takes the mathematical path of least resistance: it masterfully learns the normative baseline patterns and treats rare pathological features as negligible noise.
This dynamic creates a severe performance trade-off. As observed by Dai Dai et al. and Wei Cheng et al. in neuroimaging studies for ADHD, models trained on imbalanced datasets often exhibit high specificity (the ability to correctly identify healthy individuals) but dangerously low sensitivity (the ability to detect actual positive cases).
This sensitivity gap spans across nearly every modality of clinical machine learning:
- Physiological Signals: In automated sleep staging or cardiac audio analysis, common physiological states overwhelm rare events. Akara Supratak et al., Morteza Zabihi et al., and Chee-Ming Ting et al. have all highlighted how unmitigated class imbalances bias models toward normal heart sounds or dominant sleep stages, failing to capture critical cardiovascular or neurological anomalies.
- Medical Imaging: Whether identifying early-stage cognitive decline (Fatih Altay et al.), diagnosing kidney stones (Zhenmin Tang et al.), or differentiating glioma recurrence from radiation necrosis (Yu-Zhe Ying et al.), models consistently struggle to isolate minority-class features when healthy instances dominate the training distribution.
- Prognostics and Public Health: When predicting high-risk clinical transitions—such as whether a non-severe COVID-19 patient will deteriorate (Xiaofeng Zhu 0001 et al.) or identifying high-risk individuals in public health surveys (Kyoungsik Jeong et al.)—the sheer volume of positive or stable outcomes biases the system against catching the critical minority. This pattern even extends to veterinary medicine, where Rui Guo et al. noted that models predicting mastitis in dairy cows frequently overlook silent infections due to the overwhelming baseline of healthy subjects.
The Generative Shortcut
Majority class bias does not just impact simple classification; it also compromises complex generative tasks, such as automated radiology report generation.
When a language model or multimodal Transformer is trained to write a clinical narrative from an X-ray or CT scan, it learns the statistical properties of medical language. Because the vast majority of historical reports describe normal anatomy, the model learns that generating phrases like "the lungs are clear" or "no acute abnormality" yields a low text-generation loss.
As a result, the model takes a generative shortcut. Zhihong Chen et al. demonstrated that even when architectural improvements make these models highly fluent, they remain deeply sensitive to data bias. To maintain linguistic fluency, they frequently omit rare but critical pathological findings.
Similarly, when fine-tuning large language models on numerical clinical data, Yanjun Gao et al. found that parameter-efficient fine-tuning can cause the model's internal representations to over-index on majority classes, degrading its ability to predict rare, high-stakes events like patient mortality. Even in unstructured text classification, Efsun Sarioglu Kayi et al. showed that unsupervised topic models lose their ability to distinguish clinical categories when the underlying text distribution is heavily skewed.
Algorithmic and Structural Countermeasures
To build reliable clinical AI, researchers must explicitly force models to prioritize the minority class. Modern strategies attack this problem at three distinct levels: the data, the loss function, and the model architecture.
Data-Level Rebalancing
The most direct way to counter majority bias is to alter the training distribution so the model is forced to see the minority class more frequently. Anita Desiani et al. demonstrated that combining missing data imputation with Adaptive Synthetic Sampling (ADASYN)—which mathematically generates synthetic examples targeted at the most difficult-to-learn minority boundaries—can dramatically rescue a model's sensitivity in heart disease classification.
In low-shot scenarios, semi-supervised learning can also protect the minority class. For instance, Rui Fan 0001 et al. developed a "One-Vote Veto" self-training strategy for glaucoma diagnosis that safely leverages unlabeled data to prevent the model from collapsing toward majority-class predictions.
Loss-Level Modification
Rather than changing the data, researchers can change the mathematical penalties. By weighting the loss function, we can ensure that misclassifying a rare pathology incurs a massive penalty, while misclassifying a normal case incurs a minor one.
Hyunkyung Han et al. successfully utilized this approach in their Capsule Network architecture, tying a weighted loss function directly to cardiac metrics to ensure that rare, abnormal echocardiograms were never ignored by the network's optimization step.
Structural Constraints and Interpretability
Perhaps the most robust defense against majority bias is changing how the model is forced to process information, preventing it from taking shortcuts.
In a single-author work, Shixiang Feng addressed this by designing a dual-stream network that explicitly compares adjacent vertebrae, forcing the model to look at relative structural changes rather than getting overwhelmed by global normal baselines. To ensure models learn genuine clinical indicators rather than dataset artifacts, Özgur Güler et al. combined class-balancing objectives with interpretability constraints, forcing the network to ground its decisions in actual chest X-ray pathology.
Standard Architecture (Prone to Bias):
Medical Image ───► Deep Network ───► Fluent but Biased Report (Omits rare pathology)
Hierarchical Architecture (Bias-Resistant):
Medical Image ───► Predict Clinical Indicators ───► Generate Report (Forced to include pathology)
(e.g., Effusion: Positive)
For generative text models, this structural bottleneck is incredibly powerful. Keqiang Fan et al. developed the Image-to-Indicator Hierarchical Transformer (IIHT) to mimic human radiologists. Instead of mapping an image directly to a text report, the architecture explicitly forces the network to first predict the status of a predefined set of clinical indicators (e.g., presence or absence of specific abnormalities). Because the model is structurally required to evaluate these indicators before writing, it cannot gloss over rare conditions in favor of majority-class linguistic fluency.
By shifting the optimization focus from global accuracy to structural and clinical accountability, these techniques ensure that machine learning models serve as reliable safety nets in clinical practice, rather than echo chambers for the majority class.
Key Foundational Work
Individual classification of ADHD patients by integrating multiscale neuroimaging markers and advanced pattern recognition techniques (2012) — contributed 2 concept elements to this direction.
Classification of ADHD children through multimodal magnetic resonance imaging (2012) — contributed 1 concept elements to this direction.
Topic Modeling Based Classification of Clinical Reports (2013) — contributed 1 concept elements to this direction.