In many real-world classification tasks—from detecting credit card fraud to diagnosing rare diseases—the "interesting" class is often the rarest. Machine learning models trained on such imbalanced datasets tend to develop a "majority bias," effectively ignoring the minority class to achieve high overall accuracy.
To fix this, researchers often use oversampling, with the Synthetic Minority Over-sampling Technique (SMOTE) being the industry standard. SMOTE works by picking a minority instance and its nearest neighbors, then creating new data points along the lines connecting them. While this balances the numbers, it often fails in practice because it is "blind" to the surrounding data landscape.
The Failure of Blind Interpolation
The fundamental weakness of traditional oversampling is that it generates synthetic data without considering the proximity of the majority class. When minority samples are located near the decision boundary or are scattered as outliers, SMOTE’s linear interpolation creates "bridges" of synthetic data that cross into majority-class territory.
This phenomenon, known as class overlap, blurs the decision boundary. As Wen Chen and Yanfei Sun argue in Over-sampling algorithm for imbalanced data classification, this overgeneralization makes it harder for classifiers to distinguish between classes, often leading to an increase in false alarms. Instead of clarifying the minority distribution, blind oversampling amplifies the noise.
Categorizing Minority Instances
To move beyond blind interpolation, researchers have begun treating minority instances differently based on their local neighborhood. Bartosz Krawczyk and Przemysław Skryjomski propose a taxonomy of minority instances in Influence of minority class instance types on SMOTE imbalanced data oversampling:
- Safe: Surrounded by other minority samples.
- Borderline: Located near the decision boundary.
- Rare: Isolated but legitimate samples.
- Outliers: Noisy points that should likely be ignored.
By identifying these types, algorithms like DSMOTE can selectively oversample "borderline" samples to sharpen the boundary while avoiding "outliers" that would otherwise create synthetic noise. Similarly, Shuhao Fan introduces a dynamic denoising strategy in Local Density-Aware Oversampling with Noise Resistance for Imbalanced Data Classification to ensure that synthetic points clarify rather than obscure these critical regions.
Leveraging the Majority Class for Context
A significant shift in the field is the realization that the majority class provides essential context for where not to generate data. Traditional SMOTE only looks at the minority class, but newer methods like SOMM (Synthetic Over-sampling with Minority and Majority classes) use information from both distributions.
As described in A synthetic over-sampling method with minority and majority classes for imbalance problems, these algorithms adaptively "push" synthetic samples away from majority-class clusters. Li Liu takes this further in Counterfactual-based minority oversampling for imbalanced classification by using counterfactual theory to perturb majority samples until they cross the boundary, creating synthetic minority points exactly where the classifier is most uncertain.
Global Geometry and Structural Constraints
Beyond local neighborhoods, researchers are exploring the global geometric structure of data. Yuguang Yan and colleagues introduced OTOS (Optimal Transport for OverSampling) in Oversampling for Imbalanced Data via Optimal Transport, which uses the mathematics of optimal transport to synthesize samples that respect the global distribution rather than just local clusters.
In the domain of deep learning, this structural awareness is often enforced through hybrid frameworks. For example:
- SMOTE-CLS: Uses a Conditional Variational Autoencoder (CVAE) to "clean" the latent space before oversampling, as seen in Improving SMOTE via Fusing Conditional VAE for Data-adaptive Noise Filtering.
- RB-CCR: Combines radial basis functions with data cleaning to target specific sub-regions for generation, as detailed in RB-CCR: Radial-Based Combined Cleaning and Resampling algorithm for imbalanced data classification.
- INGB: Replaces linear interpolation with "granular balls" (hyperspheres) to create a more robust, nonlinear representation of the data, according to Min Li and Hao Zhou in INGB: Informed Nonlinear Granular Ball Oversampling Framework for Noisy Imbalanced Classification.
The Goal: Class Separability
The common thread across these diverse approaches—from optimal transport to counterfactuals—is a move away from simple "data volume" toward data quality. The objective is no longer just to make the class counts equal, but to ensure that the synthetic data preserves or enhances the class separability. By being "boundary-aware," these modern oversampling techniques prevent the introduction of semantic ambiguity, allowing models to learn the complex patterns of rare events without being drowned out by synthetic noise.