Introduction
In many real-world machine learning applications, such as fraud detection, medical diagnosis, and software defect prediction, datasets are often heavily skewed. In these scenarios, the "minority" class—the rare events of interest—is significantly outnumbered by the "majority" class. Standard classification algorithms typically struggle with such imbalanced data, as they are biased toward the majority class to maximize overall accuracy, often at the expense of failing to detect critical minority samples.
To address this, we present DDAE, a comprehensive framework that integrates four distinct strategies: data resampling, metric learning, cost-sensitive weighting, and ensemble learning. By combining these approaches, DDAE significantly improves the detection of rare classes without sacrificing overall performance, outperforming several state-of-the-art methods across diverse public datasets.
The DDAE Framework
The DDAE model is designed as a unified pipeline that tackles the imbalance problem from multiple angles simultaneously. It consists of four primary components as illustrated in the architecture diagram below.
- Data Block Construction (DBC): Instead of processing a single massive dataset, DDAE divides the data into several "balanced" blocks. It employs a hybrid approach of undersampling (partitioning the majority class) and oversampling (replicating the minority class) to ensure each block has a roughly equal class distribution.
- Data Space Improvement (DSI): This component utilizes the Large Margin Nearest Neighbor (LMNN) algorithm. It learns a feature space transformation that minimizes distances between samples of the same class while maximizing distances between different classes. This "cleans up" the data geometry, making class boundaries more distinct for the subsequent classifiers.
- Adaptive Weight Adjustment (AWA): DDAE identifies "unstable samples"—data points where a classifier might be undecided due to a mixed neighborhood of classes. AWA calculates specific weights for these samples to maximize overall gain, making the model more decisive and cost-aware.
- Ensemble Learning (EL): Finally, base classifiers (k-Nearest Neighbors) are trained on each data block. Their predictions are combined using a weighted voting mechanism derived from the AWA component to reach a final consensus.
Experimental Evaluation
The model was rigorously evaluated on 14 public datasets from the OpenML and KEEL repositories, including software defect detection (cm1, mw1, pc1) and biological data (yeast, abalone). The Imbalance Ratio (IR) in these datasets ranged from 7.19 to as high as 129.4, representing extreme classification challenges.
Key Results
DDAE was compared against state-of-the-art methods such as IML, RP, CAdaMEC, and MWMOTE. The results demonstrated:
- Superior Recall: DDAE achieved the best Recall in all 14 cases, reaching 100% on several datasets like
abalone19where competitors often failed entirely. - Balanced Performance: It consistently led in G-mean and AUC metrics, proving it maintains a strong balance between majority and minority class accuracy.
- Robustness: The framework showed relative insensitivity to specific hyperparameter choices, such as the cost ratio, which is critical for real-world deployment where exact misclassification costs are often unknown.
Discussion and Insights
The primary insight from this research is the synergy between the four components. Ablation studies revealed that removing any single part—particularly the Data Block Construction—resulted in a significant performance drop. By using metric learning as a pre-processor, DDAE simplifies the task for the base ensemble, especially in high-dimensional spaces. Furthermore, the model's focus on "unstable samples" allows it to target the areas of the feature space where errors are most likely to occur.
Conclusion and Future Work
DDAE provides a robust, multi-faceted solution for binary imbalanced classification tasks. While highly effective for tabular, feature-based data, future research will explore extending the framework to multi-class imbalanced problems and investigating the integration of different base classifiers, such as Support Vector Machines (SVMs) or Neural Networks, within the ensemble structure.
Summary updated