A Novel Model for Imbalanced Data Classification

Jian Yin, Chunjing Gan, Kaiqi Zhao, Xuan Lin, Zhe Quan, Zhi-Jie Wang

Introduction

In many real-world machine learning applications, such as fraud detection, medical diagnosis, and software defect prediction, datasets are often heavily skewed. In these scenarios, the "minority" class—the rare events of interest—is significantly outnumbered by the "majority" class. Standard classification algorithms typically struggle with such imbalanced data, as they are biased toward the majority class to maximize overall accuracy, often at the expense of failing to detect critical minority samples.

To address this, we present DDAE, a comprehensive framework that integrates four distinct strategies: data resampling, metric learning, cost-sensitive weighting, and ensemble learning. By combining these approaches, DDAE significantly improves the detection of rare classes without sacrificing overall performance, outperforming several state-of-the-art methods across diverse public datasets.

The DDAE Framework

The DDAE model is designed as a unified pipeline that tackles the imbalance problem from multiple angles simultaneously. It consists of four primary components as illustrated in the architecture diagram below.

Figure 1: The architecture of the DDAE framework, showing the flow from Data Block Construction (DBC) through Data Space Improvement (DSI) and Adaptive Weight Adjustment (AWA) to the final Ensemble Learning (EL) component.
  1. Data Block Construction (DBC): Instead of processing a single massive dataset, DDAE divides the data into several "balanced" blocks. It employs a hybrid approach of undersampling (partitioning the majority class) and oversampling (replicating the minority class) to ensure each block has a roughly equal class distribution.
  2. Data Space Improvement (DSI): This component utilizes the Large Margin Nearest Neighbor (LMNN) algorithm. It learns a feature space transformation that minimizes distances between samples of the same class while maximizing distances between different classes. This "cleans up" the data geometry, making class boundaries more distinct for the subsequent classifiers.
  3. Adaptive Weight Adjustment (AWA): DDAE identifies "unstable samples"—data points where a classifier might be undecided due to a mixed neighborhood of classes. AWA calculates specific weights for these samples to maximize overall gain, making the model more decisive and cost-aware.
  4. Ensemble Learning (EL): Finally, base classifiers (k-Nearest Neighbors) are trained on each data block. Their predictions are combined using a weighted voting mechanism derived from the AWA component to reach a final consensus.

Experimental Evaluation

The model was rigorously evaluated on 14 public datasets from the OpenML and KEEL repositories, including software defect detection (cm1, mw1, pc1) and biological data (yeast, abalone). The Imbalance Ratio (IR) in these datasets ranged from 7.19 to as high as 129.4, representing extreme classification challenges.

Key Results

DDAE was compared against state-of-the-art methods such as IML, RP, CAdaMEC, and MWMOTE. The results demonstrated:

  • Superior Recall: DDAE achieved the best Recall in all 14 cases, reaching 100% on several datasets like abalone19 where competitors often failed entirely.
  • Balanced Performance: It consistently led in G-mean and AUC metrics, proving it maintains a strong balance between majority and minority class accuracy.
  • Robustness: The framework showed relative insensitivity to specific hyperparameter choices, such as the cost ratio, which is critical for real-world deployment where exact misclassification costs are often unknown.
Figure 4: Sensitivity analysis of DDAE on the mw1 dataset, showing the impact of parameters λ, γ, cost ratio, and the number of ensemble blocks on performance.

Discussion and Insights

The primary insight from this research is the synergy between the four components. Ablation studies revealed that removing any single part—particularly the Data Block Construction—resulted in a significant performance drop. By using metric learning as a pre-processor, DDAE simplifies the task for the base ensemble, especially in high-dimensional spaces. Furthermore, the model's focus on "unstable samples" allows it to target the areas of the feature space where errors are most likely to occur.

Conclusion and Future Work

DDAE provides a robust, multi-faceted solution for binary imbalanced classification tasks. While highly effective for tabular, feature-based data, future research will explore extending the framework to multi-class imbalanced problems and investigating the integration of different base classifiers, such as Support Vector Machines (SVMs) or Neural Networks, within the ensemble structure.

Summary updated

Abstract

Recently, imbalanced data classification has received much attention due to its wide applications. In the literature, existing researches have attempted to improve the classification performance by considering various factors such as the imbalanced distribution, cost-sensitive learning, data space improvement, and ensemble learning. Nevertheless, most of the existing methods focus on only part of these main aspects/factors. In this work, we propose a novel imbalanced data classification model that considers all these main aspects. To evaluate the performance of our proposed model, we have conducted experiments based on 14 public datasets. The results show that our model outperforms the state-of-the-art methods in terms of recall, G-mean, F-measure and AUC.
BibTeX
@inproceedings{yin2020novel,
  author = {Jian Yin and Chunjing Gan and Kaiqi Zhao and Xuan Lin and Zhe Quan and Zhi-Jie Wang},
  title = {{A Novel Model for Imbalanced Data Classification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i04.6145},
}
Figure