A Parameter-Free Cleaning Method for SMOTE in Imbalanced Classification

Yuan-Ting Yan, Ruiqing Liu, Zihan Ding, Xiuquan Du, Jie Chen, Yanping ZhangIEEE Access 2019 · OpenReview

Abstract

Oversampling is an efficient technique for dealing with class-imbalance problems by generating minority class samples to balance the distribution. Synthetic Minority Over-sampling Technique (SMOTE) is a prominent example, but it often generates erroneous samples in complex scenarios. This paper proposes a parameter-free data cleaning method based on a constructive covering algorithm (CCA) to improve SMOTE. The dataset generated by SMOTE is first partitioned into a group of covers, then the hard-to-learn samples can be detected based on the characteristics of sample space distribution. Finally, a pair-wise deletion strategy is proposed to remove the hard-to-learn samples. Experimental results on 25 imbalanced datasets show that our proposed method is superior to the comparison methods in terms of various metrics, such as F-measure, G-mean, and Recall.

Introduction

In real-world applications such as medical diagnosis and fraud detection, datasets are frequently imbalanced, where the minority class contains the most critical information. Standard classifiers often struggle with such data, tending to favor the majority class. To mitigate this, the Synthetic Minority Over-sampling Technique (SMOTE) generates synthetic samples along the line segments joining minority class neighbors.

Illustration of the SMOTE algorithm generating a new synthetic sample $x_{new}$ between existing minority samples $x_i$ and $x_j$.

However, SMOTE operates without considering the majority class distribution. As shown in the illustration above, it can generate samples that overlap with majority class regions, creating "noise" that blurs decision boundaries and confuses classifiers. While researchers have tried to "clean" these datasets using methods like Tomek Links, these often require manual parameter tuning and can be inconsistent.

The SM+CCA Framework

To address the noise generated by SMOTE, we introduce a two-stage pipeline: Oversampling followed by Geometric Cleaning (SM+CCA). This framework aims to balance the dataset while sharpening the boundary between classes.

The proposed framework for imbalanced data classification, featuring data pre-processing and classification stages.

As illustrated in the framework overview, the process begins with an imbalanced dataset. After initial oversampling via SMOTE, the data undergoes a cleaning process based on the Constructive Covering Algorithm (CCA).

Geometric Cleaning via CCA

The core innovation lies in using CCA, a geometric neural network approach, to partition the feature space into spherical "covers." Unlike traditional cleaning methods that rely on $k$-nearest neighbors and require manual parameter tuning, CCA is parameter-free. It automatically adapts to the data's shape.

Samples that fall into isolated covers (containing only one sample) are flagged as "hard-to-learn" or noise. These samples typically reside deep within the territory of the opposing class or are outliers that hinder generalization.

Pair-wise Deletion Strategy

Simply deleting noisy minority samples can re-introduce class imbalance. To maintain a balanced distribution, we employ a pair-wise deletion strategy. If $t$ minority samples are identified as noise and removed, $t$ majority samples are also deleted. The majority samples selected for removal are those that most aggressively "invade" the minority space, determined by a specific distance metric (Dist_SC). This dual pruning effectively clarifies the decision boundary, making it easier for models like decision trees to generalize.

Experimental Evaluation

The SM+CCA method was rigorously tested on 25 imbalanced datasets from the UCI and KEEL repositories, with imbalance ratios ranging from 1.7 to 85.88. These datasets span various domains, including medical (e.g., breast cancer), technical (e.g., software defects), and general classification tasks.

Key Results

  • Superior Performance: SM+CCA outperformed established methods like Borderline-SMOTE, ADASYN, and SMOTE+Tomek on 21 out of 25 datasets regarding the F-measure.
  • Statistical Significance: Wilcoxon signed-rank tests confirmed that the improvements in F-measure, G-mean, Recall, AUC, and Precision are statistically significant.
  • Robustness: The method proved particularly effective for highly imbalanced datasets (IR > 4), where traditional oversampling often fails to create distinct class boundaries.

Conclusion

The proposed SM+CCA method provides a robust, parameter-free solution to the limitations of SMOTE. By leveraging the geometric properties of data through CCA and employing a balanced pair-wise deletion strategy, it simplifies the dataset's complexity and enhances the performance of classification models. Future work will explore extending this geometric approach to multi-class imbalanced problems and integrating CCA into other oversampling techniques.

Summary updated

Abstract

Oversampling is an efficient technique in dealing with class-imbalance problem. It addresses the problem by reduplicating or generating the minority class samples to balance the distribution between the samples of the majority and the minority class. Synthetic minority oversampling technique (SMOTE) is one of the typical representatives. During the past decade, researchers have proposed many variants of SMOTE. However, the existing oversampling methods may generate wrong minority class samples in some scenarios. Furthermore, how to effectively mine the inherent complex characteristics of imbalanced data remains a challenge. To this end, this paper proposes a parameter-free data cleaning method to improve SMOTE based on constructive covering algorithm. The dataset generated by SMOTE is first partitioned into a group of covers, then the hard-to-learn samples can be detected based on the characteristics of sample space distribution. Finally, a pair-wise deletion strategy is proposed to remove the hard-to-learn samples. The experimental results on 25 imbalanced datasets show that our proposed method is superior to the comparison methods in terms of various metrics, such as F-measure, G-mean, and Recall. Our method not only can reduce the complexity of the dataset but also can improve the performance of the classification model.
BibTeX
@inproceedings{yan2019parameter,
  author = {Yuan-Ting Yan and Ruiqing Liu and Zihan Ding and Xiuquan Du and Jie Chen and Yanping Zhang},
  title = {{A Parameter-Free Cleaning Method for SMOTE in Imbalanced Classification}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=7ox0FcOyr_5},
}
Figure