TL;DR
EWGAN (Entropy-based Wasserstein GAN) is a novel oversampling strategy designed to address imbalanced datasets by generating high-quality synthetic samples for minority classes. By combining the training stability of Wasserstein GANs with a unique entropy-weighting scheme inspired by Focal Loss, it produces more representative data than traditional methods like SMOTE or standard GANs, significantly boosting classification accuracy.
Introduction: The Challenge of Imbalanced Learning
In many real-world machine learning tasks—such as fraud detection or medical diagnosis—data is often "imbalanced," where the majority class far outnumbers the minority class. Standard classifiers typically struggle in these scenarios, often ignoring the minority class to maximize overall accuracy.
To mitigate this, oversampling techniques are used to create synthetic data for the minority class. While SMOTE (Synthetic Minority Over-sampling Technique) is a common choice, it often fails to capture complex data distributions. Generative Adversarial Networks (GANs) offer a more powerful alternative but face two major hurdles: training instability (mode collapse) and "label blindness," where conditional GANs (cGANs) fail to account for the degree of imbalance or the relative importance of minority samples.
Methodology: The EWGAN Approach
The authors introduce EWGAN, which enhances the generative process through two key innovations:
- Entropy-Weighted Label Vectors: Instead of standard one-hot encoding, EWGAN calculates an "importance factor" ($I_i$) for each class based on its dataset proportion. Using a formula inspired by Focal Loss ($I_i = -\frac{(1-p_i)^2}{p_i} \log(p_i)$), the model assigns higher weights to rarer classes. This directs the generator to focus more intensely on the minority distribution.
- Wasserstein GAN (WGAN) Framework: To address the instability of traditional GANs, EWGAN utilizes the Wasserstein distance. This provides smoother gradients for the generator, ensuring a more reliable training process compared to methods using KL divergence.
The workflow involves concatenating these entropy-weighted labels with real feature vectors to train the generator. Once trained, the generator uses random noise and the minority class's weighted label to produce synthetic samples that augment the original training set.
Experimental Setup and Datasets
The researchers evaluated EWGAN on two benchmark imbalanced datasets from the KEEL repository: Vowel0 and Page-blocks0. These datasets represent varying scales and dimensions, both featuring significant imbalance ratios.
- Vowel0: 988 samples, 13 dimensions, imbalance ratio of 9.98.
- Page-blocks0: 5472 samples, 10 dimensions, imbalance ratio of 8.79.
Key Results
EWGAN was compared against several baselines, including No Oversampling, Kmeans-SMOTE, standard WGAN, and cGAN, using a Support Vector Machine (SVM) as the final classifier.
As shown in the results table, EWGAN consistently outperformed all other methods:
- Vowel0: EWGAN achieved an accuracy of 0.8385, a substantial improvement over the baseline (0.5645) and cGAN (0.8135).
- Page-blocks0: EWGAN reached 0.8322, outperforming the baseline (0.8023) and standard cGAN (0.7567).
Notably, the standard (unsupervised) WGAN performed poorly on Vowel0 (0.5638), demonstrating that stability alone is insufficient; incorporating label information via entropy weighting is critical for effective balancing.
Discussion and Conclusion
The success of EWGAN lies in treating class imbalance as an information density problem. By using entropy to weight labels, the model treats the minority class as a high-value signal. Furthermore, the WGAN architecture provides the necessary stability for high-quality synthetic data generation.
While the current study focuses on binary imbalance in relatively small datasets, the results are promising. Future work will involve scaling EWGAN to higher-dimensional data and extending the framework to multi-class imbalanced learning scenarios.
Summary updated