An Effective Approach for Imbalanced Classification: Unevenly Balanced Bagging

Guohua Liang, Anthony G. CohnAAAI 2013 · DOI

Abstract

Standard bagging techniques often struggle in imbalanced classification scenarios where the minority class is significantly underrepresented. This paper proposes Unevenly Balanced Bagging (UBagging), an ensemble method that explores a range of class ratios during training. By generating a diverse set of classifiers tuned to different distribution balances, UBagging achieves superior performance compared to traditional bagging and fixed-ratio balancing techniques.

Introduction

In real-world domains like fraud detection and medical diagnosis, datasets are frequently imbalanced. Traditional machine learning algorithms, including standard Bagging (Bootstrap Aggregating), often ignore the minority class to maximize overall accuracy. While "Balanced Bagging" (BBagging) attempts to fix this by enforcing a 50/50 class ratio, a perfectly equal split is not always optimal for every dataset. UBagging addresses this by shifting the focus from finding a single perfect ratio to creating an ensemble that explores a spectrum of class distributions.

Methodology: The UBagging Algorithm

The UBagging approach generates a "diversity of perspectives" regarding the class boundary through the following steps:

  1. Fixed Positive Samples: Each bootstrap sample (bag) contains all positive instances ($|P|$) from the original dataset.
  2. Varying Negative Samples: The number of negative instances ($|N_i|$) in each bag is incrementally adjusted. The ratio starts at 50% of the positive class size ($0.5 \times |P|$) and increases in 5% increments up to 200% ($2.0 \times |P|$).
  3. Ensemble Creation: This results in 31 distinct training subsets, each with a unique class distribution. A base learner (J48 decision tree) is trained on each subset.
  4. Voting: Final predictions are determined by a simple majority vote among the 31 classifiers.

This strategy ensures the ensemble "brackets" the potential optimal training ratio for any given imbalanced problem.

Experimental Setup

The researchers evaluated UBagging using 32 imbalanced datasets from the UCI Machine Learning Repository. Performance was compared against three benchmarks: a single J48 decision tree, standard Bagging, and Equally Balanced Bagging (BBagging). Evaluation metrics included the F-value and G-mean, which are more appropriate for imbalanced data than standard accuracy.

Key Results and Discussion

UBagging demonstrated superior performance and consistency across the tested datasets.

Table 1: Performance comparison of UBagging against benchmarks across 32 datasets.

As shown in Table 1, UBagging achieved the highest average F-value (0.787) and G-mean (0.902). Notably, it also exhibited the lowest standard deviation (STD), indicating higher reliability across diverse data distributions.

Figure 1: Comparison of Average Rank of G-mean for prediction models.

Statistical analysis using the Nemenyi test confirmed that UBagging is statistically superior to the other three models at a 95% confidence interval. Figure 1 illustrates the ranking order for G-mean, where UBagging consistently holds the top rank (closest to 1.0).

The success of UBagging stems from its "Sweet Spot" strategy. Since the optimal training ratio is rarely known a priori, UBagging's range of ratios ensures that at least some models in the ensemble are optimally tuned for the specific dataset's characteristics.

Conclusion and Future Work

UBagging provides a robust framework for handling imbalanced data by leveraging distribution-based diversity. While the current study focused on J48 decision trees, future work could explore:

  • Base Learner Generalization: Testing UBagging with other algorithms like SVMs or Neural Networks.
  • Hyperparameter Optimization: Investigating if the 50%–200% range and 5% step size should be dynamically adjusted based on the dataset's intrinsic imbalance.
  • Computational Efficiency: Optimizing the number of bags to reduce training time without sacrificing performance.

Summary updated

BibTeX
@inproceedings{liang2013effective,
  author = {Guohua Liang and Anthony G. Cohn},
  title = {{An Effective Approach for Imbalanced Classification: Unevenly Balanced Bagging}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2013},
  doi = {10.1609/aaai.v27i1.8536},
}
Figure