Predicting class-imbalanced business risk using resampling, regularization, and model ensembling algorithms

Yan Wang, Xuelei Sherry Ni

Abstract

We aim at developing and improving the imbalanced business risk modeling via jointly using proper evaluation criteria, resampling, cross-validation, classifier regularization, and ensembling techniques. Area Under the Receiver Operating Characteristic Curve (AUC of ROC) is used for model comparison based on 10-fold cross validation. Two undersampling strategies including random undersampling (RUS) and cluster centroid undersampling (CCUS), as well as two oversampling methods including random oversampling (ROS) and Synthetic Minority Oversampling Technique (SMOTE), are applied. Three highly interpretable classifiers, including logistic regression without regularization (LR), L1-regularized LR (L1LR), and decision tree (DT) are implemented. Two ensembling techniques, including Bagging and Boosting, are applied on the DT classifier for further model improvement. The results show that, Boosting on DT by using the oversampled data containing 50% positives via SMOTE is the optimal model and it can achieve AUC, recall, and F1 score valued 0.8633, 0.9260, and 0.8907, respectively.

Introduction

In financial risk modeling, datasets are typically characterized by extreme class imbalance. Most companies remain solvent, while only a small fraction faces bankruptcy. Standard machine learning algorithms often succumb to the "Accuracy Trap," where a model achieves high accuracy by simply predicting the majority class. For financial institutions, however, the cost of a False Negative (missing a risky company) far outweighs the cost of a False Positive. This research explores a robust framework to enhance predictive performance in such imbalanced scenarios.

Methodology

The study evaluates a multi-stage workflow integrating various techniques to address class imbalance and improve model robustness. The researchers prioritized metrics like AUC (Area Under the ROC Curve), Recall, and F1-score to capture the model's ability to identify rare positive instances.

Comparison of undersampling and oversampling strategies to reach desired class balance.

Key components of the methodology include:

  1. Resampling Strategies: Four methods were tested: Random Undersampling (RUS), Cluster Centroid Undersampling (CCUS), Random Oversampling (ROS), and Synthetic Minority Oversampling Technique (SMOTE).
  2. Classifiers and Regularization: The study implemented Logistic Regression (LR), L1-regularized Logistic Regression (L1LR) to prevent overfitting, and Decision Trees (DT).
  3. Ensemble Learning: Bagging and Boosting were applied to Decision Trees to transform weak learners into a strong predictive model.
  4. Cross-Validation: A 10-fold cross-validation was conducted prior to resampling to prevent data leakage and ensure realistic performance estimates.
Comprehensive experimental workflow including cross-validation, resampling, modeling, and ensembling.

Data Description

The analysis utilized a massive real-world dataset from the US National Credit Bureau, encompassing over 10 million de-identified U.S. companies from 2006 to 2014. The target variable, RiskInd, identified businesses transitioning to bankruptcy between 2013 and 2014. The dataset featured 36 independent variables, including business size and liability indicators, with an initial positive (risky) rate of only 7.4%.

Results and Discussion

The experimental results highlight several key findings:

  • SMOTE Superiority: SMOTE consistently outperformed other resampling methods by creating synthetic examples that better define the minority class boundary.
  • Optimal Resampling Ratios: The ideal balance is not always 50/50. While DT performed best at a 50% ratio, LR reached its peak at a 20% positive ratio.
  • Boosting vs. Bagging: Boosting proved superior for risk detection. While Bagging improved AUC, it reduced Recall. Boosting’s iterative focus on misclassified instances was more effective at identifying elusive risky companies.
  • Top Predictors: The most significant risk indicator was the time since the last public record report (MonLstRptDatePlcRec), followed by the count of liens or judgments.

The optimal configuration was identified as Boosting on a Decision Tree using 50% SMOTE-oversampled data, achieving an AUC of 0.8633, a Recall of 0.9260, and an F1-score of 0.8907.

Conclusion

While the proposed framework significantly improves bankruptcy prediction, the optimal resampling ratio remains dataset-dependent. Future research could explore more complex models like deep neural networks or Gradient Boosted Trees (e.g., XGBoost) to further enhance performance, provided the trade-off with interpretability is acceptable for the financial industry.

Summary updated

Abstract

We aim at developing and improving the imbalanced business risk modeling via jointly using proper evaluation criteria, resampling, cross-validation, classifier regularization, and ensembling techniques. Area Under the Receiver Operating Characteristic Curve (AUC of ROC) is used for model comparison based on 10-fold cross validation. Two undersampling strategies including random undersampling (RUS) and cluster centroid undersampling (CCUS), as well as two oversampling methods including random oversampling (ROS) and Synthetic Minority Oversampling Technique (SMOTE), are applied. Three highly interpretable classifiers, including logistic regression without regularization (LR), L1-regularized LR (L1LR), and decision tree (DT) are implemented. Two ensembling techniques, including Bagging and Boosting, are applied on the DT classifier for further model improvement. The results show that, Boosting on DT by using the oversampled data containing 50% positives via SMOTE is the optimal model and it can achieve AUC, recall, and F1 score valued 0.8633, 0.9260, and 0.8907, respectively.
BibTeX
@article{wang2019predicting,
  author = {Yan Wang and Xuelei Sherry Ni},
  title = {{Predicting class-imbalanced business risk using resampling, regularization, and model ensembling algorithms}},
  journal = {International Journal of Managing Information Technology},
  year = {2019},
  doi = {10.5121/ijmit.2019.11101},
}
Figure