Abstract
We aim at developing and improving the imbalanced business risk modeling via jointly using proper evaluation criteria, resampling, cross-validation, classifier regularization, and ensembling techniques.
Introduction
Predicting business bankruptcy is a critical task for financial institutions and investors. However, the inherent rarity of bankruptcy events leads to a significant "class imbalance" problem. In typical datasets, healthy companies vastly outnumber failing ones. Standard machine learning models often fall into the "accuracy trap," where they achieve high overall accuracy by simply predicting the majority class (healthy) for all instances, thereby failing to identify the high-risk minority class. This study addresses these challenges by proposing a comprehensive framework that integrates data resampling, model regularization, and ensemble learning.
Methodology
Resampling Strategies
To mitigate class imbalance, the researchers explored two primary resampling techniques: undersampling and oversampling. Undersampling involves removing instances from the majority class to balance the ratio, while oversampling involves increasing the representation of the minority class.
A key technique employed was SMOTE (Synthetic Minority Oversampling Technique). Unlike simple random oversampling, which duplicates existing minority instances, SMOTE generates new, synthetic examples by interpolating between existing minority class points and their nearest neighbors. This approach provides the model with a more diverse set of failure patterns to learn from.
Regularization and Ensemble Learning
The study evaluated several classification algorithms:
- L1-regularized Logistic Regression (L1LR): Used to prevent overfitting and perform automatic feature selection by penalizing less significant variables.
- Bagging on Decision Trees: An ensemble method that builds multiple trees in parallel to reduce variance.
- Boosting on Decision Trees: A sequential ensemble method where each subsequent tree focuses on correcting the errors of the previous ones.
Experimental Workflow
A critical aspect of the methodology is the rigorous validation process. The authors emphasize that resampling must be performed within the cross-validation loop. Specifically, for each fold of a 10-fold cross-validation, resampling is applied only to the training set. This prevents "data leakage," where synthetic information from the training set might inadvertently influence the validation set, leading to over-optimistic performance metrics.
Data and Experiments
The research utilized a dataset from the US National Credit Bureau, encompassing over 10 million de-identified U.S. companies from 2006 to 2014. The target variable, RiskInd, identified companies that transitioned from healthy in 2013 to bankrupt in 2014 (approximately 7.4% of the sample). The models were trained on 36 independent variables, including business size, liabilities, and industry activity.
Results and Discussion
The experimental results demonstrated that Boosting on Decision Trees combined with SMOTE-oversampled data (at a 50% ratio) was the most effective approach. This "champion" model achieved an AUC of 0.8633 and a remarkably high recall of 0.9260, indicating its ability to identify over 92% of actual bankruptcies.
Key findings include:
- SMOTE Superiority: SMOTE consistently outperformed random oversampling and undersampling across various metrics.
- Optimal Ratios: While Decision Trees performed best with a 50/50 balanced split, Logistic Regression reached its peak performance at a 20% minority / 80% majority ratio.
- Boosting vs. Bagging: Boosting proved superior for recall in this risk context, whereas Bagging, while improving AUC, sometimes reduced the model's ability to catch all risky cases.
- Feature Importance: The most significant predictor of risk was identified as
MonLstRptDatePlcRec(months since the last public record report).
Conclusion
This study provides a robust framework for handling imbalanced datasets in business risk prediction. By combining synthetic oversampling (SMOTE) with sequential ensemble methods (Boosting) and ensuring rigorous cross-validation, practitioners can build models that move beyond simple accuracy to effectively capture rare but high-impact events like business bankruptcy.
Summary updated