Using Small Business Banking Data for Explainable Credit Risk Scoring

Wei Wang, Christopher Lesner, Alexander Ran, Marko Rukonic, Jason Xue, Eric ShiuAAAI 2020 · DOI

Introduction

In small business lending, accuracy must be balanced with regulatory accountability. Traditional "scorecard" models are transparent but often fail to capture complex financial nuances. Conversely, modern machine learning (ML) models like Gradient Boosted Trees offer superior accuracy but are typically "black boxes." This research from Intuit demonstrates a credit risk scoring system that bridges this gap, meeting U.S. Fair Credit Reporting Act (FCRA) requirements for "Adverse Action" reasons while outperforming traditional methods.

Methodology: Balancing Performance and Transparency

The researchers implemented a two-pronged strategy to ensure models are both high-performing and interpretable:

  1. Monotonic XGBoost: By enforcing monotonic constraints, the model ensures that the relationship between features (e.g., bank balance) and risk follows a logical direction. This prevents overfitting to noise in transaction data.
  2. SHAP-WoE Hybrid Explanations: Shapley Additive Explanations (SHAP) were used to decompose individual predictions into feature contributions. These values were then mapped to standard Adverse Action (AA) codes to provide legally required denial reasons.
SHAP values illustrating the impact of various features on the model output.

Data and Constraints as Regularization

The study utilized a massive dataset from Intuit QuickBooks Capital, comprising tens of thousands of loans and hundreds of millions of transactions. Despite the "noisy" nature of raw banking data (e.g., scraping errors), the authors found that monotonic constraints acted as a powerful regularizer, improving model robustness against artifacts in the transaction data.

Key Results

The constrained XGBoost model significantly outperformed traditional scorecards and unconstrained ML models.

  • Predictive Power: The model achieved a 7% improvement in the K-S statistic over traditional scorecards. Monotonic constraints alone boosted K-S performance by 8% compared to unconstrained XGBoost.
  • Consistency: The ML model's explanations showed a 42-52% agreement rate with legacy scorecard reasons, indicating that it identifies similar primary risk drivers but with greater precision.
Comparison of K-S statistic distributions for models with and without monotonic constraints.

Business Impact and Future Work

Deployed since July 2019, the system allows QuickBooks Capital to identify the riskiest 20% of applicants; rejecting this segment is estimated to reduce loan default rates by 20%. Future efforts focus on automating the semantic validation of explanations to ensure they remain intuitive and legally sound as feature complexity grows.

Summary updated

BibTeX
@inproceedings{wang2020using,
  author = {Wei Wang and Christopher Lesner and Alexander Ran and Marko Rukonic and Jason Xue and Eric Shiu},
  title = {{Using Small Business Banking Data for Explainable Credit Risk Scoring}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i08.7055},
}
Figure