Loan Prepayment Prediction Based on SVM-RFE and XGBoost Models

Qi Mao, Gang Liu, Zhiyu Chen, Jianwei Guo, Peng LiuProceedings of the International Conference on Information Economy, Data Modeling and Cloud Computing, ICIDC 2022, 17-19 June 2022, Qingdao, China 2022 · DOI

TL;DR

This paper introduces a robust framework for predicting loan prepayment by combining a hybrid feature selection method (Pearson + SVM-RFE) with a modified XGBoost model. By integrating a weighted cross-entropy loss function, the authors significantly improve the model's ability to identify early repayments in highly imbalanced financial datasets, outperforming standard boosting algorithms.

What Problem Does This Paper Address?

In the world of online P2P lending, "prepayment"—when a borrower pays off their loan early—is a double-edged sword. While it sounds positive, it actually hurts investors by reducing expected interest income and creates unstable cash flows for lending platforms.

Predicting this behavior is technically challenging for two main reasons:

  1. High Dimensionality: Loan data often contains dozens of variables, many of which are redundant or noisy, leading to "the curse of dimensionality."
  2. Class Imbalance: Most borrowers pay on time; only a minority prepay. Standard machine learning models tend to ignore this minority class to achieve high overall accuracy, which makes them useless for risk management.

Methods

The authors propose a two-stage pipeline to tackle these issues:

1. Hybrid Feature Selection

Instead of relying on a single selection method, the paper combines two:

  • Pearson Correlation (Filter Method): This identifies and removes highly redundant features. For example, the authors found that installment and loan_amnt had a 0.95 correlation, meaning one could be safely discarded.
  • SVM-RFE (Wrapper Method): Support Vector Machine Recursive Feature Elimination ranks the remaining features by training an SVM and iteratively removing the least important ones. This ensures the final subset is optimized for predictive power.

2. XGBoost with Weighted Cross-Entropy (XGBoost_WCE)

The core innovation lies in modifying the XGBoost loss function. Standard cross-entropy treats a "miss" on a majority-class sample the same as a "miss" on a minority-class sample. The authors introduced a weighting coefficient ($\beta$) into the binary classification loss function. This forces the model to pay a higher "penalty" for misclassifying prepayments, effectively making the model more sensitive to the minority class without needing to artificially oversample the data (like SMOTE).

Datasets

The study utilized real-world data from LendingClub, a major US P2P lending platform.

  • Scope: 36-month personal loans from 2015.
  • Sample Size: 165,636 records.
  • Class Ratio: 1:3 (Normal repayment to Early repayment).
  • Preprocessing: The authors performed extensive cleaning, including removing missing values and irrelevant variables, before splitting the data 3:1 for training and testing.

Key Insights and Ideas

  • Less is More: By using the hybrid selection method, the authors reduced the feature set from 25 to 15 variables. Surprisingly, this reduction improved every performance metric, proving that noise reduction is more valuable than raw data volume in financial modeling.
  • Algorithmic vs. Data Balancing: Many researchers try to fix imbalance by duplicating minority samples. This paper shows that modifying the loss function is a more "principled" approach that preserves the integrity of the original data distribution while achieving superior results.
  • The Redundancy Trap: The authors highlight that while SVM-RFE is great at finding relevant features, it doesn't account for correlation between them. Adding the Pearson filter first solves this "blind spot."

Key Results

The proposed XGBoost_WCE model was compared against standard XGBoost, LightGBM, and CatBoost. The results were decisive:

  • Recall: The ability to catch prepayments jumped to 0.9237, significantly higher than standard XGBoost (0.8005).
  • AUC (Area Under Curve): The model achieved an AUC of 0.9501, outperforming all other integrated learning algorithms.
  • G-mean: This metric, which measures the balance between classification accuracy of both classes, reached 0.9061, compared to 0.8594 for standard XGBoost and ~0.78 for LightGBM/CatBoost.

Limitations and Future Work

While the model is highly effective at binary classification (will they prepay or not?), the authors acknowledge a few areas for growth:

  • Temporal Resolution: The current model doesn't predict when the prepayment will occur. Future work could involve multi-category forecasting to divide the loan term into specific periods.
  • Data-Level Solutions: The authors focused on algorithmic improvements; they suggest that future research could combine their weighted loss function with data-level balancing techniques to see if further gains are possible.
  • Commercial Valuation: The paper suggests transforming these predictions into a level of "commercial value" to help platforms price loans more dynamically.

Summary updated

Abstract

The problem of large dimensionality of loan data and unbalanced data samples severely affects the classification, and the article proposes a support vector machine for feature recursive elimination and XGBoost for a loan early repayment prediction. Firstly, the combination of Pearson index and SVM-R
BibTeX
@inproceedings{mao2022loan,
  author = {Qi Mao and Gang Liu and Zhiyu Chen and Jianwei Guo and Peng Liu},
  title = {{Loan Prepayment Prediction Based on SVM-RFE and XGBoost Models}},
  booktitle = {Proceedings of the International Conference on Information Economy, Data Modeling and Cloud Computing, ICIDC 2022, 17-19 June 2022, Qingdao, China},
  year = {2022},
  doi = {10.4108/eai.17-6-2022.2322765},
}
Figure