Implementation of an Asymmetric Adjusted Activation Function for Class Imbalance Credit Scoring

Xia Li, Hanghang Zheng, Kunpeng Tao, Mao MaoarXiv 2025 · DOI · arXiv

Introduction

In the financial technology sector, credit scoring represents a classic "needle in a haystack" challenge. While the vast majority of borrowers fulfill their repayment obligations, a critical minority—typically ranging from 0.5% to 2%—default. In machine learning terms, this manifests as a severe class imbalance problem, with Imbalance Ratios (IR) frequently falling between 50 and 190.

Standard binary classifiers generally rely on the Sigmoid activation function, which is mathematically symmetric. This symmetry implicitly assumes equal costs for misclassifying "good" versus "bad" borrowers. However, for financial institutions, the cost of failing to identify a default (a false negative) far outweighs the cost of a false positive. Traditional models often over-fit to the majority class, resulting in diminished predictive accuracy for the rare but high-impact positive samples.

Methodology

To address these limitations, this research introduces the Asymmetric adjusted factor embedded Sigmoid (ASIG). Unlike the standard Sigmoid function $P = 1 / (1 + \exp(-Z))$, ASIG incorporates a dynamic adjustment factor $G$ that shifts the activation curve along the x-axis:

$$P = \frac{1}{1 + \exp(-Z + G(IR, \alpha, \beta))}$$

The factor $G$ is not a static hyperparameter; it is functionally dependent on the dataset's Imbalance Ratio. The authors established a log-linear relationship: $G = \alpha \ln(IR) + \beta$. This "lateral displacement" effectively lowers the decision threshold for the minority class as the data becomes increasingly imbalanced.

Figure 1: The proposed ASIG-embedded classifier framework, illustrating the resampling, pre-training, and comparison stages.

The implementation involves a pre-training phase on a baseline dataset to optimize the hyperparameters $\alpha$ and $\beta$ by maximizing the Area Under the Curve (AUC). The resulting ASIG function is then integrated into a LightGBM classifier utilizing Focal Loss, allowing the model to prioritize "hard" minority samples while adapting to varying levels of data skewness.

Experimental Setup

The study evaluated the ASIG framework across four diverse financial datasets to ensure robustness across different data structures and origins.

Figure 3: Summary of the four financial datasets used in the study, including sample sizes, feature counts, and the range of imbalance ratios (IR) generated through resampling.

To rigorously test the model's performance under extreme conditions, the authors employed random undersampling to generate sub-datasets with IRs ranging from 20 to 300.

Key Insights and Results

The experimental results demonstrate that the ASIG-embedded LightGBM consistently outperforms traditional classifiers, including vanilla LightGBM, XGBoost, Random Forest, and KNN.

  • Performance Gains: On the GMC baseline dataset, ASIG achieved an average AUC improvement of 1.1% over standard LightGBM and 2.2% over XGBoost.
  • Robustness at High IR: The performance gap between ASIG and traditional models widened as the Imbalance Ratio increased. While standard models experienced sharp performance degradation at high IRs, ASIG maintained stability.
  • The "Breaking Point" Phenomenon: The research identified a critical threshold for model stability. Regardless of the IR, performance variance spikes significantly if the absolute count of positive samples (defaulters) falls below approximately 200.

Conclusion

The ASIG activation function offers a novel approach to class imbalance by shifting the position of the activation curve rather than merely altering the loss function's shape. By leveraging a logarithmic relationship with the Imbalance Ratio, ASIG provides a generalized solution for tabular financial data. Future work will focus on automating the optimization of $\alpha$ and $\beta$ within the primary training loop to further streamline the implementation process.

Summary updated

Abstract

Credit scoring is a systematic approach to evaluate a borrower's probability of default (PD) on a bank loan. The data associated with such scenarios are characteristically imbalanced, complicating binary classification owing to the often-underestimated cost of misclassification during the classifier's learning process. Considering the high imbalance ratio (IR) of these datasets, we introduce an innovative yet straightforward optimized activation function by incorporating an IR-dependent asymmetric adjusted factor embedded Sigmoid activation function (ASIG). The embedding of ASIG makes the sensitive margin of the Sigmoid function auto-adjustable, depending on the imbalance nature of the datasets distributed, thereby giving the activation function an asymmetric characteristic that prevents the underrepresentation of the minority class (positive samples) during the classifier's learning process. The experimental results show that the ASIG-embedded-classifier outperforms traditional classifiers on datasets across wide-ranging IRs in the downstream credit-scoring task. The algorithm also shows robustness and stability, even when the IR is ultra-high. Therefore, the algorithm provides a competitive alternative in the financial industry, especially in credit scoring, possessing the ability to effectively process highly imbalanced distribution data.
BibTeX
@misc{li2025implementation,
  author = {Xia Li and Hanghang Zheng and Kunpeng Tao and Mao Mao},
  title = {{Implementation of an Asymmetric Adjusted Activation Function for Class Imbalance Credit Scoring}},
  year = {2025},
  eprint = {2501.12285},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.12285},
}
Figure