Abstract
This study proposes a method for imbalanced data classification based on deep probabilistic graphical models (DPGMs) to solve the problem that traditional methods have insufficient learning ability for minority class samples. To address the classification bias caused by class imbalance, we introduce variational inference optimization probability modeling, which enables the model to adaptively adjust the representation ability of minority classes and combines the class-aware weight adjustment strategy to enhance the classifier's sensitivity to minority classes. In addition, we combine the adversarial learning mechanism to generate minority class samples in the latent space so that the model can better characterize the category boundary in the high-dimensional feature space. The experiment is evaluated on the Kaggle "Credit Card Fraud Detection" dataset and compared with a variety of advanced imbalanced classification methods (such as GAN-based sampling, BRF, XGBoost-Cost Sensitive, SAAD, HAN). The results show that the method in this study has achieved the best performance in AUC, Precision, Recall and F1-score indicators, effectively improving the recognition rate of minority classes and reducing the false alarm rate.
Introduction
In real-world applications such as financial fraud detection, medical diagnosis, and cybersecurity, datasets are often characterized by extreme class imbalance. For instance, in a typical credit card transaction dataset, fraudulent activities might represent less than 0.2% of the total volume. Standard machine learning models, which prioritize overall accuracy, often fail to detect these rare but critical events, effectively treating them as noise. While traditional methods like oversampling (e.g., SMOTE) or cost-sensitive learning exist, they often suffer from overfitting or require extensive manual tuning. This study proposes a robust alternative using deep probabilistic graphical models (DPGMs).
Methodology
The proposed framework leverages the generative power of DPGMs to capture the underlying structure of minority classes through a multi-layered approach.
Deep Probabilistic Graphical Models (DPGMs)
Unlike deterministic classifiers, DPGMs model the joint probability distribution of the data. By introducing latent variables, the model learns a compressed representation of the input features, allowing it to generalize better from sparse data.
Variational Inference and Adversarial Learning
To handle the computational complexity of posterior distributions, the authors employ Variational Inference (VI) to optimize the Evidence Lower Bound (ELBO). This ensures the model accurately reconstructs data while regularizing the latent space. To further refine the decision boundaries, a Latent Adversarial Learning mechanism is introduced. A discriminator operates in the latent space, forcing the generator to produce more distinct and separable feature representations for the minority class.
Category-Aware Weighting
A dynamic weighting strategy is applied to the loss function. By scaling the loss based on the ratio of majority samples to minority samples, the model is penalized more heavily for misclassifying minority samples, effectively increasing the "signal" from rare events during training.
Experimental Setup
The model was rigorously tested on the Kaggle Credit Card Fraud Detection dataset, which contains 284,807 transactions with a fraud ratio of approximately 1:577. The data includes 30 numerical features derived from PCA. The dataset was split into training (70%), validation (15%), and testing (15%) sets to ensure robust evaluation.
Results and Discussion
The DPGM-based approach demonstrated superior performance across all key metrics compared to industry-standard baselines such as GAN-based sampling, XGBoost (Cost-Sensitive), and advanced attention models like HAN.
| Metric | Ours (DPGM) | HAN [26] | XGBoost-CS [24] |
|---|---|---|---|
| AUC | 0.941 | 0.927 | 0.903 |
| Precision | 0.822 | 0.806 | 0.779 |
| Recall | 0.785 | 0.768 | 0.735 |
| F1-Score | 0.803 | 0.786 | 0.757 |
A critical component of this success is the model's ability to create a well-separated latent space. As shown in the T-SNE visualization below, the model successfully clusters fraudulent transactions (red) away from legitimate ones (blue), despite the massive class disparity.
Conclusion
By combining probabilistic modeling with adversarial learning and variational inference, this research provides a powerful tool for imbalanced classification. The method not only improves detection rates but also offers better uncertainty modeling compared to heuristic resampling. Future work will focus on reducing computational overhead and exploring federated learning to enhance privacy in cross-institutional fraud detection.
Summary updated