Abstract
Domain adaptation (DA) typically assumes a monolithic shift between source and target distributions. However, real-world data often comprises multiple "hidden subdomains" with distinct characteristics. This paper introduces HSAV (Hidden Subdomain Adaptation with Variable Number of Subdomains), a model-agnostic framework designed to adapt target data to pre-trained "black-box" source models. HSAV automatically discovers hidden subdomains, aligns them through sparse mappings, and handles uncertainty in subdomain counts. Experimental results on large-scale fraud detection datasets demonstrate significant performance gains and enhanced interpretability compared to state-of-the-art deep learning methods.
Introduction
In many industrial machine learning applications, practitioners rely on high-performing "black-box" models trained on historical data. When new data arrives with shifted distributions—a phenomenon known as domain adaptation—these models often become unreliable. A significant challenge in this context is that real-world data is rarely uniform; it is often a collection of "hidden subdomains," such as different user demographics or seasonal behavioral shifts, for which labels are unavailable.
Traditional DA techniques often require retraining the source model or assume a single global transformation. However, in many settings, retraining is impossible due to the complexity of legacy ensembles (e.g., mixtures of SVMs and decision trees) or the loss of original training expertise. The HSAV framework addresses this by transforming target data to match the source model's expectations without modifying the model itself.
The HSAV Framework
HSAV operates on three key pillars to bridge the gap between source and target domains while maintaining interpretability:
- Target-to-Source Transformation: HSAV is model-agnostic because it transforms the target data to "look like" the source data. This allows it to work with any pre-trained classifier, from neural networks to gradient-boosted trees.
- Automated Subdomain Discovery: The method identifies internal data shifts by maximizing inter-subdomain divergence. This is particularly effective for temporal data, where time can serve as a primary axis for discovering seasonal or gradual shifts.
- Variable Subdomain Numbers and Sparse Mapping: Unlike previous methods that require a fixed number of subdomains, HSAV uses a reweighting matrix ($A$) to aggregate predictions across multiple possible subdomain counts. Furthermore, it employs a sparse mapping matrix ($S$) to align target subdomains with source subdomains, providing clear insights into how different data segments relate to one another.
Experimental Evaluation
The authors evaluated HSAV on two large-scale tabular datasets focused on fraud detection: the Kaggle Fraud Detection Dataset and a private industrial dataset containing millions of transactions from Europe. These datasets were chosen to simulate real-world temporal drift and device-based subdomain shifts.
Key Results
- Performance Gains: On the Kaggle dataset, HSAV achieved an average improvement of 18.24% in PR AUC (Precision-Recall Area Under Curve) over a baseline with no adaptation.
- Superiority over Deep Learning: HSAV consistently outperformed popular deep domain adaptation methods like DANN and DAN. This is particularly notable on tabular data, where traditional deep learning models often struggle compared to the tree-based models (like LightGBM) that HSAV can easily wrap.
- Stability and Interpretability: The framework showed lower variance across random states compared to adversarial methods. Its sparse mapping provides "subdomain-level" interpretability, allowing practitioners to verify which "new" data groups are being treated like specific "old" groups.
Conclusion and Future Work
HSAV provides a robust, interpretable solution for adapting legacy models to multi-faceted data shifts. By treating domains as collections of hidden sub-structures rather than monoliths, it achieves higher precision in high-stakes fields like finance. While currently limited by the quadratic growth of adaptation functions relative to subdomain counts, future research will focus on scaling the method to handle hundreds of micro-clusters and exploring multi-dimensional discovery axes beyond time.
Summary updated