Interpretable Domain Adaptation for Hidden Subdomain Alignment in the Context of Pre-trained Source Models

Luxin Zhang, Pascal Germain, Yacine Kessaci, Christophe Biernacki

Abstract

Domain adaptation (DA) typically assumes a monolithic shift between source and target distributions. However, real-world data often comprises multiple "hidden subdomains" with distinct characteristics. This paper introduces HSAV (Hidden Subdomain Adaptation with Variable Number of Subdomains), a model-agnostic framework designed to adapt target data to pre-trained "black-box" source models. HSAV automatically discovers hidden subdomains, aligns them through sparse mappings, and handles uncertainty in subdomain counts. Experimental results on large-scale fraud detection datasets demonstrate significant performance gains and enhanced interpretability compared to state-of-the-art deep learning methods.

Introduction

In many industrial machine learning applications, practitioners rely on high-performing "black-box" models trained on historical data. When new data arrives with shifted distributions—a phenomenon known as domain adaptation—these models often become unreliable. A significant challenge in this context is that real-world data is rarely uniform; it is often a collection of "hidden subdomains," such as different user demographics or seasonal behavioral shifts, for which labels are unavailable.

Traditional DA techniques often require retraining the source model or assume a single global transformation. However, in many settings, retraining is impossible due to the complexity of legacy ensembles (e.g., mixtures of SVMs and decision trees) or the loss of original training expertise. The HSAV framework addresses this by transforming target data to match the source model's expectations without modifying the model itself.

The HSAV Framework

HSAV operates on three key pillars to bridge the gap between source and target domains while maintaining interpretability:

  1. Target-to-Source Transformation: HSAV is model-agnostic because it transforms the target data to "look like" the source data. This allows it to work with any pre-trained classifier, from neural networks to gradient-boosted trees.
  2. Automated Subdomain Discovery: The method identifies internal data shifts by maximizing inter-subdomain divergence. This is particularly effective for temporal data, where time can serve as a primary axis for discovering seasonal or gradual shifts.
  3. Variable Subdomain Numbers and Sparse Mapping: Unlike previous methods that require a fixed number of subdomains, HSAV uses a reweighting matrix ($A$) to aggregate predictions across multiple possible subdomain counts. Furthermore, it employs a sparse mapping matrix ($S$) to align target subdomains with source subdomains, providing clear insights into how different data segments relate to one another.
Subdomain weights and sparse mapping matrices illustrating the alignment between target and source hidden structures.

Experimental Evaluation

The authors evaluated HSAV on two large-scale tabular datasets focused on fraud detection: the Kaggle Fraud Detection Dataset and a private industrial dataset containing millions of transactions from Europe. These datasets were chosen to simulate real-world temporal drift and device-based subdomain shifts.

Key Results

  • Performance Gains: On the Kaggle dataset, HSAV achieved an average improvement of 18.24% in PR AUC (Precision-Recall Area Under Curve) over a baseline with no adaptation.
  • Superiority over Deep Learning: HSAV consistently outperformed popular deep domain adaptation methods like DANN and DAN. This is particularly notable on tabular data, where traditional deep learning models often struggle compared to the tree-based models (like LightGBM) that HSAV can easily wrap.
  • Stability and Interpretability: The framework showed lower variance across random states compared to adversarial methods. Its sparse mapping provides "subdomain-level" interpretability, allowing practitioners to verify which "new" data groups are being treated like specific "old" groups.

Conclusion and Future Work

HSAV provides a robust, interpretable solution for adapting legacy models to multi-faceted data shifts. By treating domains as collections of hidden sub-structures rather than monoliths, it achieves higher precision in high-stakes fields like finance. While currently limited by the quadratic growth of adaptation functions relative to subdomain counts, future research will focus on scaling the method to handle hundreds of micro-clusters and exploring multi-dimensional discovery axes beyond time.

Summary updated

Abstract

Domain adaptation aims to leverage source domain knowledge to predict target domain labels. Most domain adaptation methods tackle a single-source, single-target scenario, whereas source and target domain data can often be subdivided into data from different distributions in real-life applications (e.g., when the distribution of the collected data changes with time). However, such subdomains are rarely given and should be discovered automatically. To this end, some recent domain adaptation works seek separations of hidden subdomains, w.r.t. a known or fixed number of subdomains. In contrast, this paper introduces a new subdomain combination method that leverages a variable number of subdomains. Precisely, we propose to use an inter-subdomain divergence maximization criterion to exploit hidden subdomains. Besides, our proposition stands in a target-to-source domain adaptation scenario, where one exploits a pre-trained source model as a black box; thus, the proposed method is model-agnostic. By providing interpretability at two complementary levels (transformation and subdomain levels), our method can also be easily interpreted by practitioners with or without machine learning backgrounds. Experimental results over two fraud detection datasets demonstrate the efficiency of our method.
BibTeX
@inproceedings{zhang2022interpretable,
  author = {Luxin Zhang and Pascal Germain and Yacine Kessaci and Christophe Biernacki},
  title = {{Interpretable Domain Adaptation for Hidden Subdomain Alignment in the Context of Pre-trained Source Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i8.20890},
}

Loading related content...

Figure