Heterogeneous Domain Adaptation With Adversarial Neural Representation Learning: Experiments on E-Commerce and Cybersecurity

Mohammadreza Ebrahimi, Yidong Chai, Hao Helen Zhang, Hsinchun Chen

Abstract

Learning predictive models in new domains with scarce training data is a growing challenge in modern supervised learning scenarios. This incentivizes developing domain adaptation methods that leverage the knowledge in known domains (source) and adapt to new domains (target) with a different probability distribution. This becomes more challenging when the source and target domains are in heterogeneous feature spaces, known as heterogeneous domain adaptation (HDA). While most HDA methods utilize mathematical optimization to map source and target data to a common space, they suffer from low transferability. Neural representations have proven to be more transferable; however, they are mainly designed for homogeneous environments. Drawing on the theory of domain adaptation, we propose a novel framework, Heterogeneous Adversarial Neural Domain Adaptation (HANDA), to effectively maximize the transferability in heterogeneous environments. HANDA conducts feature and distribution alignment in a unified neural network architecture and achieves domain invariance through adversarial kernel learning. Three experiments were conducted to evaluate the performance against the state-of-the-art HDA methods on major image and text e-commerce benchmarks. HANDA shows statistically significant improvement in predictive performance. The practical utility of HANDA was shown in real-world dark web online markets. HANDA is an important step towards successful domain adaptation in e-commerce applications.

Introduction: The Challenge of Heterogeneous Domain Adaptation

In supervised learning, the scarcity of labeled data in target domains often necessitates the use of Domain Adaptation (DA) to leverage knowledge from a data-rich source domain. While standard DA assumes homogeneous feature spaces, real-world applications frequently encounter Heterogeneous Domain Adaptation (HDA). In HDA, the source and target domains differ not only in their distributions but also in their fundamental feature representations—for instance, transferring knowledge from English text to Russian text, or from low-level image descriptors to deep neural features.

Conceptual overview of Heterogeneous Domain Adaptation (HDA) where source and target domains reside in different feature spaces.

Existing HDA methods primarily rely on shallow mathematical optimizations to find a common latent space. However, these approaches often fail to capture the complex, non-linear relationships required for high transferability in sophisticated tasks like image recognition or multilingual sentiment analysis.

The HANDA Framework

To address these limitations, we propose Heterogeneous Adversarial Neural Domain Adaptation (HANDA). HANDA is a unified neural architecture designed to perform feature alignment and distribution alignment simultaneously, rather than sequentially.

The architecture of the HANDA framework, integrating shared dictionary learning, adversarial kernel matching, and a shared classifier.

The framework consists of three core components:

  1. Shared Dictionary Learning (SDL): This module utilizes domain-specific encoders to project heterogeneous source and target features into a shared latent embedding space.
  2. Nonparametric Adversarial Kernel Matching: To align the distributions within the shared space, HANDA employs a minimax game based on Maximum Mean Discrepancy (MMD). A critic network ($M$) attempts to find a kernel that maximizes the distance between domain distributions, while the alignment network ($N$) learns representations that minimize this distance, ensuring domain invariance.
  3. Shared Classifier: A final classification layer is trained on the aligned representations using labeled data from both domains, ensuring the learned features remain discriminative for the target task.

Experimental Evaluation

We evaluated HANDA across three diverse benchmarks to test its robustness:

  • Reuters Multilingual Collection: A text dataset covering five languages (English, French, German, Italian, Spanish). HANDA achieved an average accuracy improvement of ~2% over baseline HDA methods.
  • Office31 – Caltech256: An image benchmark involving transitions between SURF and DeCAF features. HANDA improved average recognition accuracy by 1.1%.
  • Dark Web Online Markets: A real-world cybersecurity application involving over 100,000 listings across nine illegal markets. HANDA demonstrated significant utility, increasing the Area Under the Curve (AUC) for threat detection by 3% in Russian markets and 4% in French markets.

Key Insights and Discussion

A critical finding of this research is the superiority of simultaneous alignment. By training the feature projection and distribution matching components together, the classification loss directly informs the alignment process, leading to more robust transferability.

Furthermore, the Adversarial MMD approach proved more effective than fixed-kernel methods. By allowing a neural network to "search" for domain discrepancies, HANDA can close complex distribution gaps that shallow methods miss. This was particularly evident in "zero-shot" scenarios within the Dark Web dataset, where the model successfully identified specific threat categories (like Trojans) in the French domain despite having no French training examples for that category.

Conclusion

HANDA represents a significant advancement in HDA by combining the transferability of neural representations with the robustness of adversarial learning. While future work may explore adding attention mechanisms for better interpretability or accounting for temporal shifts in dynamic markets, HANDA currently provides a powerful tool for cross-domain knowledge transfer in complex e-commerce and cybersecurity environments.

Summary updated

Abstract

Learning predictive models in new domains with scarce training data is a growing challenge in modern supervised learning scenarios. This incentivizes developing domain adaptation methods that leverage the knowledge in known domains (source) and adapt to new domains (target) with a different probability distribution. This becomes more challenging when the source and target domains are in heterogeneous feature spaces, known as heterogeneous domain adaptation (HDA). While most HDA methods utilize mathematical optimization to map source and target data to a common space, they suffer from low transferability. Neural representations have proven to be more transferable; however, they are mainly designed for homogeneous environments. Drawing on the theory of domain adaptation, we propose a novel framework, Heterogeneous Adversarial Neural Domain Adaptation (HANDA), to effectively maximize the transferability in heterogeneous environments. HANDA conducts feature and distribution alignment in a unified neural network architecture and achieves domain invariance through adversarial kernel learning. Three experiments were conducted to evaluate the performance against the state-of-the-art HDA methods on major image and text e-commerce benchmarks. HANDA shows statistically significant improvement in predictive performance. The practical utility of HANDA was shown in real-world dark web online markets. HANDA is an important step towards successful domain adaptation in e-commerce applications.
BibTeX
@article{ebrahimi2022heterogeneous,
  author = {Mohammadreza Ebrahimi and Yidong Chai and Hao Helen Zhang and Hsinchun Chen},
  title = {{Heterogeneous Domain Adaptation With Adversarial Neural Representation Learning: Experiments on E-Commerce and Cybersecurity}},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2022},
  doi = {10.1109/tpami.2022.3163338},
}
Figure