Joint Information Preservation for Heterogeneous Domain Adaptation

Peng Xu, Zhaohong Deng, Kup‐Sze Choi, Jun Wang, Shitong Wang

Abstract

Domain adaptation aims to assist the modeling tasks of the target domain with knowledge of the source domain. The two domains often lie in different feature spaces due to diverse data collection methods, which leads to the more challenging task of heterogeneous domain adaptation (HDA). A core issue of HDA is how to preserve the information of the original data during adaptation. In this paper, we propose a joint information preservation method to deal with the problem. The method preserves the information of the original data from two aspects. On the one hand, although paired samples often exist between the two domains of the HDA, current algorithms do not utilize such information sufficiently. The proposed method preserves the paired information by maximizing the correlation of the paired samples in the shared subspace. On the other hand, the proposed method improves the strategy of preserving the structural information of the original data, where the local and global structural information are preserved simultaneously. Finally, the joint information preservation is integrated by distribution matching. Experimental results show the superiority of the proposed method over the state-of-the-art HDA algorithms.

Introduction

Domain Adaptation (DA) has become a cornerstone of modern machine learning, allowing models trained on a "source" dataset to generalize to a related "target" dataset. However, Heterogeneous Domain Adaptation (HDA) presents a significantly steeper challenge: the source and target domains often possess entirely different feature dimensions—for instance, matching high-dimensional image descriptors with lower-dimensional text vectors.

Existing HDA methods often struggle with two primary issues: the neglect of explicit paired data (such as an image and its corresponding caption) and the incomplete preservation of data structure during the projection into a shared subspace. Most current algorithms focus on either local neighborhood relationships or global class-wide structures, but rarely both. This paper introduces the Joint Information Preservation (JIP) framework to bridge these gaps.

The JIP Methodology

The JIP algorithm constructs a shared subspace using symmetric transformation matrices, optimized through a unified objective function comprising three pillars:

  1. Distribution Matching: Utilizing Joint Distribution Adaptation (JDA), the model minimizes the distance between marginal and conditional distributions across domains using Maximum Mean Discrepancy (MMD).
  2. Paired Information Preservation: By integrating Canonical Correlation Analysis (CCA), JIP ensures that paired samples from disparate domains are projected as close as possible in the shared subspace, using them as "anchors" for alignment.
  3. Hierarchical Structural Preservation:
    • Local Structure: Employs Locality Preserving Projections (LPP) to maintain original neighborhood relationships.
  4. Global Structure: Uses Linear Discriminative Analysis (LDA) to maximize inter-class distance while minimizing intra-class variance.

The resulting optimization is formulated as a generalized eigenvalue decomposition problem, solved iteratively using a pseudo-label refinement strategy to handle unlabeled target data.

Experimental Evaluation

We evaluated JIP on three diverse benchmarks: Caltech-Office (image classification), IXMAS (cross-view action recognition), and WIKI (multi-modal retrieval). JIP consistently outperformed state-of-the-art methods such as TIT, TNT, and CDLS.

On the Caltech-Office dataset, JIP achieved an average accuracy of 82.05%. In the IXMAS dataset, JIP reached a peak accuracy of 80.38% across 20 cross-view tasks, demonstrating its robustness in handling perspective shifts.

Comparison of JIP against state-of-the-art HDA algorithms on the IXMAS dataset.

In the WIKI dataset experiments, the importance of paired data was evident: as the proportion of paired samples increased from 10% to 40%, JIP’s performance scaled significantly, reaching an average accuracy of 73.99%.

Ablation and Sensitivity Analysis

A critical aspect of JIP is the synergy between its components. Ablation studies confirmed that removing any single element—paired information, local structure, or global structure—resulted in a measurable drop in accuracy. This validates the "joint" philosophy of the framework.

Ablation study showing the performance gain when incorporating joint information preservation components (alpha and lambda).

While highly effective, the model does exhibit sensitivity to its regularization hyperparameters ($\alpha, \beta, \lambda$), which currently require grid search optimization. This remains an area for future refinement.

Conclusion

Joint Information Preservation (JIP) offers a comprehensive solution for HDA by treating paired data and hierarchical structures as essential components of the adaptation process. By maximizing correlation and preserving both local and global integrity, JIP sets a new standard for accuracy in heterogeneous environments. Future work will focus on developing adaptive strategies to automate hyperparameter selection, further enhancing the model's usability in fully unsupervised settings.

Summary updated

Abstract

Domain adaptation aims to assist the modeling tasks of the target domain with knowledge of the source domain. The two domains often lie in different feature spaces due to diverse data collection methods, which leads to the more challenging task of heterogeneous domain adaptation (HDA). A core issue of HDA is how to preserve the information of the original data during adaptation. In this paper, we propose a joint information preservation method to deal with the problem. The method preserves the information of the original data from two aspects. On the one hand, although paired samples often exist between the two domains of the HDA, current algorithms do not utilize such information sufficiently. The proposed method preserves the paired information by maximizing the correlation of the paired samples in the shared subspace. On the other hand, the proposed method improves the strategy of preserving the structural information of the original data, where the local and global structural information are preserved simultaneously. Finally, the joint information preservation is integrated by distribution matching. Experimental results show the superiority of the proposed method over the state-of-the-art HDA algorithms.
BibTeX
@misc{xu2019joint,
  author = {Peng Xu and Zhaohong Deng and Kup‐Sze Choi and Jun Wang and Shitong Wang},
  title = {{Joint Information Preservation for Heterogeneous Domain Adaptation}},
  year = {2019},
  eprint = {1905.08924},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1905.08924},
}
Figure