Locality Preserving Joint Transfer for Domain Adaptation

Jingjing Li, Mengmeng Jing, Ke Lu, Lei Zhu, Heng Tao Shen

Introduction

Domain Adaptation (DA) is a critical challenge in machine learning, aimed at bridging the gap between a "source" domain with abundant labeled data and a "target" domain where labels are scarce or unavailable. Traditional approaches often suffer from two primary limitations: they either focus exclusively on feature-level distribution alignment or sample-level re-weighting, and they frequently overlook the intrinsic manifold structure of the data. Furthermore, many existing methods are restricted to homogeneous scenarios where source and target features share the same dimensionality.

To address these issues, the Locality Preserving Joint Transfer (LPJT) framework is proposed. LPJT simultaneously aligns feature distributions and selects representative "landmark" samples while preserving the local geometric structure of the data. This joint optimization ensures a more robust transfer, particularly in heterogeneous settings.

The LPJT Framework

The core of LPJT lies in its unified objective function, which integrates three synergistic components:

  1. Feature-Level Distribution Matching: Using Maximum Mean Discrepancy (MMD), LPJT minimizes the distance between source and target distributions in a shared latent space. By learning separate projection matrices for each domain, it seamlessly handles heterogeneous data.
  2. Sample-Level Landmark Selection: The model assigns weights to source samples. "Landmarks"—samples that closely resemble the target distribution—receive higher weights, acting as a bridge for knowledge transfer, while outliers are suppressed.
  3. Locality Preservation: To maintain the data's manifold structure, LPJT employs Laplacian graphs. An intrinsic graph preserves the proximity of similar samples, while a penalty graph ensures that samples from different classes remain distinct in the projected space.
Overview of the LPJT process: (a) original space, (b) distribution matching, and (c) sample re-weighting to select landmarks.

The interaction between these components is iterative. Improved feature alignment facilitates the identification of better landmarks, which in turn leads to more precise distribution matching. This "jointness" is a key driver of the model's performance.

Experimental Evaluation

LPJT was evaluated across several diverse benchmarks, including Office+Caltech, CMU PIE, Office-Home, and the large-scale VisDA 2017 challenge.

On the CMU PIE face recognition dataset, LPJT achieved an average accuracy of 82.36%, significantly outperforming the previous state-of-the-art baseline, JGSA (62.74%). In the Office+Caltech benchmark, LPJT surpassed popular deep learning models such as DAN and DDC, demonstrating that sophisticated subspace learning can be more effective than end-to-end neural networks in certain DA scenarios.

Visualization of the learned latent space, showing well-separated clusters corresponding to different classes after adaptation.

The model also exhibits high computational efficiency, typically converging in fewer than 10 iterations. This makes it viable for large-scale applications like VisDA, where it successfully transfers knowledge from synthetic 2D renderings to real-world photography.

Discussion and Conclusion

A significant insight from this work is that geometry acts as a safeguard against "negative transfer." By enforcing locality preservation, LPJT prevents the alignment process from collapsing different classes into the same region of the latent space. Additionally, the framework shows that handcrafted features (e.g., SURF) can complement modern deep features, suggesting that traditional computer vision techniques still hold value in the era of deep learning.

While LPJT is powerful, it currently requires manual hyperparameter tuning to balance locality and variance. Future work may focus on automating this process or integrating locality-preserving mechanisms directly into deep neural network architectures. In summary, LPJT provides a robust, efficient, and versatile solution for both homogeneous and heterogeneous domain adaptation.

Summary updated

Abstract

Domain adaptation aims to leverage knowledge from a well-labeled source domain to a poorly-labeled target domain. A majority of existing works transfer the knowledge at either feature level or sample level. Recent researches reveal that both of the paradigms are essentially important, and optimizing one of them can reinforce the other. Inspired by this, we propose a novel approach to jointly exploit feature adaptation with distribution matching and sample adaptation with landmark selection. During the knowledge transfer, we also take the local consistency between samples into consideration, so that the manifold structures of samples can be preserved. At last, we deploy label propagation to predict the categories of new instances. Notably, our approach is suitable for both homogeneous and heterogeneous domain adaptation by learning domain-specific projections. Extensive experiments on five open benchmarks, which consist of both standard and large-scale datasets, verify that our approach can significantly outperform not only conventional approaches but also end-to-end deep models. The experiments also demonstrate that we can leverage handcrafted features to promote the accuracy on deep features by heterogeneous adaptation.
BibTeX
@article{li2019locality,
  author = {Jingjing Li and Mengmeng Jing and Ke Lu and Lei Zhu and Heng Tao Shen},
  title = {{Locality Preserving Joint Transfer for Domain Adaptation}},
  journal = {IEEE Trans. Image Process.},
  year = {2019},
  doi = {10.1109/tip.2019.2924174},
}
Figure