Differentially Private Optimal Transport: Application to Domain Adaptation

Nam Lê Tien, Amaury Habrard, Marc Sebban

Introduction

In modern machine learning, Domain Adaptation (DA) is a critical technique for transferring knowledge from a labeled "source" domain to an unlabeled "target" domain. Optimal Transport (OT) has emerged as a powerful framework for DA by finding the most efficient mapping between these two distributions. However, standard OT requires direct access to raw data from both domains, posing significant privacy risks when dealing with sensitive information like medical records or personal photos.

This paper introduces Differentially Private Optimal Transport (DPOT) and its application, Differentially Private Domain Adaptation (DPDA). These frameworks provide rigorous $(\epsilon, \delta)$-differential privacy guarantees while maintaining high utility for cross-domain tasks.

Methodology: DPOT and DPDA

The proposed framework addresses privacy by ensuring that the source data remains protected throughout the transport process.

Differentially Private Optimal Transport (DPOT)

The core of the approach lies in two steps:

  1. Random Projection: The source party applies a Johnson-Lindenstrauss (JL) transform to project high-dimensional data into a lower-dimensional subspace. This preserves pairwise distances while reducing the sensitivity of the data.
  2. Noise Addition: Gaussian noise is added to the projected data. To maintain accuracy, the authors implement a "de-biasing" step that subtracts the noise variance from the cost matrix, ensuring the calculated Wasserstein distance remains a faithful approximation of the true distance.

Differentially Private Domain Adaptation (DPDA)

DPDA extends DPOT to perform the actual adaptation. It utilizes a Private Barycentric Mapping to move source samples toward the target domain using only the privacy-protected coupling matrix. Furthermore, it employs Histogram Queries with Laplacian noise to handle source labels privately, ensuring that even the category distribution is not leaked.

Architecture of the DPOT and DPDA framework showing the flow from source data (Xs) to the target model output.

Experimental Evaluation

The authors evaluated DPDA across three standard benchmarks:

  • Office-Caltech: A classic dataset for object recognition across different domains (Amazon, DSLR, Webcam).
  • VisDA: A large-scale synthetic-to-real challenge with 200,000 images.
  • Office-Home: A diverse dataset featuring 65 object classes across Art, Clipart, Product, and Real-World domains.

Key Results and Insights

The experiments demonstrate that privacy does not necessarily come at the cost of performance.

Performance on Office-Caltech

On the Office-Caltech benchmark, DPDA achieved an average accuracy of 91.2%, which is remarkably close to the non-private baseline of 93.0%. This indicates that the noise introduced for privacy has a minimal impact on the model's ability to align distributions.

Accuracy comparison of DPDA against other private and non-private methods on the Office-Caltech dataset.

Comparison with State-of-the-Art

DPDA significantly outperforms existing private methods like PATE. For instance, on the VisDA dataset, DPDA reached 68.8% accuracy, surpassing PATE’s 60.6%. In semi-supervised settings with very few labeled target samples, DPDA closed the gap with non-private methods to within 1%.

The Role of Minibatches

A practical insight from the study is the efficiency of minibatch processing. By applying the "Privacy Accountant" technique to small subsets of data, the framework can achieve much lower privacy budgets ($\epsilon$)—and thus lower noise levels—without compromising the overall privacy guarantee.

Limitations and Future Directions

While highly effective, the method shows sensitivity to dataset size. On smaller subsets like DSLR, a higher privacy budget ($\epsilon=20$) was required to maintain performance. Additionally, the framework introduces hyperparameters such as projection dimensions and noise levels that require careful tuning.

Future work could involve extending DPOT to Federated Learning environments, where multiple parties collaborate on training without ever sharing local raw data, further broadening the scope of privacy-preserving machine learning.

Summary updated

Abstract

Optimal transport has received much attention during the past few years to deal with domain adaptation tasks. The goal is to transfer knowledge from a source domain to a target domain by finding a transportation of minimal cost moving the source distribution to the target one. In this paper, we address the challenging task of privacy preserving domain adaptation by optimal transport. Using the Johnson-Lindenstrauss transform together with some noise, we present the first differentially private optimal transport model and show how it can be directly applied on both unsupervised and semi-supervised domain adaptation scenarios. Our theoretically grounded method allows the optimization of the transportation plan and the Wasserstein distance between the two distributions while protecting the data of both domains.We perform an extensive series of experiments on various benchmarks (VisDA, Office-Home and Office-Caltech datasets) that demonstrates the efficiency of our method compared to non-private strategies.
BibTeX
@inproceedings{tien2019differentially,
  author = {Nam Lê Tien and Amaury Habrard and Marc Sebban},
  title = {{Differentially Private Optimal Transport: Application to Domain Adaptation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/395},
}
Figure