Vertical Federated Learning without Revealing Intersection Membership

Jiankai Sun, Xin Yang, Yuanshun Yao, Aonan Zhang, Weihao Gao, Junyuan Xie, Chong WangarXiv 2021 · OpenReview

Abstract Vertical Federated Learning (vFL) allows multiple parties that own different attributes of the same data entity to jointly train a model. Traditionally, vFL relies on Private Set Intersection (PSI) to align data instances, which inadvertently reveals "intersection membership"—the knowledge of exactly which individuals are shared between organizations. In many privacy-sensitive sectors, revealing this membership is prohibited. We propose FLORIST, a vFL framework based on Private Set Union (PSU) that allows each party to keep sensitive membership information private. Instead of identifying the intersection, our PSU protocol generates the union of samples as training instances. We introduce strategies to generate synthetic features and labels to handle samples that belong to the union but not the intersection. Extensive experiments on real-world datasets show that FLORIST protects intersection membership privacy while maintaining high model utility.

Introduction

In Vertical Federated Learning, organizations like banks and e-commerce sites often hold complementary attributes for a shared set of users. To train a joint model, they must first align their records. The standard approach uses Private Set Intersection (PSI) to identify common IDs. While PSI hides the underlying data values, it explicitly reveals the identities of users in the intersection. For example, a pharmacy should not necessarily know which of its customers are also patients at a specific hospital.

This "membership leakage" is a significant barrier to vFL adoption in highly regulated industries. The challenge lies in training a model effectively when parties are not allowed to know which specific records actually match. FLORIST addresses this by shifting the paradigm from intersection-based alignment to union-based alignment.

Methods: The FLORIST Framework

The FLORIST (Federated Learning withOut Revealing InterSecTions) framework consists of two primary components designed to mask the intersection while enabling joint training.

1. Private Set Union (PSU) for ID Alignment

Instead of finding the intersection, parties utilize a novel PSU protocol based on Diffie-Hellman key exchange. This protocol generates a set of universal identifiers (UIDs) representing the union of all samples from all parties. Each party knows which UIDs correspond to their own records, but they cannot distinguish whether a UID they do not own belongs solely to another party or is part of a hidden intersection. This creates a "crowd" in which the intersection is effectively hidden.

Overview of the FLORIST framework, illustrating PSU-based ID alignment and synthetic data generation.

2. Synthetic Data Generation and Calibration

Training on the union introduces "holes" where data is missing. For a given UID, the Active Party (holding labels) might be missing the label, or the Passive Party (holding features) might be missing the features. FLORIST employs two main strategies:

  • Label-Majority: In imbalanced datasets (common in CTR prediction), missing labels are assigned the majority class (e.g., '0').
  • Fea-Sampling: The passive party fills missing features by sampling from its own real feature distribution to mimic the marginal distribution of real data.

Because training on synthetic data shifts the predicted probability distribution, the authors apply a logits shift technique to calibrate the model, ensuring final predictions remain accurate despite the noisy training process.

Experimental Evaluation

FLORIST was evaluated on two massive real-world benchmarks: Criteo (~45 million records) and Avazu (~40 million records). Both are highly imbalanced binary classification tasks for click-through rate (CTR) prediction.

Utility and Privacy Trade-offs

The framework demonstrates remarkable utility preservation. On the Criteo dataset, even when 95% of the labels were synthetic, the Test AUC only dropped by approximately 0.67% after proper calibration. As shown in the performance analysis, the fea-sampling strategy maintains a high AUC even as the ratio of synthetic features increases.

Test AUC performance as a function of the synthetic feature ratio for various generation strategies.

To verify privacy, the authors employed "Spectral Attacks" (SVD-based outlier detection). While standard feature generation methods often lead to total membership leakage (Attack AUC near 1.0), FLORIST’s strategies kept the Attack AUC between 0.5 and 0.57, indicating that an adversary is essentially guessing.

Conclusion

FLORIST provides a robust solution for vFL in privacy-constrained environments by replacing PSI with PSU and utilizing intelligent synthetic data generation. While training on the union increases computational overhead due to larger dataset sizes, the framework successfully mitigates membership leakage without significantly sacrificing model performance. Future work will focus on extending these synthetic strategies to balanced and multi-class datasets, as well as improving scalability for multi-party scenarios.

Summary updated

Abstract

Vertical Federated Learning (vFL) allows multiple parties that own different attributes (e.g. features and labels) of the same data entity (e.g. a person) to jointly train a model. To prepare the training data, vFL needs to identify the common data entities shared by all parties. It is usually achieved by Private Set Intersection (PSI) which identifies the intersection of training samples from all parties by using personal identifiable information (e.g. email) as sample IDs to align data instances. As a result, PSI would make sample IDs of the intersection visible to all parties, and therefore each party can know that the data entities shown in the intersection also appear in the other parties, i.e. intersection membership. However, in many real-world privacy-sensitive organizations, e.g. banks and hospitals, revealing membership of their data entities is prohibited. In this paper, we propose a vFL framework based on Private Set Union (PSU) that allows each party to keep sensitive membership information to itself. Instead of identifying the intersection of all training samples, our PSU protocol generates the union of samples as training instances. In addition, we propose strategies to generate synthetic features and labels to handle samples that belong to the union but not the intersection. Through extensive experiments on two real-world datasets, we show our framework can protect the privacy of the intersection membership while maintaining the model utility.
BibTeX
@misc{sun2021vertical,
  author = {Jiankai Sun and Xin Yang and Yuanshun Yao and Aonan Zhang and Weihao Gao and Junyuan Xie and Chong Wang},
  title = {{Vertical Federated Learning without Revealing Intersection Membership}},
  year = {2021},
  eprint = {2106.05508},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2106.05508},
}
Figure