Abstract
PIVODL is a novel framework designed to address a fundamental limitation in Vertical Federated Learning (VFL): the assumption that all ground-truth labels reside with a single "active" party. In many real-world scenarios, labels are fragmented across multiple organizations. PIVODL enables the training of Gradient Boosting Decision Trees (GBDT) in this distributed setting while protecting against sensitive label leakage. By integrating additive homomorphic encryption, a unique "source-split" client protocol, and partial differential privacy, PIVODL maintains high model accuracy while significantly reducing the risk of differential attacks.
Introduction
Vertical Federated Learning (VFL) has emerged as a powerful paradigm for collaborative machine learning, allowing organizations with different feature sets for the same user base to train models without sharing raw data. However, existing VFL systems typically rely on a "Guest" party that holds all the labels. This paper introduces PIVODL to solve the challenges that arise when labels are scattered across multiple clients, a common occurrence in sectors like finance and healthcare.
The Distributed Label Challenge
In a standard VFL setup, computing the gradients and Hessians necessary for GBDT training is straightforward because one party has all the necessary information. When labels are distributed, these calculations become fragmented. Simply sharing labels is not an option due to privacy regulations and competitive concerns.
As shown in the figure above, different clients ($u_1, u_2, u_3$) may hold partial labels for overlapping sets of user IDs. Standard VFL fails in this environment because it cannot easily aggregate the necessary statistics for tree splitting without compromising privacy.
Security Vulnerabilities: Differential Attacks
Beyond the logistical challenge of distributed labels, VFL faces significant privacy risks. Even when data is encrypted, an adversary can perform a "differential attack." By analyzing the differences in gradient sums between adjacent feature splits, a malicious participant can isolate the gradient of a single individual and deduce their private label.
PIVODL addresses this by decoupling the roles of data ownership and mathematical computation. It introduces a Source-Split Client protocol where a "Source Client" owns the feature data but a randomly selected "Split Client" performs the impurity gain calculations. Because the Split Client does not know which user IDs correspond to the data points they are processing, the differential attack vector is effectively neutralized.
Methodology
The PIVODL framework employs a multi-layered security architecture:
- Additive Homomorphic Encryption (HE): Using the Paillier cryptosystem, clients can aggregate encrypted gradients and Hessians. This allows for the calculation of global statistics without any party seeing individual decrypted values.
- Partial Differential Privacy (DP): To balance utility and security, PIVODL applies Gaussian noise specifically to the final leaf weights before they are transmitted. This "Partial DP" approach prevents clients from reverse-engineering labels from the final model while maintaining high accuracy.
- Instance Thresholding: The protocol automatically rejects any potential split that contains fewer than a minimum number of samples, ensuring that no individual can be isolated within a small leaf node.
Experimental Evaluation
The authors evaluated PIVODL on three benchmarks: Credit Card (classification), Bank Marketing (classification), and Appliances Energy Prediction (regression). The datasets were partitioned to simulate a distributed label environment.
Key Results
- Accuracy: PIVODL demonstrated negligible performance degradation. On the Credit Card dataset, it achieved approximately 82% accuracy, nearly identical to non-private centralized models.
- Security: The framework proved highly effective against label-guessing attacks. Without protection, an attacker could guess labels with ~68% accuracy. With PIVODL’s Partial DP, this dropped to between 14% and 40%, making the attack less effective than a random coin flip (50%).
- Scalability: While encryption introduces computational overhead—increasing training time from ~350 seconds to ~800 seconds for six trees on the Energy dataset—the communication overhead remains manageable at approximately 25MB for small models.
Conclusion
PIVODL represents a significant step forward for VFL, making it viable for industries where labels are as distributed as the features themselves. By combining role separation with robust cryptographic and differential privacy techniques, it provides a secure path for collaborative GBDT training. Future research will focus on reducing the computational cost of homomorphic encryption to support massive, real-time datasets.
Summary updated