Abstract
Vertical Federated Learning (VFL) is a collaborative machine learning paradigm designed to preserve privacy by allowing participants to train models on distributed features without sharing raw data. However, this paper reveals that VFL is significantly vulnerable to data leakage during the training phase. We introduce VFLRecon, a posterior-difference-based attack capable of reconstructing private labels and features with high accuracy, even when standard encryption techniques are employed. To counter these vulnerabilities, we propose VFLDefender, a gradient-obfuscation defense that protects sensitive information by randomizing gradient magnitudes while maintaining model utility. Our evaluations across six diverse datasets demonstrate that VFLRecon achieves up to 92% reconstruction accuracy, while VFLDefender effectively mitigates these attacks with negligible impact on performance.
Introduction
Federated Learning (FL) has emerged as a critical technology for privacy-preserving AI. While Horizontal FL (partitioning by samples) is well-documented, Vertical Federated Learning (VFL)—where participants hold different features for the same set of individuals—remains less explored regarding its security during training. The core promise of VFL is that sensitive local data, such as a bank's customer features or an e-commerce site's labels, never leaves the participant's premises.
This research identifies a critical gap: most privacy research focuses on the inference stage, leaving the training stage vulnerable. We demonstrate that "honest-but-curious" participants can exploit the mathematical relationship between model updates and training data to reverse-engineer the private information of their collaborators.
Methodology
The Attack: VFLRecon
VFLRecon is a posterior-difference-based attack founded on the principle that changes in a participant's local "bottom model" between training steps are directly correlated with the labels and features held by other participants. The attack follows a three-step process:
- Shadow Training: The adversary uses a small shadow dataset to train a reconstruction model.
- Feature/Label Recovery: The reconstruction model learns to map differences in model weights and gradients to missing private data.
- Execution: The trained model is applied to the target VFL training process to extract features and labels.
The Defense: VFLDefender
Traditional defenses like Differential Privacy (DP) often degrade model utility by introducing excessive noise. VFLDefender employs a more surgical approach through Gradient Obfuscation. By randomizing the magnitude (norm) of gradients while preserving their direction, VFLDefender breaks the statistical link required by VFLRecon without confusing the model's learning process.
Experimental Evaluation
Datasets
The proposed methods were evaluated on six benchmark datasets covering tabular and image modalities:
- Tabular: Sensorless Drive Diagnosis, Criteo (ad clicks), and Avazu (ad impressions).
- Image: CIFAR-10, BHI (breast histopathology), and CelebA (face attributes).
Results and Key Insights
Our experiments yielded several critical findings regarding VFL security:
- Potency of VFLRecon: The attack achieved a label reconstruction accuracy of 92% on the Criteo dataset. For feature reconstruction, the Mean Square Error (MSE) was as low as 0.01, indicating nearly identical data recovery.
- Ineffectiveness of Encryption: Standard "encrypted aggregation" (e.g., Homomorphic Encryption) fails to stop VFLRecon. While encryption hides the values exchanged, it does not hide the effect those values have on weight updates.
- Defense Performance: VFLDefender successfully reduced reconstruction accuracy (e.g., from 86% to 69% on Sensorless Drive) with only a ~1% accuracy loss, significantly outperforming standard DP which caused a 35% drop for similar protection levels.
Conclusion
This study highlights that VFL's privacy guarantees are fragile during the training phase. VFLRecon proves that even minimal "shadow data" (as little as 0.2% of the total) is sufficient for high-fidelity data reconstruction. However, by focusing on gradient magnitude rather than direction, VFLDefender provides a robust path forward for secure collaborative learning. Future work will investigate these vulnerabilities in non-neural architectures like Gradient Boosted Decision Trees and explore black-box adversarial scenarios.
Summary updated