All You Need Is Hashing: Defending Against Data Reconstruction Attack in Vertical Federated Learning

Pengyu Qiu, Xuhong Zhang, Shouling Ji, Yuwen Pu, Ting Wang

Introduction

Vertical Federated Learning (VFL) has emerged as a powerful paradigm allowing multiple organizations—such as a bank and an e-commerce platform—to collaboratively train machine learning models on shared users without exchanging raw, sensitive data. In VFL, parties only share intermediate "embeddings" or representations. However, recent research has exposed a critical vulnerability: Data Reconstruction Attacks. Because deep neural networks capture intricate correlations, an adversary or a curious server can exploit these shared embeddings to reconstruct original inputs, such as private financial records or images. Existing defenses, like differential privacy or correlation distance constraints, often struggle to balance privacy with model utility.

The HashVFL Framework

To address this, the authors introduce HashVFL, a framework that replaces standard floating-point embeddings with one-way binary hash codes (sequences of +1 and -1). By collapsing continuous values into discrete bits, the precision required for data reconstruction is permanently discarded, creating a robust "lossy" shield.

The HashVFL architecture showing the flow from local datasets to the global top model via hash layers.

Overcoming Technical Hurdles

Implementing deep hashing in a federated setting introduces three primary challenges:

  1. Learnability (The Gradient Problem): The Sign function used for binarization has zero gradients almost everywhere, which breaks standard backpropagation. HashVFL utilizes a Straight-Through Estimator (STE) to allow gradients to bypass the non-differentiable layer during training.
  2. Bit Balance (Information Maximization): To ensure each bit carries maximum information (entropy), the framework employs a Batch Normalization (BN) layer before hashing. This centers the feature distribution around zero, ensuring an approximately equal distribution of +1 and -1 values across samples.
  3. Consistency: Different parties must produce compatible hash codes for the same sample. Instead of expensive cross-party comparisons, HashVFL uses Pre-defined Binary Codes as "anchors." Each party optimizes its local hash code to match these targets using a Cosine Similarity loss, reducing computational complexity from $O(N^2)$ to $O(N)$.

Key Insights and Ideas

  • Hashing as a One-Way Shield: Unlike encryption, which can be reversed with a key, hashing is inherently lossy. Once data is binarized, the original high-fidelity information is gone.
  • The "Voting" Effect: Interestingly, HashVFL improved performance on some tabular datasets like CRITEO. The authors suggest that binarization acts as a robust feature extractor, forcing the model to "vote" on key features rather than being distracted by noisy floating-point values.
  • Inherent Robustness: The discrete nature of the hash codes makes the model more resilient to adversarial examples (e.g., PGD attacks), as small perturbations often fail to flip the binary bits.

Key Results

HashVFL was evaluated across image (MNIST, CIFAR10, FER), tabular (CBPD, CRITEO), and text (IMDb) datasets. The results demonstrate that the framework provides a formidable defense without sacrificing performance.

  • High Utility: Model accuracy remains near baseline levels. For example, on MNIST, accuracy only shifted from 98.99% to 97.75%. In the case of the CRITEO tabular dataset, performance actually increased from 70.72% to 72.94%.
  • Attack Defeat: Under rigorous threat models where the attacker knows the victim's architecture, reconstruction attempts yielded only meaningless noise. Statistical metrics like the Structural Similarity Index (SSIM) remained extremely low.
Visualizations of reconstructed data across different bit lengths, demonstrating that the output is unintelligible noise.

Limitations and Future Work

While HashVFL effectively blocks raw data reconstruction, the authors note that Attribute Inference might still be possible. Future research could explore how generative models might use inferred attributes to create "semantic" likenesses of data. Additionally, balancing "feature importance" across parties remains a challenge if one party holds a disproportionate amount of data. Finally, determining the optimal hash code length for different data types remains an empirical task for practitioners.

Summary updated

BibTeX
@article{qiu2024all,
  author = {Pengyu Qiu and Xuhong Zhang and Shouling Ji and Yuwen Pu and Ting Wang},
  title = {{All You Need Is Hashing: Defending Against Data Reconstruction Attack in Vertical Federated Learning}},
  journal = {IEEE Trans. Inf. Forensics Secur. 2024},
  year = {2024},
  doi = {10.1109/tifs.2024.3356164},
}
Figure