Cloud-based Federated Boosting for Mobile Crowdsensing

Zhuzhu Wang, Yilong Yang, Yang Liu, Ximeng Liu, Brij B. Gupta, Jianfeng MaarXiv 2020 · arXiv

In the world of mobile crowdsensing (MCS), data is the new oil. Companies want to build high-performance models using data from thousands of volunteer smartphones, but this creates a massive privacy headache. How do you train a powerful model like XGBoost without ever seeing the users' raw data, and how do you prevent those same users from stealing the model once it's built?

A team of researchers from Xidian University, Fuzhou University, and NIT Kurukshetra recently proposed FedXGB, a new federated learning architecture designed to solve these exact problems using the power of secret sharing.

TL;DR

FedXGB is a privacy-preserving federated learning framework for XGBoost that uses secret sharing to protect both user data and the resulting model. It achieves less than 1% accuracy loss compared to standard XGBoost while remaining robust against user dropouts and sophisticated GAN-based reconstruction attacks.

What Problem Does This Paper Address?

Standard Federated Learning (FL) allows users to keep their data local and only upload calculated gradients. However, the authors identify three critical gaps in current FL systems:

  1. Data Reconstruction Attacks: Malicious servers can use Generative Adversarial Networks (GANs) to "reverse engineer" raw user data from the gradients they upload.
  2. Model Privacy Leakage: Most FL systems send the global model back to users for local updates. In a crowdsensing scenario, a "curious" user could easily steal this trained model—the service provider's intellectual property.
  3. User Instability: Mobile users frequently drop out due to poor connectivity. Many FL architectures simply fail or lose progress if a user disappears mid-round.

While existing solutions like SecureBoost exist, they often rely on Homomorphic Encryption (HE), which is computationally expensive and typically designed for "vertical" data splitting (where different features of the same person are held by different companies), rather than the "horizontal" splitting common in mobile apps.

Methods

The core of FedXGB is the transition from expensive encryption to efficient Secret Sharing (SS). The architecture introduces a three-tier system: a Central Cloud Server, multiple Edge Servers, and the Users (see Figure 2).

![Fig.

  1. Entities of FedXGB](/figures/art_71992a7dbd024e61b8adc746641afae7)

The framework consists of two main protocols:

  • SecBoost (Secure CART Building): To build the decision trees (CARTs) that make up XGBoost, users calculate their local gradients but mask them using a secret masking function (SecMask). These masked values are aggregated by edge servers and then the central server. Because the values are shared as "secrets," no single entity sees the plaintext data.
  • SecPred (Secure Prediction): This protects the model during the inference phase. Instead of sending the full tree to the user, the model parameters and the user's data are split into secret shares and sent to edge servers. The edge servers perform secure comparisons (SecCmp) to navigate the tree, returning only the final result to the user.

To handle User Dropout, FedXGB uses a $(t, n)$ threshold scheme. Users share pieces of their private mask keys with each other. If a user drops out, the edge server can collect enough "shares" from other active users to reconstruct the necessary keys and complete the training round without starting over.

Datasets

The authors evaluated FedXGB using two widely recognized benchmarks:

  • ADULT: A dataset for predicting whether an individual's income exceeds $50k/year (123 features, ~48,000 instances).
  • MNIST: The classic handwriting digit recognition dataset (784 features, 70,000 instances).

The experiments simulated a realistic crowdsensing environment with 300 users and 10 edge servers.

Key Insights and Ideas

  • Breaking the GAN Attack: The authors point out that GAN-based attacks rely on solving optimization problems over continuous gradients. By using XGBoost's tree structure—which partitions space into discrete regions—and masking the values via secret sharing, the attack becomes "noncontinuous and nondifferentiable," effectively blinding the GAN.
  • Edge Servers as Privacy Buffers: In FedXGB, edge servers aren't just for reducing latency; they act as a vital architectural layer for secret sharing and key reconstruction, making the system much more robust than a simple client-server model.
  • Efficiency over Encryption: By opting for Secret Sharing over Homomorphic Encryption, the authors significantly reduced the computational burden on mobile devices, which have limited battery and CPU power.

Key Results

  • High Fidelity: FedXGB maintains the high performance of XGBoost, showing less than 1% accuracy loss across both datasets compared to a non-private model.
  • Speed: In head-to-head comparisons, FedXGB was faster than the HE-based SecureBoost. For example, the total runtime for a specific training stage was roughly 37 seconds for FedXGB vs. 46 seconds for SecureBoost.
  • Scalability: While user runtime grows linearly with the number of participants, the communication overhead per user actually decreases as more users are added, because the data samples are spread thinner.
  • Resilience: The system remained stable even with a 30% user dropout rate, a common occurrence in real-world mobile networks.

Limitations and Future Work

The paper primarily operates under the honest-but-curious security model. This means it assumes that while servers and users might try to learn extra information, they will still follow the protocol rules. It does not explicitly address "malicious" actors who might intentionally upload garbage data to poison the model.

Future work could involve extending this framework to handle actively malicious participants or optimizing the communication overhead, which remains a challenge for any secret-sharing-based system in a bandwidth-constrained mobile environment.

Summary updated

Abstract

The application of federated extreme gradient boosting to mobile crowdsensing apps brings several benefits, in particular high performance on efficiency and classification. However, it also brings a new challenge for data and model privacy protection. Besides it being vulnerable to Generative Adversarial Network (GAN) based user data reconstruction attack, there is not the existing architecture that considers how to preserve model privacy. In this paper, we propose a secret sharing based federated learning architecture FedXGB to achieve the privacy-preserving extreme gradient boosting for mobile crowdsensing. Specifically, we first build a secure classification and regression tree (CART) of XGBoost using secret sharing. Then, we propose a secure prediction protocol to protect the model privacy of XGBoost in mobile crowdsensing. We conduct a comprehensive theoretical analysis and extensive experiments to evaluate the security, effectiveness, and efficiency of FedXGB. The results indicate that FedXGB is secure against the honest-but-curious adversaries and attains less than 1% accuracy loss compared with the original XGBoost model.
BibTeX
@misc{wang2020cloud,
  author = {Zhuzhu Wang and Yilong Yang and Yang Liu and Ximeng Liu and Brij B. Gupta and Jianfeng Ma},
  title = {{Cloud-based Federated Boosting for Mobile Crowdsensing}},
  year = {2020},
  eprint = {2005.05304},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2005.05304},
}

Loading related content...

Figure