Where Does This Data Come From? Enhanced Source Inference Attacks in Federated Learning

Haiyang Chen, Xiaolong Xu, Xiang Zhu, Xiaokang Zhou, Fei Dai, Yansong Gao, Xiao Chen, Shuo Wang, Hongsheng Hu

Introduction

Federated Learning (FL) is often promoted as a privacy-preserving paradigm because raw data remains on the client’s device. However, this paper introduces Enhanced Source Inference Attacks (ESIAs), demonstrating that a malicious server can actively manipulate the training process to "unmask" which specific client contributed a particular data sample. By combining gradient ascent with data augmentation, the researchers show that an active adversary can identify data origins with significantly higher accuracy than previous passive methods.

The Problem: Identifying Data Sources

In standard FL, clients share model updates (gradients) rather than raw data. While this mitigates direct exposure, it remains vulnerable to Membership Inference Attacks (determining if a record was in the training set). This research addresses a more invasive threat: Source Inference. If a server identifies a sensitive record within the global training set, can it pinpoint the exact hospital or user that provided it?

Existing source inference attacks are primarily "passive," merely observing updates. These often fail because behavioral differences between clients are not sufficiently distinct. This paper explores an "active" threat model where a server intentionally sabotages the global model to force the true data owner to reveal themselves through their local training response.

Methodology: The Trap-and-Trace Strategy

The core of the ESIA approach is a "trap-and-trace" strategy involving two main components:

  1. Active Gradient Ascent: In each training round, the malicious server takes the target record ($z$) and intentionally performs gradient ascent on the global model. This deliberately increases the model's loss on that specific record, essentially "breaking" the model's ability to recognize that data point.
  2. Data Augmentation: To ensure robustness, the server targets an "augmented set" of the record (including rotations, flips, and noise) rather than just the original sample.
The ESIA workflow: A malicious server uses gradient ascent and data augmentation to 'trap' a target record, then identifies the source client by observing which one 'repairs' the model during local training.

The Logic: When the "broken" global model is sent to clients, most will continue training on their own unrelated data. However, the source client—who actually owns the target record—will naturally minimize the loss on its local data, thereby "repairing" the damage the server inflicted. By comparing updated models, the server identifies the client showing the lowest loss on the target record and its variants.

Experimental Setup

The authors evaluated ESIAs across diverse benchmarks:

  • Image Data: MNIST, FEMNIST, CIFAR-10, and CIFAR-100.
  • Tabular/Synthetic Data: A custom Synthetic IID dataset and the Purchase dataset.
  • Heterogeneity Control: A Dirichlet distribution (parameterized by $\alpha$) was used to simulate varying levels of Non-IID data, a common characteristic of real-world FL.

Key Results and Insights

ESIAs were tested against five major FL frameworks (FedAvg, FedSGD, FedMD, FedPer, and FedProx) and consistently outperformed existing baselines.

Comparison of Attack Success Rate (ASR) for ESIAs against baseline methods (ASI, SIAs, RG) across different datasets and data heterogeneity levels (alpha).
  • Superior Accuracy: In a FedAvg scenario with CIFAR-10 ($\alpha=0.1$), ESIA achieved an Attack Success Rate (ASR) of 74.3%, significantly higher than standard passive attacks (62.5%).
  • The Impact of Local Training: Increasing the number of local epochs ($E$) improved the ASR. For the Purchase dataset, the ASR rose from 73.2% to 78.3% as local training increased, suggesting that more local optimization makes clients more "visible" to the attacker.
  • Non-IID Data as a Risk Factor: While data heterogeneity is a challenge for FL performance, it aids attackers. The more unique a client's data distribution is, the easier it is to pinpoint them as the source of a specific record.

Conclusion

This research serves as a stark reminder that "keeping data on-device" is not a complete privacy solution. Active adversaries can exploit the natural behavior of local optimization to bypass the perceived anonymity of Federated Learning. Future work may involve exploring spatio-temporal features to identify sources in even more complex environments, such as healthcare networks.

Summary updated

Abstract

Federated learning (FL) enables collaborative model training without exposing raw data, offering a privacy-aware alternative to centralized learning. However, FL remains vulnerable to various privacy attacks that exploit shared model updates, including membership inference, property inference, and gradient inversion. Source inference attacks further threaten FL by identifying which client contributed a specific training sample, posing severe risks to user and institutional privacy. Existing source inference attacks mainly assume passive adversaries and overlook more realistic scenarios where the server actively manipulates the training process. In this paper, we present an enhanced source inference attack that demonstrates how a malicious server can amplify behavioral differences between clients to more accurately infer data origin. Our approach introduces active training manipulation and data augmentation to expose client-specific patterns. Experimental results across five representative FL algorithms and multiple datasets show that our method significantly outperforms prior passive attacks. These findings reveal a deeper level of privacy vulnerability in FL and call for stronger defense mechanisms under active threat models.
BibTeX
@inproceedings{chen2025where,
  author = {Haiyang Chen and Xiaolong Xu and Xiang Zhu and Xiaokang Zhou and Fei Dai and Yansong Gao and Xiao Chen and Shuo Wang and Hongsheng Hu},
  title = {{Where Does This Data Come From? Enhanced Source Inference Attacks in Federated Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/536},
}
Figure