FedSumUp:Secure Federated Learning Without Client-Side Training for Resource-Constrained Edge Devices

Jingwei Zhao, Ziyuan Ren, Yuanyuan Song, Mingcong Xu, Yiming Li, Dashan Gao, Bing Luo, Wei Chen, Xiaojin ZhangOpenReview 2026 · Submission to ICLR 2026 · Rejected · OpenReview

TL;DR

FedSumUp is a new federated learning framework that offloads the heavy lifting of data condensation from resource-constrained edge devices to a central server. By using a pre-trained Variational Autoencoder (VAE) to generate privacy-preserving data templates, it slashes client-side computational overhead by over 90% while significantly hardening the system against membership inference attacks.

What Problem Does This Paper Address?

Federated Learning (FL) allows multiple devices (clients) to train a shared model without ever sharing their raw, private data. A recent, communication-efficient branch of this field is Federated Data Condensation (FDC). Instead of sending massive model gradients, clients synthesize and upload a tiny, highly representative "synthetic" dataset.

However, the authors identify two major roadblocks for FDC:

  1. Privacy Leakage: Most current methods use real images as "templates" to start the synthesis process. The authors prove that these synthetic images often retain enough visual and semantic similarity to the originals that an attacker can perform a Membership Inference Attack (MIA)—essentially guessing if a specific person's data was used in training.
  2. Computational Burden: Generating these synthetic datasets requires complex optimization. For a low-power edge device (like a smart sensor or a wearable), this process is simply too power-hungry and slow.

Methods

The authors propose FedSumUp, which introduces a "client-training-free" paradigm. The workflow shifts from the client to the server:

  • Client-Side (Lightweight): Instead of optimizing pixels, the client performs a single, one-pass encoding. It uses a frozen, general-purpose VAE to turn a small batch of local images into compact latent codes. It also calculates the mean feature vector of its data using the current global model. These two pieces of "summary" information are sent to the server.
  • Server-Side (Heavy Lifting): The server takes these latent codes and performs a two-phase optimization:
  • Phase 1: It optimizes the latent codes to ensure the decoded images match the client's original feature distribution (using Maximum Mean Discrepancy or MMD loss).
  • Phase 2: It performs a refined optimization directly on the pixel values of the decoded images to maximize their utility for training the global model.
  • Privacy Buffer: Because the VAE is pre-trained on a separate, public dataset and its weights are frozen, it acts as a filter that strips away "personalized" features (like a unique scratch on a shoe) while retaining "common" features (the general shape of a shoe).

Datasets

The framework was evaluated on three standard computer vision benchmarks:

  • MNIST: Handwritten digits.
  • FashionMNIST: Clothing items.
  • CIFAR-10: General objects (birds, planes, cars, etc.).

These datasets were used to test the model's performance in non-IID (non-independent and identically distributed) settings, which simulate real-world scenarios where different users have very different types of data.

Key Insights and Ideas

  • The "Real-Data Mode" Trap: The authors highlight a subtle loophole in prior research. Previous works claimed privacy benefits while using random noise for evaluation but real data for utility testing. FedSumUp proves that using real data as a template is a major privacy risk, even if the final image looks "distorted" to the human eye.
  • VAE as a Privacy Filter: One of the most elegant ideas here is using the VAE not just for compression, but for privacy. By forcing the data through a latent space trained on public data, the model naturally discards the "outlier" features that make individual data points identifiable.
  • Server-Side Migration: By moving the optimization to the server, the authors transform FDC from a process that takes thousands of seconds on a client to one that takes less than five minutes, making it viable for the "Internet of Things" (IoT).

Key Results

  • Efficiency: FedSumUp reduced client-side computation time by over 90% compared to FedSD2C. On MNIST, client time dropped from 3100 seconds to just 190 seconds.
  • Privacy: The method achieved near-random MIA accuracy (approx. 0.50), meaning an attacker has no better than a coin-flip's chance of identifying a user's data. This outperformed all baselines that used real-data templates.
  • Robustness: In extreme non-IID settings (where data is highly fragmented across clients), FedSumUp maintained a much higher accuracy (54%) compared to traditional methods like FedProx (9.9%).

Limitations and Future Work

  • Utility Trade-off: While FedSumUp is highly secure, there is a slight "utility tax." On complex datasets like CIFAR-10, the accuracy is lower than methods that use raw-data templates because the privacy-preserving VAE strips away some useful (but potentially sensitive) information.
  • VAE Dependency: The current model relies on a general-purpose VAE. The authors suggest that future work could involve designing "personalized VAEs" or exploring how this framework handles text data and Large Language Models (LLMs).
  • Advanced Attacks: While it resists standard MIA, the authors acknowledge the need to test against even more sophisticated, emerging privacy attacks.

Summary updated

Abstract

Horizontal Federated Learning (HFL) enables multiple clients with private data to collaboratively train a global model without sharing their local data. As a research branch of HFL, Federated Data Condensation with Distribution Matching (FDCDM) introduces a novel collaborative paradigm where clients upload small synthetic datasets instead of gradients and parameters. FDCDM faces two key challenges: privacy leakage risk, where synthetic data may leak the privacy of real data; and high computational cost on the client side, which limits the deployment capability of FDCDM on resource-constrained devices. To address these challenges, we propose FedSumUp, an improved FDCDM method. The core designs of FedSumUp include: generating initial data templates based on a Variational Autoencoder (VAE); and migrating the entire synthetic data optimization process to the server side, requiring clients only to upload distilled synthetic data and the mean of raw data features without exposing the original data itself. Experimental results on multiple real-world datasets demonstrate that FedSumUp achieves notable advantages in the following aspects: drastically reducing the visual similarity between synthetic and real data, and effectively resisting membership inference attacks; significantly lowering client-side computational overhead, making it deployable on edge devices. FedSumUp is the first work to systematically analyze privacy risks in FDCDM from the perspective of data similarity, providing a new direction for building efficient and privacy-preserving federated learning frameworks.
BibTeX
@misc{zhao2026fedsumup,
  author = {Jingwei Zhao and Ziyuan Ren and Yuanyuan Song and Mingcong Xu and Yiming Li and Dashan Gao and Bing Luo and Wei Chen and Xiaojin Zhang},
  title = {{FedSumUp:Secure Federated Learning Without Client-Side Training for Resource-Constrained Edge Devices}},
  year = {2026},
  url = {https://openreview.net/forum?id=5rjSeZCM6l},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2026},
}
Figure