Federated Black-Box Adaptation for Semantic Segmentation

Jay Nitin Paranjape, Shameema Sikder, S. Swaroop Vedula, Vishal M. Patel

TL;DR

Federated Learning (FL) is designed to protect data privacy, but recent attacks show that sharing gradients or model weights can actually leak private training data. This paper introduces BlackFed, a "black-box" federated learning framework for semantic segmentation that successfully trains collaborative models without sharing any gradients or model architectures between the client and server. This drastically reduces the attack surface while maintaining high accuracy.

What Problem Does This Paper Address?

Federated Learning allows multiple institutions—like different hospitals—to collaboratively train an AI model without ever sharing their raw, sensitive data. Standard FL methods (like FedAvg) achieve this by having clients train local models and share their weights or gradients with a central server.

However, recent cybersecurity research has exposed a massive flaw: bad actors can use "gradient inversion" attacks to reverse-engineer those shared gradients and reconstruct the original private images.

To fix this, researchers have looked at Split Neural Networks (Split-NNs), where the client runs the first few layers of a model and the server runs the rest. But even Split-NNs usually require the server to send gradients back to the client during backpropagation. This paper asks: Can we collaboratively train a dense, pixel-level semantic segmentation model without passing any gradients or model architecture information between the client and the server?

Comparison of Traditional FL, Split-NN, and BlackFed

Methods: The BlackFed Approach

To achieve this "black-box" adaptation, the authors propose BlackFed, which relies on a Split-NN architecture but completely severs the backward gradient flow.

Here is how they make it work:

  • Alternating Optimization: Instead of standard backpropagation across the network split, BlackFed factorizes the training.
  • Client-Side Zero-Order Optimization (ZOO): The client updates its small, local sub-network without gradients. It uses a technique called SPSA-GC (Simultaneous Perturbation Stochastic Approximation with Gradient Correction), which essentially tweaks the model weights slightly, observes how the loss changes, and approximates a gradient to update itself.
  • Server-Side First-Order Optimization (FOO): The server, which holds the bulk of the heavy model (e.g., DeepLabv3), updates its weights using standard gradient descent (Adam-W optimizer).
  • Beating Catastrophic Forgetting (BlackFed v2): Because the server trains on different clients in a round-robin fashion, it tends to "forget" the data distributions of older clients. To fix this, BlackFed v2 introduces a simple but highly effective trick: the server maintains a "hashmap" (a memory bank) of its weights indexed by the client ID. During inference, the server simply loads the specific weights tailored to the calling client.

Datasets

The authors evaluated BlackFed on both natural computer vision and medical imaging datasets, treating different data sources as distinct FL clients:

  • Cityscapes: Urban street scenes, split into 18 clients based on different cities.
  • CAMVID: Road-view segmentation, split into 4 clients.
  • ISIC: Skin lesion segmentation, split into 3 clients based on different challenge years (2016, 2017, 2018).
  • PolypGen: Colon polyp segmentation, split into 6 clients representing different hospitals.

Key Insights and Results

  • Gradients aren't strictly necessary for collaborative feature extraction: You can successfully align a client's local feature extractor with a global server model using purely Zero-Order Optimization, completely closing the gradient-leakage vulnerability.
  • Optimization order is critical: The authors found that in their alternating setup, they must update the client first, followed by the server. If the server updates first, the ZOO process becomes highly unstable and performance collapses.
  • Strong Out-of-Distribution (OOD) Generalization: BlackFed v2 vastly outperformed individual, isolated training. For example, on the CAMVID dataset, clients training alone achieved an average OOD mean Intersection over Union (mIoU) of ~0.51. BlackFed v2 boosted this to ~0.70.
  • Matches "White-Box" Baselines: Despite the strict privacy constraints (no gradients, no model sharing), BlackFed v2 performs on par with "White-box" Split-NNs (where gradients are shared) and centralized training (where all data is pooled together).
  • Massive Compute Offloading: Because the client only runs a tiny 2-layer convolutional network and uses ZOO, the computational burden on the edge device is drastically reduced. For a DeepLabv3 architecture, standard FL requires the client to process 353.6 GFLOPS. BlackFed reduces the client-side compute to just 26.6 GFLOPS, making it highly viable for resource-constrained edge devices.

Limitations and Future Work

  • Vulnerability to Generative Attacks: While gradients and weights are hidden, the client and server still exchange intermediate features and final segmentation masks. The authors acknowledge a need to investigate if advanced generative AI models could somehow reverse-engineer client data using just these shared masks.
  • Client Selection Policies: The current method uses a simple round-robin approach to select clients during training. The authors suggest future work should explore smarter client-selection policies, especially to handle scenarios where data distributions are wildly skewed between institutions.

Summary updated

Abstract

Federated Learning (FL) is a form of distributed learning that allows multiple institutions or clients to collaboratively learn a global model to solve a task. This allows the model to utilize the information from every institute while preserving data privacy. However, recent studies show that the promise of protecting the privacy of data is not upheld by existing methods and that it is possible to recreate the training data from the different institutions. This is done by utilizing gradients transferred between the clients and the global server during training or by knowing the model architecture at the client end. In this paper, we propose a federated learning framework for semantic segmentation without knowing the model architecture nor transferring gradients between the client and the server, thus enabling better privacy preservation. We propose \textit{BlackFed} - a black-box adaptation of neural networks that utilizes zero order optimization (ZOO) to update the client model weights and first order optimization (FOO) to update the server weights. We evaluate our approach on several computer vision and medical imaging datasets to demonstrate its effectiveness. To the best of our knowledge, this work is one of the first works in employing federated learning for segmentation, devoid of gradients or model information exchange. Code: https://github.com/JayParanjape/blackfed/tree/master
BibTeX
@inproceedings{paranjape2024federated,
  author = {Jay Nitin Paranjape and Shameema Sikder and S. Swaroop Vedula and Vishal M. Patel},
  title = {{Federated Black-Box Adaptation for Semantic Segmentation}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=Fp3JVz5XE7},
}
Figure