SVAFD: A Secure and Verifiable Co-Aggregation Protocol for Federated Distillation

Tian Wen, Sheng Sun, Yuwei Wang, Peiyan Chen, Zhiyuan Wu, Min Liu, Bo GaoarXiv 2025 · DOI · arXiv

Introduction

Federated Learning (FL) typically requires participants to share the same model architecture, a constraint that often clashes with the reality of diverse hardware capabilities. Federated Distillation (FD) offers a solution by allowing clients to share "logits" (raw model outputs) instead of large weight matrices, enabling heterogeneous model training with lower communication costs.

However, FD introduces significant security vulnerabilities. Most existing Secure Aggregation (SA) protocols assume a "semi-honest" server. In practice, a malicious server can forge results, collude with clients to leak private data, or distribute poisoned global knowledge to sabotage the network. Furthermore, traditional security tools designed for model weights are often ineffective at filtering malicious logit outputs.

The threat model in standard Federated Distillation, highlighting inference attacks (T1) and poisoning attacks (T2, T3).

The SVAFD Protocol

To address these gaps, the authors propose SVAFD (Secure and Verifiable Co-Aggregation Protocol for Federated Distillation). SVAFD shifts the power balance from a server-dominant model to a collaborative "co-aggregation" approach. The protocol is built on three primary pillars:

  1. Multilateral Co-Aggregation: SVAFD redefines the server's role. Using Lagrange Coding Computation (LCC), clients encode their logits into shares and exchange them within sub-groups for local aggregation. The server is relegated to decoding the final result and providing a mathematical proof of its correctness.
  2. Quality-Aware Knowledge Filtration (CAL): To mitigate poisoning, the protocol utilizes Class Average Logits (CAL). Because logits reflect the underlying data distribution, clients can use Locality-Sensitive Hashing (LSH) to identify benign peers and filter out malicious outliers before the aggregation process begins.
  3. Verifiable Signatures: Utilizing bilinear pairings, the protocol enables clients to verify the integrity of the server's aggregation without the server ever accessing unencrypted logits.
The SVAFD architecture, illustrating the three-stage process: Knowledge Filtration, Knowledge Aggregation, and Knowledge Verification.

Evaluation and Key Results

The protocol was rigorously tested using the SVHN and FashionMNIST benchmarks across heterogeneous architectures, including ResNet8, ResNet16, and ResNet20.

Key Insights:

  • Robust Defense: SVAFD successfully defended against 10 distinct attack types, including label flipping and inference attacks. In high-malice scenarios, SVAFD improved model accuracy from 36.46% to 61.11% compared to unprotected FD.
  • Efficiency: Despite the cryptographic overhead, client-side tasks remain lightweight. Authentication and deblinding take less than 10ms for a client in a 1,000-participant network.
  • Precision: The LCC encoding process introduces negligible relative error (approximately $10^{-7}$), ensuring that security does not compromise model quality.

Conclusion and Future Work

SVAFD demonstrates that decentralizing trust in Federated Distillation is both feasible and efficient. By leveraging the unique statistical properties of logits and offloading heavy verification tasks to the server, it provides a robust framework for secure collaborative learning. Future research aims to reduce the communication complexity of the all-to-all sharing model and extend the framework to support non-linear aggregation functions.

Summary updated

BibTeX
@misc{wen2025svafd,
  author = {Tian Wen and Sheng Sun and Yuwei Wang and Peiyan Chen and Zhiyuan Wu and Min Liu and Bo Gao},
  title = {{SVAFD: A Secure and Verifiable Co-Aggregation Protocol for Federated Distillation}},
  year = {2025},
  eprint = {2505.13319},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.13319},
}
Figure