A Multi-Prototype-Guided Federated Knowledge Distillation Approach in AI-RAN Enabled Multi-Access Edge Computing System

Luyao Zou, Hayoung Oh, Chu Myaet Thwal, Apurba Adhikary, Seohyeon Hong, Zhu HanarXiv 2026 · DOI

TL;DR

This paper introduces MP-FedKD, a novel federated learning approach for AI-RAN enabled MEC systems that tackles data heterogeneity using multi-prototype knowledge distillation. By employing self-knowledge distillation, a hierarchical clustering method for generating multiple prototypes per class, and a new loss function, MP-FedKD significantly improves accuracy and reduces errors compared to existing methods.

What Problem Does This Paper Address?

The integration of Artificial Intelligence (AI) with Radio Access Networks (RAN) and Multi-Access Edge Computing (MEC) promises to revolutionize wireless communication by enhancing efficiency and responsiveness. Federated Learning (FL) is a promising paradigm for training AI models in these distributed MEC environments, allowing edge devices to collaboratively train a global model without sharing raw data. However, a major challenge in FL, particularly in MEC, is data heterogeneity, where data across edge devices is non-independent and identically distributed (non-IID). This non-IID nature can lead to model divergence and degraded performance. Existing solutions often struggle with information loss when representing data with a single prototype per class or lack the specific adaptations needed for the AI-RAN MEC context.

Methods

The proposed MP-FedKD approach tackles data heterogeneity through several key innovations:

  • Self-Knowledge Distillation (SKD): Instead of relying on a separate teacher network, SKD uses the previous round's local model as the teacher to guide the current round's local model training. This is particularly effective for handling non-IID data without the overhead of pre-training a teacher model.
  • Conditional Hierarchical Agglomerative Clustering (CHAC): To address the information loss associated with single prototypes, MP-FedKD employs a multi-prototype strategy. CHAC is a novel clustering method based on Hierarchical Agglomerative Clustering (HAC) that generates multiple prototypes for each class on each local edge device. This allows for a richer representation of class features.
  • Prototype Alignment (PA): This mechanism allows global prototypes to learn from local embeddings generated by the previous round's local model. This helps align local representations with the shared global model and prototypes, mitigating information loss.
  • LEMGP Loss: A novel loss function is introduced, inspired by the COREL loss. It comprises an attractive part (weighted MSE loss) to pull local embeddings towards their corresponding global prototypes and a repulsive part (logarithmic function and weighted MSE loss) to push them away from prototypes of other classes.
  • Multi-Prototype and Model Aggregation: Both global models and global prototypes are aggregated on the central server.

Datasets

The paper evaluates MP-FedKD using six datasets:

  • CIFAR-10
  • MNIST
  • Fashion-MNIST
  • EuroSAT
  • M+F (combination of MNIST and Fashion-MNIST)
  • C+E (combination of CIFAR-10 and EuroSAT)

The experiments utilize various non-IID settings, including "Same Domain" (using Dirichlet distributions with parameters like 0.3, 0.5, 0.7, 0.9) and "Distinct Domain" (combining datasets like M+F and C+E). No new benchmarks were introduced; existing datasets were adapted to simulate non-IID conditions.

Key Insights and Ideas

  • Multi-prototype is better than single-prototype: The paper strongly suggests that using multiple prototypes per class captures more nuanced feature information than a single averaged prototype, leading to better performance.
  • SKD is a practical solution for non-IID FL: Leveraging the previous local model as a teacher is an effective and efficient way to combat data heterogeneity in FL without requiring a separate, pre-trained teacher network.
  • Prototype alignment bridges local and global knowledge: Aligning global prototypes with historical local embeddings helps to better integrate local insights into the global model, reducing information loss.
  • LEMGP loss effectively guides local learning: The proposed LEMGP loss, with its attractive and repulsive components, effectively guides local embeddings to be close to their own class's global prototype and distant from others.
  • CHAC offers an advantage over K-Means: The hierarchical nature of CHAC provides richer information for prototype generation compared to simpler methods like K-Means, leading to improved accuracy.

Key Results

  • MP-FedKD consistently outperforms state-of-the-art baselines (FedProx, FedProto, FedAS, MOON, E-FPKD, FedALA) across various datasets and non-IID settings.
  • Accuracy improvements ranged from 1.98% to 28.70% on the EuroSAT dataset with 10 clients.
  • The proposed CHAC method achieved approximately 1.03% and 1.02% higher accuracy than K-Means-based MP-FedKD on M+F and Fashion-MNIST datasets, respectively.
  • The method demonstrated robustness, with accuracy gradually increasing and surpassing baselines as training progressed towards convergence.
  • Ablation studies showed that both the prototype alignment mechanism and the LEMGP loss are crucial for the method's effectiveness, with their removal leading to significant accuracy drops.

Limitations and Future Work

  • Computational Complexity: While the paper focuses on improving accuracy and robustness, the computational complexity of CHAC on each client is noted as O(|Dm,c|³), which could be a consideration for resource-constrained edge devices.
  • Hyperparameter Tuning: The performance of the MP-FedKD approach relies on several hyperparameters (e.g., $\lambda$, $\mu_1$, $\mu_2$, $\mu_3$), which require careful tuning.
  • Future Work: The paper implicitly suggests that further optimization of the CHAC algorithm for efficiency and exploring more advanced prototype alignment strategies could be valuable future research directions. The authors also mention that the current work does not consider personalized FL schemes, which could be an avenue for future exploration.

Summary updated

BibTeX
@misc{zou2026multi,
  author = {Luyao Zou and Hayoung Oh and Chu Myaet Thwal and Apurba Adhikary and Seohyeon Hong and Zhu Han},
  title = {{A Multi-Prototype-Guided Federated Knowledge Distillation Approach in AI-RAN Enabled Multi-Access Edge Computing System}},
  year = {2026},
  eprint = {2603.09727},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.09727},
}
Figure