Prototype Exchange for Heterogeneous Federated Learning

76 concepts60 papers1 proposal

Traditional federated learning frameworks, such as Federated Averaging (FedAvg), rely on a fundamental assumption: all participating clients must share the exact same neural network architecture. During each training round, clients optimize millions or billions of local parameters and transmit these high-dimensional weights or gradients back to a central server for aggregation.

This weight-sharing approach introduces two severe bottlenecks. First, the communication payload is massive, making it impractical for bandwidth-constrained edge devices. Second, it enforces strict architectural homogeneity, preventing a client with a lightweight mobile model from collaborating with a client running a massive Vision Transformer.

To bypass these limitations, a rapidly growing paradigm shifts the medium of collaboration from model weights to class prototypes.

Shifting from Weights to Semantic Prototypes

A prototype is the average embedding vector computed by a model's feature extractor for a specific category or cluster. Instead of sharing the internal parameter configurations of their models, clients compute and exchange these low-dimensional semantic summaries.

As pioneered by Guodong Long et al. in FedProto: Federated Prototype Learning across Heterogeneous Clients, clients pass their local class-specific mean feature vectors to a central server. The server aggregates these vectors to form global prototypes and broadcasts them back to the network. Local models then use these global prototypes as anchors, regularizing their local feature spaces so that different devices map the same semantic concepts to the same regions of an abstract embedding space.

This mechanism works because neural networks trained on similar tasks naturally develop comparable semantic layouts in their latent layers, regardless of their underlying architectures. By aligning these latent spaces directly, a diverse pool of client models can seamlessly collaborate. Indeed, empirical evaluations in the comprehensive benchmark library HtFLlib demonstrate that models of completely different internal dimensions can successfully align their representations using class-specific mean feature vectors.

Why It Works: Communication and Privacy Gains

The primary intuition behind prototype exchange is that the semantic essence of a dataset can be compressed into a handful of representative centroids. This compression yields massive advantages in efficiency and security:

  • Extreme Communication Efficiency: Instead of uploading gigabytes of gradients, a client only uploads a few low-dimensional vectors (one per class). Yue Tan et al. showed in Federated Learning from Pre-Trained Models: A Contrastive Learning Approach that this shift can slash communication payloads by over 99%. To push this further, Biwei Yan et al. developed FedRFQ, which uses an exponential pooling mechanism to strip out redundant parameters from shared prototypes, ensuring that only high-fidelity, highly distinct features are transmitted.
  • Inherent Privacy Abstraction: Sharing raw gradients leaves systems vulnerable to gradient inversion attacks that can reconstruct private user data. Prototypes, however, are heavily aggregated group-level representations. Qiang Meng et al. leveraged this property in Improving Federated Learning Face Recognition via Privacy-Agnostic Clusters, demonstrating that sharing sanitized cluster representations instead of individual class centers resolves the "privacy-utility paradox" inherent to sensitive biometric tasks.

Taming Data Heterogeneity and Domain Skew

Real-world federated settings suffer from severe data heterogeneity (non-IID data), where clients possess highly skewed class distributions or entirely different data domains. Prototypes act as a global compass to keep local models from drifting or overfitting to their biased local distributions.

When a client lacks samples for certain classes, its local classifier typically collapses. To fix this, Huancheng Chen et al. proposed FedDPMS, where clients share noisy, differentially private latent means to allow data-poor clients to synthesize artificial samples for missing classes. Similarly, Jianqing Zhang et al. introduced FedKTL, utilizing compact prototypes to guide a server-side pre-trained generator to create task-relevant synthetic data for local refinement.

To handle domain skew—where the same class looks completely different across clients—Yu Qiao et al. designed FedCCL, which replaces simple feature averaging with a dual-layer clustering strategy to capture fine-grained class information across diverse domains.

Expanding Beyond Classification

The flexibility of prototype exchange has allowed it to spread far beyond standard image classification:

  • Graph and Multi-View Clustering: Jingxin Liu et al. extended the paradigm to unsupervised graph learning in Federated Graph-Level Clustering Network by exchanging cluster prototypes to discover global structures without exposing raw graph nodes.
  • Class-Incremental Learning: In dynamic environments where clients continuously encounter new classes, catastrophic forgetting is a major risk. Jiahua Dong et al. developed an anti-forgetting model (LGA) that transmits perturbed prototype gradients, while Xin Gao et al. introduced FedProK to achieve efficient knowledge transfer without the computational overhead of storing raw data buffers.
  • Spatio-Temporal Forecasting: Qingxiang Liu et al. introduced FUELS, demonstrating that prototypes can capture periodic temporal patterns and spatial semantics, slashing communication costs by 94% in forecasting tasks.

Key Foundational Work

Federated Learning from Pre-Trained Models: A Contrastive Learning Approach (2022) — contributed 2 concept elements to this direction.
FedProto: Federated Prototype Learning across Heterogeneous Clients (2022) — contributed 2 concept elements to this direction.
FedHAP: Federated Hashing With Global Prototypes for Cross-Silo Retrieval (2022) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_prototype_based_federated_learning_b6442c3b,
  title = {prototype-based federated learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/prototype-based-federated-learning/rdr_b6442c3b78904852841fa581175e870a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_prototype_based_federated_learning_b6442c3b_v1,
  title = {prototype-based federated learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/prototype-based-federated-learning/rdr_b6442c3b78904852841fa581175e870a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake