Global Feature Anchors for Robust Federated Learning

This direction is merged into another direction: prototype-based feature alignment. Its last version stays readable and citable here.

Version 1 of 1105 papers126 conceptsfrozen 2026-10-04

In standard Federated Learning, the central server acts as a librarian, averaging the "books" (model weights) written by various clients. This works well if every client is writing about the same topic. However, in the real world, data is non-IID (not identically and independently distributed). One client might only see images of cats, while another only sees cars. When we average their weights, the resulting model often performs poorly because the local models have "drifted" too far into their own specialized niches.

The emerging research direction across these 101 papers suggests a more effective way to collaborate: instead of just averaging weights, we should provide clients with Global Feature Anchors. These anchors act as a shared coordinate system, ensuring that even if clients see different data, they all learn to speak the same "feature language."

The Problem of Feature Drift

When a model trains on a local, biased dataset, its internal representation space—the way it mathematically describes a "cat" or a "car"—becomes warped. Qinbin Li and colleagues demonstrate in Model-Contrastive Federated Learning that this "local drift" is the primary barrier to convergence.

If Client A represents a "cat" as a point in the top-left of its feature space and Client B represents it in the bottom-right, averaging their weights results in a global model that is confused. The weights are mathematically averaged, but the meanings behind those weights are fundamentally misaligned.

Anchoring via Global Prototypes

To solve this, researchers like Jun Zhang in FedFA propose maintaining global feature centroids (or prototypes).

Think of these as "North Stars" in the feature space. The server calculates the average representation of each class across all clients and sends these back to the participants. During local training, a client doesn't just try to classify its own data; it also tries to ensure that its local features for a "cat" are close to the global "cat" anchor.

This mechanism provides several intuitive benefits:

  1. Consistency: It forces diverse local models to map their specific data into a unified representation space.
  2. Calibration: As noted in FedFA, it allows a client to calibrate its classifier even for classes it has never seen, by using the global anchors as "virtual" training samples.
  3. Stability: Qiuyu Chen shows in Cluster-level Feature Alignment that aligning with global anchors provides a much smoother optimization path than comparing individual images in small, noisy mini-batches.

Contrastive and Structural Alignment

The "anchoring" doesn't have to be a simple distance loss. Several papers use more sophisticated alignment strategies:

  • Model-Contrastive Learning: In the MOON framework (Qinbin Li et al.), the global model itself serves as a positive anchor. Local models are trained to produce representations similar to the global model (the anchor) and dissimilar to their own previous local versions.
  • Neural Collapse: Chenxi Huang and others in FedETF take this a step further by fixing the classifier head to a mathematically ideal geometric structure called an Equiangular Tight Frame (ETF). By fixing the "target" locations in the feature space for every client, they force all decentralized models to align their internal representations to a rigid, global structure from the very first epoch.
  • Cross-Distillation: Sungwon Han in FedX uses the global model as a "teacher" to guide local updates, effectively distilling the collective knowledge of the network into each local node to prevent them from overfitting to their biased local data.

Bridging Heterogeneity

One of the most powerful aspects of feature anchoring is its ability to handle heterogeneity—not just in data, but in model architectures.

Standard FedAvg requires every client to use the exact same neural network because you cannot average the weights of a ResNet with those of a MobileNet. However, if we focus on aligning features rather than weights, we can bypass this restriction.

In FedHARM, Anestis Kastellos demonstrates that by using a small shared dataset to create architecture-agnostic feature descriptors, diverse models can collaborate. Similarly, Chun-Yin Huang in DeSA uses synthetic anchors—lightweight, generated data samples—to allow completely different models to transfer knowledge and align their feature spaces without a central coordinator.

Why It Works: A Unified Coordinate System

The intuition behind this entire research direction is that aggregation is easier in a shared space.

By providing a global reference—whether it's a prototype (DCCFSSL), a fixed classifier (NCDSFL), or a shared public dataset (FedCA)—we ensure that the "meaning" of the features remains consistent across the network. This prevents the "vicious cycle" of model divergence, reduces communication costs by requiring fewer rounds to reach consensus, and allows federated systems to remain robust even when individual clients have highly skewed or incomplete views of the world.

Papers in this version

Showing 60 of 105 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_global_feature_anchors_for_robust_federa_ef3613de,
  title = {Global Feature Anchors for Robust Federated Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/global-feature-anchors-for-robust-federated-learning/rdr_ef3613de636a40b785a92a7c4454cdce},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_global_feature_anchors_for_robust_federa_ef3613de_v1,
  title = {Global Feature Anchors for Robust Federated Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/global-feature-anchors-for-robust-federated-learning/rdr_ef3613de636a40b785a92a7c4454cdce/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}