Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferring

Zhengquan Luo, Yunlong Wang, Zilei Wang, Zhenan Sun, Tieniu Tan

Introduction

Federated Learning (FL) enables multiple clients to collaboratively train models without sharing raw data. However, a significant challenge arises from "attributes skew"—a form of non-i.i.d. data where non-causal features, such as backgrounds or lighting, vary across clients. For example, a bird classifier trained on images of birds in blue skies (Client A) may fail when applied to birds on tree branches (Client B) because the model "entangles" the background with the object.

Conceptual overview of DFL: separating causal invariant features from domain-specific attributes to prevent negative transfer.

Standard FL frameworks aggregate entire models, causing locally valid but globally irrelevant attributes to pollute the global model. This leads to unstable convergence and poor generalization. To address this, the authors propose Disentangled Federated Learning (DFL).

The DFL Methodology

The core of DFL is the separation of model weights into invariant (causal) and domain-specific branches. This is achieved through three primary mechanisms:

  1. Two-Branch Architecture: Each client maintains an Invariant branch ($E_c$) to capture cross-domain causal features and a Specific branch ($E_s$) for local contextual information.
  2. Mutual Information (MI) Disentanglement: DFL employs MI constraints to enforce separation. It maximizes MI between local and global invariant branches to align causal features while minimizing MI between the invariant and specific branches to filter out domain-specific noise.
  3. Invariant Aggregation & Diversity Transferring: Only the invariant branches are aggregated globally. To prevent local overfitting, "Diversity Transferring" augments local representations with latent specific features shared by other clients.
The DFL framework architecture, illustrating the dual-branch extraction process and the MI-based optimization strategy.

Experimental Evaluation

The researchers evaluated DFL against state-of-the-art methods like FedAvg, FedProx, and FedGen across several datasets, including Colored-MNIST, 3Dshapes, Office-Caltech10, and DomainNet.

Key Results

  • Superior Accuracy: On Colored-MNIST, DFL achieved an accuracy of 95.91%, significantly outperforming FedAvg (88.88%) and FedProx (89.93%).
  • Efficient Convergence: DFL demonstrates faster convergence rates, reaching high accuracy in fewer communication rounds than competing methods.
  • Robustness and Scalability: The system's performance improves as the number of clients increases, benefiting from the richer variety of specific attributes available for diversity transferring.
  • Interpretability: Grad-CAM visualizations confirm that DFL’s invariant branch focuses precisely on the actual objects of interest rather than background elements.

Conclusion

DFL provides a robust solution to attribute skew in federated environments. By theoretically proving that partial aggregation of invariant branches is stable and practically demonstrating significant gains in accuracy and convergence, the authors offer a path toward more generalizable and interpretable federated models. Future work aims to scale DFL using frameworks like MindSpore to handle even larger, more complex datasets.

Summary updated

BibTeX
@inproceedings{luo2022disentangled,
  author = {Zhengquan Luo and Yunlong Wang and Zilei Wang and Zhenan Sun and Tieniu Tan},
  title = {{Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferring}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=HXVq0vBwPH},
}
Figure